Docs update for new features

Added runner pod spec docs
Added force-unlock docs

@chanwit any other notable use cases you would like me to add a doc for?

@tomhuang12, thank you so much.
Would it be possible for you to help write the doc for spec.storeReadablePlan, please?

@chanwit sorry I was on vacation last week. I will work on it on a separate PR.

No problem, @tomhuang12 :+1:


ARROW-10327: [Rust] [DataFusion] Replace iterator of batches by iterator of future batch

This PR is a proposal to change our iterators over RecordBatch to Future<RecordBatch>, thereby making our compute operations over a single batch as the "unit of work".
The rational here is that our expensive operations are over record batches, which are the ones that benefit from being split in smaller units (and multi-threaded).
This PR also places some tokio::spawn on some ops, with the expectation that the scheduler can multi-thread them.
The micro-benchmarks are not very indicative as this affects larger sizes, but as a rough idea, I get -50% improvement for larger math projections and +5% to +20% degradation for aggregations.
The aggregations have a rough implementation where a mutex blocks the whole calculation, which may explain the result.

Benchmarks
Math
sqrt_20_9               time:   [7.6861 ms 7.7328 ms 7.7805 ms]
                        change: [+20.288% +21.552% +22.732%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

Benchmarking sqrt_20_12: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 9.4s, enable flat sampling, or reduce sample count to 50.
sqrt_20_12              time:   [1.7891 ms 1.7954 ms 1.8020 ms]
                        change: [-38.768% -37.852% -36.578%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe

sqrt_22_12              time:   [8.4315 ms 8.5324 ms 8.6348 ms]
                        change: [-39.677% -38.299% -36.893%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

sqrt_22_14              time:   [11.515 ms 11.759 ms 12.054 ms]
                        change: [-48.307% -47.200% -45.854%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

Aggregates:
aggregate_query_no_group_by 15 12
                        time:   [831.70 us 836.65 us 842.22 us]
                        change: [+8.0888% +9.9290% +11.904%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  7 (7.00%) high severe

aggregate_query_group_by 15 12
                        time:   [5.9246 ms 5.9763 ms 6.0367 ms]
                        change: [+3.1496% +4.1417% +5.1472%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) high mild
  7 (7.00%) high severe

aggregate_query_group_by_with_filter 15 12
                        time:   [3.4054 ms 3.4322 ms 3.4597 ms]
                        change: [+26.844% +27.870% +28.979%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

If I'm understanding this correctly, this approach will mean that ordering is no longer deterministic and will cause problems for operations like SortExec which in turn will impact other future operators like SortAggregateExec and SortMergeJoinExec.

(at least currently) all batches are placed in a single batch before sorting.

join_all preserves the order:

The returned future will drive execution for all of its underlying futures, collecting the results into a destination Vec in the same order as they were provided.

@andygrove and @alamb , I agree with you: the other pattern is sufficient. Thanks a lot for the comments and for the teaching 💯

Thank you for doing the work to drive this forward -- it is super awesome
On Fri, Oct 16, 2020 at 11:19 AM Jorge Leitao<EMAIL_ADDRESS>wrote:

Closed #8480 https://github.com/apache/arrow/pull/8480.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/arrow/pull/8480#event-3886924131, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AADXZMKI7GQPWUZB6GX3K3DSLBP7JANCNFSM4STMP6CA
.


Library agnostic version

Hello. First of all, I love potok. It is by far the best system I have ever used for the application state management.
The only thing that bothers me a little is the RXJS dependency. As I moved to use shadow-cljs and npm version of the RXJS, it seems it makes my build quite big. It has to use rxjs-compat package, and I guess beicon is not optimising for this usage.
I believe beicon could be upgraded to use the latest version from the npm, but I am looking to some other reactive libraries (namely kefir), and think if it would be possible and feasible to create something based on the ideas from the potok, but using kefir. Even without the intermediate layer similar to beicon, where reactive primitives would be part of this "new" potok, so you have everything from one place.
I can understand, this is out of the scope for this project, and you are busy, but I still would love to hear, what you think about this kind of the approach.
Again thank you very much for all the work in your projects, they help make my job a lot easier.  Cheers.

Hi @pepe ! Nice to hear that!
About the beicon and rxjs. The problem here is that beicon uses a builitin version of rxjs and the latest release uses the 5.x branch of rxjs and includes a lot of unneded stuff (that is wrong). Right now im working on a beicon upgrade to the latest rxjs. That version will be also be bundled but the difference is that the bundled rxjs is google closure compiled and compatible so the dead code elimination should do much better work. ( The npm consuming stuff is not working fine at least for now, because i have cuerdas library that the latest version uses :npm-deps and many people have many problems with this version ).
About using different reactive primitives, I think thar is very feasible. Right now I don't know if it make sense think in somethink implemented inside potok, or just take the potok ideas and implement them in an other library... (potok is really really small)...
If you have anything in mind, I'm open to hear your ideas.

Thank you, for the answer!
I am using the latest rxjs from the npm without any problems (with rxjs-compat thou), but it is just the small set of the features and cljs only.
Which brings me to another angle, which just came to my mind, that I do not care about the JVM stuff at all, so only half of the beicon code is usable for me. On the other hand, I am also using rxhttp which have to be brought over.
I guess I will wait for the new version of the beicon and see how it works. If you think, there is anything I can help you with; I can try to do it.
Feel free to close this.


More robust patching of Ruler.measure

I recently tried to figure out why terrain-ruler was occasionally breaking for me, but only on Chrome and never on Firefox. The culprit ended up being that I had turned on javascript minification in Foundry and that Chrome was returning the minified source code for Ruler.measure, which was breaking the patching code. I also hit cases where the patching failed since Ruler had already been patched by drag-ruler.
This PR contains my attempt to make the patching code more robust to work in both cases. It does:

Change the logic for removing the function signature to not rely on newlines. Instead, it finds the first ) (assumed to belong to the function signature), and then the first { afterwards.
Change the search for segments, {gridSpaces to a regex which allows an arbitrary number of spaces in between the tokens.
Rather than just looking at the prototype of Ruler, traverse its prototype chain trying to find the original Foundry Ruler class. It attempts to patch any prototype that is called Ruler and has a measure methods, and stops once it is successful.

Minification in Foundry? I've never heard of that. Is that a v10 feature or am I missing something?
In any case, making the patching more robust is a good idea, and I'll have a look the next time I'm doing some work on one of my foundry modules :)

Hmm, maybe I've misattributed why the source code was minified. I've been using the felddy/foundryvtt-docker container and its FOUNDRY_MINIFY_STATIC_FILES variable. It seems to lead to minifyStaticFiles being set in options.json, which I can't really find any documentation on weirdly enough. When I was messing around in the javascript console the cause for my errors was definitely the minified source code, but maybe that was caused by something else.
In that case, my changes are probably not as necessary as I thought :) They should still handle the case of drag-ruler being loaded before terrain-ruler though and make the patching code less dependent on the source code formatting.

Embarassingly, the next time I'm doing some work on my modules is now. With the v10 compatible version, patching the measure function is no longer necessary. Thanks for your work though!


Fix Detekt for CouroutineTemplate

Issue
Detekt used to work for CoroutineTemplate, but something went wrong after merging this pull request. It's working correctly for RxJavaTemplate, we'll need to dig deeper in why it's not working for CoroutineTemplate and fix it accordingly.
Steps to reproduce

@Tuubz I will take a look on this issue.

@hoangnguyen92dn Awesome 🤩

@hoangnguyen92dn @Tuubz I just found that Detekt started to report NO SOURCE error from this PR https://github.com/nimblehq/android-templates/pull/132. Before that PR, it works flawlessly 🤪

As we're facing this issue on CI in the new project too, let me investigate and fix this instead of @hoangnguyen92dn 💪


Can't clone from github, always clone fails. (MEGH-3257)

Can't clone from github, always fails to clone, some can succeed, and some have been unable to clone.
For example: fatal: clone of 'https://github.com/espressif/json_parser.git'
The standard commands are used:
git clone --recursive https://github.com/espressif/esp-rainmaker.git

@Ocean100 , I see some gitclone.com paths in your logs. Are you using some caching logic via gitclone.com? The default repository  and submodules do not have gitclone.com URLs, so seems more like an environment issue to me.

@shahpiyushv
Using gitclone.com is an attempt to speed things up, actually the same result without this.
I tried using git clone --recursive git://github.com/espressif/esp-rainmaker.git
It is found that most files can be cloned normally, and some small files can also be cloned manually according to the prompts.
I don't quite understand the difference between https and git. I am trying to find the reason here.

The final reason: network problem, I have used someone else's computer remotely and can clone successfully.

Closing this issue since it has been resolved.


Multi-process isn't working

Hello guys, and thank you for your awesome library!
I'm currently struggling with making test_multi_process_translation.py script working. When it comes to the multi-process part, the following error occures:
2021-03-04 19:00:46 | INFO | easynmt.EasyNMT | Start multi-process pool on devices: cuda:0 Traceback (most recent call last):   File "test_multi_process_translation.py", line 80, in <module>     process_pool = model.start_multi_process_pool()   File "/data/home/k.kirillova/anaconda/envs/fairseq/lib/python3.7/site-packages/easynmt/EasyNMT.py", line 258, in start_multi_process_pool     p.start()   File "/data/home/k.kirillova/anaconda/envs/fairseq/lib/python3.7/multiprocessing/process.py", line 112, in start     self._popen = self._Popen(self)   File "/data/home/k.kirillova/anaconda/envs/fairseq/lib/python3.7/multiprocessing/context.py", line 284, in _Popen     return Popen(process_obj)   File "/data/home/k.kirillova/anaconda/envs/fairseq/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 32, in __init__     super().__init__(process_obj)   File "/data/home/k.kirillova/anaconda/envs/fairseq/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__     self._launch(process_obj)   File "/data/home/k.kirillova/anaconda/envs/fairseq/lib/python3.7/multiprocessing/popen_spawn_posix.py", line 47, in _launch     reduction.dump(process_obj, fp)   File "/data/home/k.kirillova/anaconda/envs/fairseq/lib/python3.7/multiprocessing/reduction.py", line 60, in dump     ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'BaseFairseqModel.make_generation_fast_.<locals>.train'
Could you please give me any ideas on how to fix this? Thanks in advance!

Same for me...

Will have a look. So far it appears to work the the opus-mt model, but the fairseq models throw this exception. Not sure why.
As soon as Transformers 4.4.0 is released, this project will get finally rid of the dependency of fairseq and will use transformers only. Then, the mbart50 and m2m model will also work in multi process.

Thanks a lot!

I got it working on Windows 10 in Docker and was able to convert the thing to work with 418m model.
I then tried to convert the model to work in AWS Lambda with Container support and got to the point that AWS lambda doesn't support multiprocessing. Without going to the depths and try myself to change the fairseq model loading implementation do you @nreimers  know whether there is easy option to load model without multiprocessing (Either now or once transformers 4.4.0 will be ready)?

@R4ZZ3
Does the traditional translate() method work for you?
The test_multi_process_translation.py  script makes only sense if you want to translate using multiple GPUs.

No,
It breaks down in the model loading step. I am trying to use the regular translate with some changes (works locally):
{
"errorMessage": "[Errno 38] Function not implemented",
"errorType": "OSError",
"stackTrace": [
"  File "/var/lang/lib/python3.8/imp.py", line 234, in load_module\n    return load_source(name, filename, file)\n",
"  File "/var/lang/lib/python3.8/imp.py", line 171, in load_source\n    module = _load(spec)\n",
"  File "", line 702, in _load\n",
"  File "", line 671, in _load_unlocked\n",
"  File "", line 783, in exec_module\n",
"  File "", line 219, in _call_with_frames_removed\n",
"  File "/var/task/app.py", line 30, in \n    model = EasyNMT('m2m_100_418M', cache_folder='/model')\n",
"  File "/var/lang/lib/python3.8/site-packages/easynmt/EasyNMT.py", line 86, in init\n    module_class = import_from_string(self.config['model_class'])\n",
"  File "/var/lang/lib/python3.8/site-packages/easynmt/util.py", line 56, in import_from_string\n    module = importlib.import_module(module_path)\n",
"  File "/var/lang/lib/python3.8/importlib/init.py", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n",
"  File "", line 1014, in _gcd_import\n",
"  File "", line 991, in _find_and_load\n",
"  File "", line 975, in _find_and_load_unlocked\n",
"  File "", line 671, in _load_unlocked\n",
"  File "", line 783, in exec_module\n",
"  File "", line 219, in _call_with_frames_removed\n",
"  File "/var/lang/lib/python3.8/site-packages/easynmt/models/Fairseq.py", line 12, in \n    from fairseq import checkpoint_utils, distributed_utils, tasks, utils\n",
"  File "/var/lang/lib/python3.8/site-packages/fairseq/init.py", line 24, in \n    import fairseq.pdb  # noqa\n",
"  File "/var/lang/lib/python3.8/site-packages/fairseq/pdb.py", line 16, in \n    _stdin_lock = multiprocessing.Lock()\n",
"  File "/var/lang/lib/python3.8/multiprocessing/context.py", line 68, in Lock\n    return Lock(ctx=self.get_context())\n",
"  File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 162, in init\n    SemLock.init(self, SEMAPHORE, 1, 1, ctx=ctx)\n",
"  File "/var/lang/lib/python3.8/multiprocessing/synchronize.py", line 57, in init\n    sl = self._semlock = _multiprocessing.SemLock(\n"
]
}

@R4ZZ3
Does the traditional translate() method work for you?
The test_multi_process_translation.py script makes only sense if you want to translate using multiple GPUs.

Could you explain, why it just makes sense to run the script using multiple GPUs? Is there no way to speed up the translation process by using multiple cpu cores?

@PanicButtonPressed
The transformer  model uses out-of-the-box already multiple cores. So it does not make a difference if you use 1 process that encodes on two cores (and e.g. can translate 100 sentences / minute) or two processes which fight for the compute power (and might only encode 40 sentences / minute each).
But for GPUs only 1 GPU is used per process. If you have 2 GPUs, each process can use an individual GPU.

Some background why the multiprocessing doesn't work:
https://aws.amazon.com/blogs/compute/parallel-processing-in-python-with-aws-lambda/

Tested all models, it works for me


Add support for jsonpath defaults

This PR includes two related changes to the JsonPath template helper:

output nothing (empty string) when no path matched instead of error message
allow overriding the default value when no path matched

The default value is static only. There may be a more generic solution to apply defaults for example by adding a "default" helper.
Gradle build didn't work for me on Oracle JDK 14 so I just ran a subset of the unit tests in Intellij.
Let me know if you want docs updated.

@tomakehurst Do you have any feedback on this?
The original discussion was at https://groups.google.com/g/wiremock-user/c/vg3PWVzmc1Q/m/gvOMzN3cBAAJ.

Sorry for the delay getting to this.
I'm going to merge but make a small change, which is to keep the original behaviour when there's no default specified and the JSONPath doesn't match anything. This is to avoid a breaking change. It'll still be possible to pass an empty string as the default if required.

Actually, scratch that. The current default behaviour is pretty misleading. I'll keep it your way.

Marvelous! Thank you :)


Autocomplete doesn't work with $ObjMap types

Autocomplete doesn't currently work with $ObjMap types.

This works now.


Install latest cosmwasm-check on CI

I found out locally that cosmwasm-check fails on cw1-whitelist binary with:
target/wasm32-unknown-unknown/release/cw1_whitelist.wasm: failure
Error during static Wasm validation: Wasm bytecode could not be deserialized. Deserialization error: "reference-types not enabled: zero byte expected (at offset 0x139a)"

This PR is for consideration to detect such cases by using the latest version of cosmwasm-check and to showcase the issue.

The error is due to Rust 1.82 enabled reference types by default, while cw-plus uses an older version. Closing this issue.


Vulnerability roundup 111: librecad-2.2.0-rc2: 2 advisories [8.8]

search, files

[ ] CVE-2021-45341 CVSSv3=8.8 (nixos-21.11, nixos-unstable)
[ ] CVE-2021-45342 CVSSv3=7.8 (nixos-21.11, nixos-unstable)

CVE details
CVE-2021-45341
A buffer overflow vulnerability in CDataMoji of the jwwlib component of LibreCAD 2.2.0-rc3 and older allows an attacker to achieve Remote Code Execution using a crafted JWW document.
CVE-2021-45342
A buffer overflow vulnerability in CDataList of the jwwlib component of LibreCAD 2.2.0-rc3 and older allows an attacker to achieve Remote Code Execution using a crafted JWW document.

Scanned versions: nixos-21.11: 2128d0aa28e; nixos-unstable: 5aaed40d22f.
Cc @Kiwi
Cc @viric

Fixed in 6896348d0fad85cfa3975d729b0279537981edfb and 57e6253b8c03e3bbf95e798085ee0fcf73fa6870.


v4.1.1 throws error while installing in yarn workspaces

Support reference number

none
Description

Same like https://github.com/sphereio/sphere-stock-import/issues/103
I use yarn as my package manager and i have a monorepo and use yarn workspaces
The project structure looks like this:
--package.json
--yarn.lock
--subfolder1
  |--package.json
--subfolder2
  |--package.json

So all the subfolders have a package.json but the lock file is only in the root. That's how yarn organizes workspaces.
If I add the sphere-node-product-csv-sync package in version 4.1.1 to the path subfolder1/package.json and run yarn install the following error occures on all enviroments (CI and locally): (version 4.1.0 works well)
yarn install
/usr/bin/yarn
yarn install v1.22.4
[1/4] Resolving packages...
[2/4] Fetching packages...
info<EMAIL_ADDRESS>The platform "linux" is incompatible with this module.
info<EMAIL_ADDRESS>is an optional dependency and failed compatibility check. Excluding it from installation.
info<EMAIL_ADDRESS>The platform "linux" is incompatible with this module.
info<EMAIL_ADDRESS>is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
error /home/runner/work/path/to/repo/node_modules/sphere-node-product-csv-sync: Command failed.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Exit code: 1
Command: npx npm-force-resolutions
Arguments:
Directory: /home/runner/work/path/to/repo/node_modules/sphere-node-product-csv-sync
Output:
npx: installed 5 in 2.142s
ENOENT: no such file or directory, open './package-lock.json'

Expected Behavior

Should work the same way as the version 4.1.0 without throwing an error while installing
Current Behavior

Throws error like written above.
Context

I just want to update from 4.1.0 to 4.1.1. Does not work with this error.
Possible Solution

Remove command npx npm-force-resolutions from preinstall script and maybe resolve the vulnerable dependencies directly in the package-lock.json?

Thanks again!
cc @VineetKumarKushwaha
(caused by PR https://github.com/sphereio/sphere-node-product-csv-sync/pull/293)

I can confirm that this issue also occurs when trying to use the library globally with NPM. Would be great if this can be picked up soon.

Released new patch version and verified by installing globaly with npm and yarn both.
@Tobi-mmt Please reopen if you still get error after updating to the latest.


Remove console "spam"

My stance on logging is that if there isn't any error, you should have a clean one.  HMR messages are still there but we'll disable them with a custom client I intend to implement as part of #89.


Retrieve compiled SQL query

Hello and thanks for this wonderful lib !
On my project, I use AWS S3 Select feature on CSVs stored in S3. It allows executing a SQL-like query directly on those CSVs:
import { S3Client, SelectObjectContentCommand } from '@aws-sdk/client-s3';

const s3Client = new s3Client()

const Response = await s3Client.send(
  new SelectObjectContentCommand({
    ... // some serialization options
    ExpressionType: 'SQL',
    Expression: `
      SELECT ...
      FROM S3Object
      WHERE ...
    `
  })
);

I would really like using kysely to build such queries but for the moment I cannot:

I haven't found a way to simply retrieve the compiled SQL query string to provide it to the S3Client
The Kysely class constructor requires a dialect whereas I don't really need one (other solution is to build an adapter for S3Select but that's a whole lotta work)

Would it be possible to make dialect optional in the constructor (and throw an error if none is found when needed) and expose a compileQuery method on the db to retrieve the compiled query ?

Hey 👋
You can brew a dialect with existing built-in components and replace the driver with built-in DummyDriver.
This is in the browser example.

Hi @igalklebanov !
Thanks for the response, I missed the DummyDriver class :)
I had seen the compile method but it still requires to hydrate the parameters in the sql query. Is there a way to not have to do that ? Small but it's always nice to reduce verbosity and custom code.
Still, I'll give it a try ! Thanks again 👍

Normally you should never "hydrate" parameters to the SQL string. That's a sure way to get your system pwned. For that reason, there's no built in way to do it.

I see, high risk of SQL injection there.
Sadly there is no Parameters attribute in the S3Select interface, so I'll have to create a dangerouslyHydrateParameters helper to do that 🤷♂ Many thanks !

I see, high risk of SQL injection there.
Sadly there is no Parameters attribute in the S3Select interface, so I'll have to create a dangerouslyHydrateParameters helper to do that man_shrugging Many thanks !

Can you share your helper?

Sadly there is no Parameters attribute in the S3Select interface

Yeah, I understand that in this case it's necessary. I was just explaining why a helper like that won't be added to Kysely.

Can you share your helper?

const parameters = ['foo', 'bar']
'select $1 from $2'.replace(/(\$\d+)/g, (arg) => parameters[parseInt(arg.slice(1))-1])

@wirekang I went for this:
const dangerouslyHydrateSQLParameters = (
  sql: string,
  parameters: readonly unknown[],
): string => {
  for (const parameter of parameters) {
    sql = sql.replace('?', `'${String(parameter)}'`);
  }

  return sql;
};

@ThomasAribart Got inspired and implement this based on your blog post. 🤗

@igalklebanov Awesome ! 🤩


Pool Token can only be transferred to verified lenders

Codecov Report

Merging #140 (676bf36) into master (a3b35fd) will decrease coverage by 0.00%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##           master     #140      +/-   ##
==========================================
- Coverage   78.39%   78.38%   -0.01%
==========================================
  Files          16       16
  Lines        1685     1689       +4
  Branches      309      311       +2
==========================================
+ Hits         1321     1324       +3
- Misses        364      365       +1

Impacted Files
Coverage Δ

contracts/Pool/Pool.sol
94.23% <80.00%> (-0.25%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3b35fd...676bf36. Read the comment docs.


Makefile: Use $(go env GOPATH) instead of environment variable

This avoids the Makefile trying to install golangci-lint to /bin/
which the user may not have write permission to. And by default, the
user may not have manually configured a GOPATH environment variable,
but running $(go env GOPATH) can still return a useful path (such as
$HOME/go/bin).
Fixes #444

Hi @cworth-gh , thanks for your contribution, looking forward to merging it :+1:

Hi @cworth-gh , thanks for your contribution, looking forward to merging it :+1:

Changed as per discussion and pushed in a1ebb447


ocw-www: about page

As a user, I'd like to find out more about OCW
Designs and Mockups
desktop: https://invis.io/N2121HAUUW3J#/461217843_About_Us
mobile: https://invis.io/FU1226TX74CV#/461387536_Mobile_-About_Us-Subnav-_Scroll
mobile timeline: https://invis.io/65123QVGCNPG#/459413988_Mobile_Timeline
copy: https://docs.google.com/document/d/1NTIIJ_q0Vqyv9ZUFU66sVBM9n9EfJcTxct3mJMseDWs/edit
Acceptance Criteria:

[ ] for now, this should all be part of the ocw-www theme. In the future, we may consider exposing some of it through studio.
[ ] As much as possible, styles should follow existing styles (e.g. fonts & colors)
[ ] One long, scrolling page
[ ] on-page nav should stay sticky, at top of viewport when user scrolls down (mobile UX TBD)
[ ] clicking on item in on-page nav should smooth-scroll down to the appropriate section, not jump.
[ ] SRC images TBD. For now grab what you can from InVision
[ ] video at https://www.youtube.com/watch?v=0aAEamhJHUI
[ ] "Chalk Radio Podcast" can be an embed  from SimpleCast: <iframe height="200px" width="100%" frameborder="no" scrolling="no" seamless src="https://player.simplecast.com/a9349695-05d2-4150-bba6-271cbf875ba0?dark=false"></iframe>
[ ] For the OCW Staff section, copy names, titles and photos from https://openlearning.mit.edu/our-team?term_node_tid_depth[72]=72
[ ] Update the ocw-www home page to link to link "Learn More about the OCW Mission" to this page
[ ] Update the site header (ocw-www and ocw-courses) to link "About OCW" to this page.

Out of Scope

Omit "Educators Start Here" section and button until #issue is unblocked
Omit "Find Courses with Instructor Insights" section & button until #issue is unblocked
Omit "Open Textbooks" section & button until #279 is unblocked
Omit "Open Learning Library" section & button until #280 unblocked

@pdpinch What is the expected URL here?  is /about okay or did we want /about-us?

@pdpinch The page contains several links in the "milestones" section.  I can assume where some of these links go but it would be best to have specific URLs for each link either here or in the design.

Best to keep it the same as the current site, /about
@pdpinchhttps://github.com/pdpinch What is the expected URL here? is /about okay or did we want /about-us?

Sorry, they should have been in the Google doc. But rather than delay things with busy work, you can scrape the URLs from https://ocw.mit.edu/about/milestones/

@pdpinch https://github.com/pdpinch The page contains several links in the "milestones" section. I can assume where some of these links go but it would be best to have specific URLs for each link either here or in the design.

@pdpinch Thanks, I didn't notice they were in the Doc.  I should have checked there first.


Add clearer details on a personal desktop

When you select a personal desktop rather than a pooled desktop what number of desktops should be selected to create within the host pool? If you want to create 10 personal desktops, should you add the 10 users as the comma separated list and then select 10 vm's to be created?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 970202b7-165f-18df-e928-c47c45a18755
Version Independent ID: d5393b62-4211-72f5-fc7a-363f68f6d8a2
Content: Create a Windows Virtual Desktop host pool by using the Azure Marketplace - Azure
Content Source: articles/virtual-desktop/create-host-pools-azure-marketplace.md
Service: virtual-desktop
GitHub Login: @Heidilohr
Microsoft Alias: helohr

Thanks for the question. We are currently investigating and will update you shortly.

We actually support Windows Virtual Desktop in Tech Communities
https://techcommunity.microsoft.com/t5/Windows-Virtual-Desktop/bd-p/WindowsVirtualDesktop
@ChristianMontoya Can you please share your insights/inputs here. Thanks :)

@gerrynicol : You do not necessarily have to list all users there in the UI. You can add users later. However, you should create the number of VMs that you'd like to have (so all 10).

Thanks @ChristianMontoya :)
@gerrynicol We will now close this issue. If there are further questions regarding this matter, please tag me in a comment. I will reopen it and we will gladly continue the discussion.


Slashing

Slashing should be implemented based on the set of slashing conditions. Ideally there should be E2E tests that verify that slashing functions correctly (e.g., using the byzantine node to invoke incorrect behavior).
Slashing conditions

[ ] Transaction scheduler committee

[ ] Leader

[ ] Create/send a schedule of batches with R/W conflict
[ ] Create/send a schedule of batches with W/W conflict
[ ] Do not create/send a schedule
[ ] Send different batches to different compute committee members
[ ] Create/send a schedule of batches where multiple compute committees will emit roothash messages

[ ] Verifier

[ ] Sign a schedule of batches with R/W conflict
[ ] Sign a schedule of batches with W/W conflict
[ ] Do not sign a valid schedule

[ ] Compute committee

[ ] Compute/sign/send a wrong result
[ ] Not revert when transaction violates the R/W set
[ ] Not send the result to merge committee
[ ] Send different results to different merge committee members

[ ] Merge committee

[ ] Merge storage roots incorrectly
[ ] Not send the result to consensus committee

[ ] Storage committee

[ ] Return a wrong result to read request
[ ] Return an invalid merged root
[ ] Longer-term loss of availability

[x] Consensus committee

[x] Signing different votes for the same height

All runtime-related slashing conditions (and thresholds to run nodes) should probably be configurable per-runtime.

In addition to slashing, misbehaving nodes should also be frozen (similar to how validators are frozen for double signing), with freezing maybe being configurable per runtime (maybe in addition to there being global parameters?).


Cannot connect to WLST : weblogic stop in k8s pod fails. (FMW 122130)

Hi Team,
Need help in k8s regarding FMW 122130, applied both the patches p27117282_122130_Generic.zip and p29135930_122130_Generic.zip as mentioned in the readme
Using the available FMW 122130 docker file, I have built and create a new container, where admin-server is in running status, same is accessible from outside the machine as well. Able to deploy the application as well and able to connect from other machines as well.
But in k8s it fails.
I created a yaml for deployment and created a service with type as NodePort and when I tried to connect to an admin-server console outside the machine it is successfully connecting using the node port.
but, when I tried to deploy the application, is not able to connect to the Weblogic server and when i connect to the pod and when I try to stop the weblogic I m getting below error.
Error:
[oracle@dom-pdc-two-77df786599-qsq8c bin]$ ./stopWebLogic.sh
Stopping Weblogic Server...
Initializing WebLogic Scripting Tool (WLST) ...
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands
Connecting to t3://dom-pdc-two-77df786599-qsq8c:9771 with userid  ...
WLST detected that the RuntimeMBeanServer is not enabled. This
might happen if the RuntimeMBeanServer is disabled via the JMXMBean.
Please ensure that this MBeanServer is enabled. Online WLST cannot
function without this MBeanServer.
This Exception occurred at Thu Jun 06 20:13:53 UTC 2019.
java.lang.SecurityException: Anonymous attempt to get to a JNDI resource
Problem invoking WLST - Traceback (innermost last):
File "/u01/oracle/user_projects/domains/pdc_k8s_1_/shutdown-AdminServer.py", line 3, in ?
File "", line 19, in connect
File "", line 553, in raiseWLSTException
WLSTException: Error occurred while performing connect : "Cannot connect to WLST." : Anonymous attempt to get to a JNDI resource
Use dumpStack() to view the full stacktrace :
Done
Stopping Derby Server...
Derby server stopped.
[oracle@dom-pdc-two-77df786599-qsq8c bin]$
Folder structure and its permission:
[oracle@dom-pdc-two-77df786599-qsq8c bin]$ ls -lrt /u01
total 0
drwxr-xr-x 1 root   root    24 Jun  6 18:17 java
drwxr-xr-x 1 root   root    22 Jun  6 18:17 perl
drwxr-xr-x 1 oracle oradba  36 Jun  6 18:22 db_client
drwxr-xr-x 1 oracle oracle 438 Jun  6 19:32 oracle
Oracle is the folder where FMW is installed and user domain is created under user_projects.
I also created below role but didnt work.
kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
Kindly help me.
Thanks,
Meena

Deployment yaml:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: dom-pdc-two
spec:
replicas: 1
template:
metadata:
labels:
app: dom-pdc-two
spec:
# hostNetwork: true
containers:
- name: dom-pdc-two
image: pdcapp:<IP_ADDRESS>.0
command: ["sh"]
args: ["/u02/container-scripts/installOrStartPdc.sh"]
ports:
- containerPort: 9771
name: pdc-http-port

Service yaml
apiVersion: v1
kind: Service
metadata:
name: dom-pdc-two
labels:
app: dom-pdc-two
spec:
type: NodePort
selector:
app: dom-pdc-two
ports:
- name: pdc-http-port
protocol: TCP
port: 9771
targetPort: 9771

And also /u01 is root:root permission
but folders under /u01 are oracle:oracle permission

Hi @meenaal, we just released operator version 2.2.0 today, which is the first operator version to work with FMW infrastructure domains.  Are you using this newest version?  Documentation for working with FMW infrastructure domains is here: https://oracle.github.io/weblogic-kubernetes-operator/userguide/managing-domains/fmw-infra/

I followed
https://github.com/oracle/docker-images/tree/master/OracleFMWInfrastructure

Ah, okay.  The docker-images project has instructions for running in stand-alone Docker containers.  This project is for an operator that assists with running WebLogic in Kubernetes.  I would suggest that in order to run in Kubernetes, you use our operator and the documentation I linked to above.

Really appreciate. But my task is to deliver both domain and application related docker and k8s files.
So I need to fix the issue.

Yes, that's what we have in this project.  The operator is a controller for a custom resource type, Domain, representing a WebLogic domain.  Based on the Domain, the operator creates the necessary Pods and Services to run WebLogic server instances.
I'm not sure how to help on your issue, as we've never tested running the FMW infrastructure images in K8s without the operator.

Would you consider coming to our public Slack channel?  It might be more efficient to get more people involved in the conversation.  It is here: https://weblogic-slack-inviter.herokuapp.com/

Thanks, I m part of CGU unit, as part of private cloud initiative we are asked to deliver the docker files and k8s file for FMW Infra. with AdminServer domain and deploying CGBU application. With Docker, I didn't face any issue, it is working fine, only in k8s t3 is not connecting when I tried to stop the server and I m not able access WebLogic server using localhost as well. I will post the same in the slack as well.

Closing this as we are discussing the requirements separately and the use case described in the issue is not using the operator.


[diem-framework] Add specifications to DiemId module

This PR adds specifications to the DiemId module, as well as updates the documentation of the new scripts to contain additional abort categories and reasons.
Testing
Spec only changes (and two asserts). So no new tests.

/land


Create Task to Add Dislike Activity to Logged-In User

GitHub Issue: Create Task to Add Dislike Activity to Logged-In User
Objective: Develop a task that adds a dislike activity to the logged-in user's activity feed. This task should run automatically after a dislike is created successfully for an article.
Description:
Tasks:

Create Dislike Activity Task:

Develop a task named AddDislikeActivity that accepts the articleId as a parameter.
This task will be responsible for adding a dislike activity to the logged-in user's activity feed.

Task Execution Logic:

Within the AddDislikeActivity task, implement the logic to identify the article for which the dislike was created and the logged-in user.
Add the dislike activity to the logged-in user's activity feed.

Integration with Dislike Creation:

Integrate the AddDislikeActivity task with the dislike creation process so that it runs automatically after a dislike is created successfully for an article.

Error Handling:

Implement error handling within the task to handle any exceptions or issues that may occur during the addition of the dislike activity.

Acceptance Criteria:

The AddDislikeActivity task is implemented and accepts the articleId as a parameter.
The task identifies the article for which the dislike was created and the logged-in user.
A dislike activity is added to the logged-in user's activity feed after a dislike is created successfully for an article.
The task runs automatically after a dislike is created.
Proper error handling is in place to address any exceptions or issues during the addition of the dislike activity.

i would like to work on this

Acceptance Criteria:

Please read the Acceptance Criteria: carefully before submitting the PR. Thanks!


OpenSCD incorrectly handles namespaces in new project with import

Describe the bug
I did the following:

Create a new project
Add an icd file
Save and look at the output

The output looks like:
<SCL xmlns="http://www.iec.ch/61850/2003/SCL" version="2007" revision="B">

	<Communication>
		<SubNetwork name="W01">
			<ConnectedAP iedName="XAT_SEL_411L_2" apName="S1">
				<Address>

<IP_ADDRESS>

<IP_ADDRESS>

<IP_ADDRESS>

0001

00000001

0001

				</Address>
				<GSE ldInst="CFG" cbName="Ctl">
					<Address>

01-0C-CD-01-00-00

0000

4

3E8

					</Address>
					<MinTime unit="s" multiplier="m">4</MinTime>
					<MaxTime unit="s" multiplier="m">1000</MaxTime>
				</GSE>
			</ConnectedAP>
		</SubNetwork>
	</Communication>
	<IED desc="ICD-411L-2S-R001-V0 for firmware R200-V0 or higher" name="XAT_SEL_411L_2" type="SEL_411L_2S" manufacturer="SEL" configVersion="ICD-411L-2S-R001-V0-Z200006-D20210701" originalSclVersion="2007" originalSclRevision="B">
		<Private type="SEL_IedInfo">
			<esel:ModelNumber xmlns:esel="http://www.selinc.com/2006/61850">SEL-411L-2S</esel:ModelNumber>
			<esel:ModelVersionMin xmlns:esel="http://www.selinc.com/2006/61850">R200</esel:ModelVersionMin>
			<esel:ClassFileVersion xmlns:esel="http://www.selinc.com/2006/61850">006</esel:ClassFileVersion>
			<esel:ClassFileDescription xmlns:esel="http://www.selinc.com/2006/61850" default="false">411L-2S - Sampled Values Subscriber R200-V0 or higher (Ed. 2 support, Controllable mod/beh)</esel:ClassFileDescription>
			<esel:IcdFilePath xmlns:esel="http://www.selinc.com/2006/61850">C:\Users\tilbrookm\AppData\Local\SEL\AcSELerator\Architect\Palette\SEL-411L-2_Template.CID</esel:IcdFilePath>
			<esel:CidSentTime xmlns:esel="http://www.selinc.com/2006/61850">07/20/2022 13:35:44</esel:CidSentTime>
		</Private>

Namespace declarations appear to be incorrectly.

Each element within the esel namespace has a namespace declaration on the element.
The SCL element does not have a namespace declaration for esel

This is on openscd.github.
I expect the namespace to be registered on the root element and not repeated.
The initial icd file appears correct:
<?xml version="1.0" encoding="UTF-8"?>
<SCL xmlns:esel="http://www.selinc.com/2006/61850" version="2007" revision="B" xmlns="http://www.iec.ch/61850/2003/SCL">
  <Header id="XAT_Prot1" version="32" revision="1.0" toolID="AcSELerator Architect <IP_ADDRESS>4" nameStructure="IEDName">
    <History>
      <Hitem version="32" revision="1.0" when="08/19/2022 13:51:55" who="SANC\tilbrookm" what="IED exported from Project XAT_Prot1" />
    </History>

  <Communication>
    <SubNetwork name="W01">
      <ConnectedAP iedName="XAT_SEL_411L_2" apName="S1">
        <Address>

<IP_ADDRESS>

<IP_ADDRESS>

<IP_ADDRESS>

0001

00000001

0001

        </Address>
        <GSE ldInst="CFG" cbName="Ctl">
          <Address>

01-0C-CD-01-00-00

0000

4

3E8

          </Address>
          <MinTime unit="s" multiplier="m">4</MinTime>
          <MaxTime unit="s" multiplier="m">1000</MaxTime>
        </GSE>
      </ConnectedAP>
    </SubNetwork>
  </Communication>
  <IED desc="ICD-411L-2S-R001-V0 for firmware R200-V0 or higher" name="XAT_SEL_411L_2" type="SEL_411L_2S" manufacturer="SEL" configVersion="ICD-411L-2S-R001-V0-Z200006-D20210701" originalSclVersion="2007" originalSclRevision="B">
    <Private type="SEL_IedInfo">
      <esel:ModelNumber>SEL-411L-2S</esel:ModelNumber>
      <esel:ModelVersionMin>R200</esel:ModelVersionMin>
      <esel:ClassFileVersion>006</esel:ClassFileVersion>
      <esel:ClassFileDescription default="false">411L-2S - Sampled Values Subscriber R200-V0 or higher (Ed. 2 support, Controllable mod/beh)</esel:ClassFileDescription>
      <esel:IcdFilePath>C:\Users\tilbrookm\AppData\Local\SEL\AcSELerator\Architect\Palette\SEL-411L-2_Template.CID</esel:IcdFilePath>
      <esel:CidSentTime>07/20/2022 13:35:44</esel:CidSentTime>

To Reproduce
XAT_SEL-411L-2.ICD.zip
Steps to reproduce the behavior:

Using the attached icd file.
Create a new Edition 2 project.
Import the ICD file.
Save the project.
Open the scd file and examined the first 40 lines.
Observe that namespaces are incorrectly handled.

I think this must have happened recently...

No, not recently, at least 40 commits old.

FWIW I discovered this because xmldiff barfed at me when comparing some files:
14:47 $ xmldiff XAT_Prot1_MAC_Updated_SEL_Sub.scd Test_From_ICD_SEL_NR_Initial.scd
Traceback (most recent call last):
  File "/usr/bin/xmldiff", line 33, in <module>
    sys.exit(load_entry_point('xmldiff==2.4', 'console_scripts', 'xmldiff')())
  File "/usr/lib/python3/dist-packages/xmldiff/main.py", line 116, in diff_command
    result = diff_files(args.file1, args.file2, diff_options=diff_options,
  File "/usr/lib/python3/dist-packages/xmldiff/main.py", line 50, in diff_files
    return _diff(etree.parse, left, right,
  File "/usr/lib/python3/dist-packages/xmldiff/main.py", line 37, in _diff
    right_tree = parse_method(right, parser)
  File "src/lxml/etree.pyx", line 3536, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1897, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1917, in lxml.etree._parseFilelikeDocument
  File "src/lxml/parser.pxi", line 1811, in lxml.etree._parseDocFromFilelike
  File "src/lxml/parser.pxi", line 1201, in lxml.etree._BaseParser._parseDocFromFilelike
  File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
  File "/home/mulhollandd/Documents/open-scd-testing/Prot1_ICD_Files/Test_From_ICD_SEL_NR_Initial.scd", line 51900
lxml.etree.XMLSyntaxError: Attribute xmlns:esel redefined, line 51900, column 146

Hmmmm I think this may just be a feature of the way OpenSCD serialises the XML and may be quite valid (?).
MDN suggests this if I understand correctly:

For XML serializations, Element and Attr nodes are always serialized with their namespaceURI intact. This may mean that a previously-specified prefix or default namespace may be dropped or altered.

In which case perhaps this can/should be closed.

We discussed this today, and it was suggested that if the namespaces are transferred from the imported document first to the project document and then the nodes are cloned, the namespaces should not be repeated on the imported elements.
The behaviour at the moment is "safe" because it avoids namespace conflicts by only applying the namespace at the point at which it is required (thanks @dlabordus).
I thought I had tried this but was unsuccessful with this approach, but I will provide further information soon.
We agreed the ideally the namespaces would be declared on the root node so it seems the issue remains relevant.

@dlabordus thanks for the discussion this evening (and FYI @ca-d).
I have opened (a draft PR not for merging) in #1080.
The output from running this with the attached is something like this, which is not what we hoped would occur.
Did I get my order of operations incorrect? All gentle mockery gratefully accepted :smile_cat:
My steps are:

Create new project.
Import from the attached ICD file.
Save project.
Inspect XML in editor. Arrrgh! The worst of both worlds. :angry: :shrug:

<SCL xmlns="http://www.iec.ch/61850/2003/SCL" version="2007" revision="B" release="4" xmlns:esel="http://www.selinc.com/2006/61850">

	<Communication>
		<SubNetwork name="W01">
			<ConnectedAP iedName="XAT_SEL_411L_2" apName="S1">
				<Address>

<IP_ADDRESS>

<IP_ADDRESS>

<IP_ADDRESS>

0001

00000001

0001

				</Address>
				<GSE ldInst="CFG" cbName="Ctl">
					<Address>

01-0C-CD-01-00-00

0000

4

3E8

					</Address>
					<MinTime unit="s" multiplier="m">4</MinTime>
					<MaxTime unit="s" multiplier="m">1000</MaxTime>
				</GSE>
			</ConnectedAP>
		</SubNetwork>
	</Communication>
	<IED desc="ICD-411L-2S-R001-V0 for firmware R200-V0 or higher" name="XAT_SEL_411L_2" type="SEL_411L_2S" manufacturer="SEL" configVersion="ICD-411L-2S-R001-V0-Z200006-D20210701" originalSclVersion="2007" originalSclRevision="B">
		<Private type="SEL_IedInfo">
			<esel:ModelNumber xmlns:esel="http://www.selinc.com/2006/61850">SEL-411L-2S</esel:ModelNumber>
			<esel:ModelVersionMin xmlns:esel="http://www.selinc.com/2006/61850">R200</esel:ModelVersionMin>
			<esel:ClassFileVersion xmlns:esel="http://www.selinc.com/2006/61850">006</esel:ClassFileVersion>

See attached:
XAT_SEL-411L-2.ICD.zip
I notice we call cloneNode and not importNode as MDN suggests and I do wonder why but perhaps that is a distraction.


enh: add trace logs

Codecov Report
Attention: Patch coverage is 63.01565% with 520 lines in your changes missing coverage. Please review.

Project coverage is 69.73%. Comparing base (88035cb) to head (ce236e1).

Files
Patch %
Lines

controller/ws/tmq/tmq.go
57.08%
178 Missing and 28 partials :warning:

controller/ws/ws/handler.go
64.26%
133 Missing and 1 partial :warning:

controller/ws/query/ws.go
58.46%
75 Missing and 1 partial :warning:

controller/rest/restful.go
59.50%
47 Missing and 2 partials :warning:

db/async/row.go
80.32%
12 Missing :warning:

controller/rest/error_resp.go
63.15%
7 Missing :warning:

db/tool/notify.go
0.00%
7 Missing :warning:

controller/rest/auth.go
14.28%
6 Missing :warning:

controller/rest/table_vgid.go
25.00%
6 Missing :warning:

controller/ws/schemaless/schemaless.go
76.47%
4 Missing :warning:

... and 6 more

Additional details and impacted files
@@            Coverage Diff             @@
##              3.0     #332      +/-   ##
==========================================
- Coverage   70.03%   69.73%   -0.31%
==========================================
  Files          85       85
  Lines        9883    10463     +580
==========================================
+ Hits         6922     7296     +374
- Misses       2280     2481     +201
- Partials      681      686       +5

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Adding feathub link for feedback.

Got Feathub set up after Andrew found it on Hacker News. Just adding the link to the Feedback Form on the ReadMe.

Coverage remained the same at 76.92% when pulling 47e2543dc6e39c9b147e033319931c0372f71c73 on steph-test into ea77ffecb66dfa3a64a8ce4e4ef06c26bbb6e971 on master.

:+1:


Potential matplotlib issue

In [31]: matplotlib.__version__
Out[31]: '2.0.0b4'

multiple scans in a LivePlot and tried to scale with l
In [30]: Traceback (most recent call last):
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/backends/backend_qt5.py", line 314, in keyPressEvent
    FigureCanvasBase.key_press_event(self, key, guiEvent=event)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/backend_bases.py", line 1846, in key_press_event
    self.callbacks.process(s, event)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/cbook.py", line 550, in process
    proxy(*args, **kwargs)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/cbook.py", line 417, in __call__
    return mtd(*args, **kwargs)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/backend_bases.py", line 2651, in key_press
    key_press_handler(event, self.canvas, self.canvas.toolbar)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/backend_bases.py", line 2555, in key_press_handler
    ax.figure.canvas.draw()
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 162, in draw
    FigureCanvasAgg.draw(self)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/backends/backend_agg.py", line 463, in draw
    self.figure.draw(self.renderer)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/figure.py", line 1129, in draw
    renderer, self, dsu, self.suppressComposite)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/axes/_base.py", line 2384, in draw
    mimage._draw_list_compositing_images(renderer, self, dsu)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/image.py", line 139, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/axis.py", line 1126, in draw
    ticks_to_draw = self._update_ticks(renderer)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/axis.py", line 959, in _update_ticks
    tick_tups = [t for t in self.iter_ticks()]
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/axis.py", line 959, in <listcomp>
    tick_tups = [t for t in self.iter_ticks()]
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/axis.py", line 904, in iter_ticks
    self.major.formatter.set_locs(majorLocs)
  File "/home/xf23id1/conda_envs/collection/lib/python3.5/site-packages/matplotlib/ticker.py", line 839, in set_locs
    vmin = math.log(vmin) / math.log(b)
ValueError: math domain error

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at<EMAIL_ADDRESS>
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    %config Application.verbose_crash=True


Update mache references

Update mache references. Resolves #329.

Going to merge these changes, to resolve #329. The deletion of Cori-specific code and addition of Perlmutter-specific code will be done as part of #310.


latency issue?

Hi,
I've been looking for a solution to do remote testing on rpi. Like QT applications run on my PI and just trying to display it back on my local computer.
Both the RPI and my PC are on the same local, fast network.
In general telepi works great. However I do seem to have latency issue (using netcat with mplayer as well as vlc way). The longer it streams the bigger the latency. I could live with sub 10sec latency but from my experience the latency I get is 30sec and increases all the time.
Do you have a way around it?

I've not looked at the code for a few years but I might consider investigating some improvements after the end of the month. If the latency is increasing over time it could be that the display frame rate does not match the encoding frame rate causing it to drift.

So when I use VLC I am getting many warnings:
[009ba400] mux_ts mux debug: adjusting rate at -189605/213366 (95/1) [009ba400] mux_ts mux warning: packet with too strange dts on pid 100 (h264)(dts=43395664149,old=43395955067,pcr=43395955067)
Every time it happens it looks like the latency increases.
Here is the full log: https://pastebin.com/crC6hhYJ
I tried to look this up but the only explanation I could understand is between timing issue or encoding issue. This made me think - can we use MJPEG instead of AVC?


Problem Retrieving Data WinForms App

Hi,
I just want to say this API is great!  I have been playing around with it in a vb.net console app and it's working perfect.
However, in my vb.net winforms app I'm having trouble retrieving data. It shows the client logged in as true, has saved tokens, etc. When I try to retrieve a simple quote it just freezes, unlike the console app. It looks like the json body is null. Am I missing something simple?
Thanks!
John

Hi John,
Thanks. TD has really poor error messages. If you’re getting a null json that means you probably sent a request to the server that TDA didn’t like. I’ve never seen the problem in c#. Try using the tester project to see if it’s working there before trying it in your winforms app.
Also you’re not the first person trying to build an app with it. I’m in the works on building a TDA plugin for QuantConnect/Lean on GitHub. Lean might be a good solution for what you are trying to do.
Brian

From: Cheytac011 @.>
Sent: Sunday, December 19, 2021 3:59:48 PM
To: bmello4688/TDAmeritradeApi @.>
Cc: Subscribed @.***>
Subject: [bmello4688/TDAmeritradeApi] Problem Retrieving Data WinForms App (Issue #7)
Hi,
I just want to say this API is great! I have been playing around with it in a vb.net console app and it's working perfect.
However, in my winforms app I'm having trouble retrieving data. It shows the client logged in as true, has saved tokens, etc. When I try to retrieve a simple quote it just freezes, unlike the console app. It looks like the json body is null. Am I missing something simple?
Thanks!
John
—
Reply to this email directly, view it on GitHubhttps://github.com/bmello4688/TDAmeritradeApi/issues/7, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB3WTQTIWAYUBXFPOECYSL3URZBUJANCNFSM5KMJFZNQ.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

Hi Brian,
The tester console app works perfect for me in vb.net. No problems at all. After I enter the SMS code in my vb.net winforms application it shows the client being logged in etc. However, even a simple line of code like this causes the application to freeze.
dim quote = client.MarketDataApi.GetQuote("MSFT").Result
I'm just not sure why it works in the vb console and not vb winforms. The only difference is I retrieve the SMS code via a textbox from a dialogresult. After I enter the SMS the rest of the code executes after the await login. I have seen another gentleman who was using your API in a winforms app just browsing through some old issues so not sure what I'm doing wrong.
John


Modify fan speed via HASS-miner

There have been some requests to be able to control fixed fan speed via HASS-miner.  After the merge of #39, this requires:

A switch entity to enable or disable manual fan control,
and a number entity to set manual fan speed.

This would then have to call miner.get_config() on update, then depending on switch status:

Set cfg.temp_mode to "manual"/"disabled" or "auto" depending on the switch,
and set cfg.fan_speed to an integer between 1 and 100.

This would be brilliant! Was looking for exactly this. Would be great if it will be integrated

Yes, it would be excellent if we could control the fan speed from the integration. Or at least throttle them down to a low run when the BOS is stopped. This would significantly reduce the power consumption of the machine. As for power management, it would also be excellent if it were possible to control (on/off) individual hash boards. This would manage the power consumption perhaps better than if there is only the possibility to set 2450 to 3200VA range (with a standard power supply).
Anyway, thank you Schnizel for a great job. I used to control the Miner S19 PRO remotely only by switching the contactor. I have destroyed 3pcs APW power supply in two years, because the switching power supply probably does not tolerate frequent shocks when switched on hard.

This may complicate things somewhat.  I'm not sure setting fan speed on shutdown is possible, I know it can be set to the minimum possible speed by disabling bosminer_monitor, but that should be done 3-5 minutes after shutdown to prevent frying any boards.  Not sure how any other firmwares would handle this either; both Whatsminer and Antminer stock firmware can be shut down, and whatsminers can be tuned, but not sure how they will react with manually setting 0 speed when in shutdown mode.
Regarding disabling boards, that might be something we could do automatically?  Assume each board draws a minimum of 800W (2400/3), then shut down boards at 2400W and 1600W, although I don't know how BOS will react to this, another thing I've not tested myself.  It's possible it will try to use 2400W on that single board, but it's also possible it will split it up evenly across all boards, and ignore the disabled one.

This may complicate things somewhat. I'm not sure setting fan speed on shutdown is possible, I know it can be set to the minimum possible speed by disabling bosminer_monitor, but that should be done 3-5 minutes after shutdown to prevent frying any boards. Not sure how any other firmwares would handle this either; both Whatsminer and Antminer stock firmware can be shut down, and whatsminers can be tuned, but not sure how they will react with manually setting 0 speed when in shutdown mode.
Regarding disabling boards, that might be something we could do automatically? Assume each board draws a minimum of 800W (2400/3), then shut down boards at 2400W and 1600W, although I don't know how BOS will react to this, another thing I've not tested myself. It's possible it will try to use 2400W on that single board, but it's also possible it will split it up evenly across all boards, and ignore the disabled one.

Hello,

I have tested the BOS firmware with individual boards shutting down. Firmware responds correctly, performance drops by a third when one hashboard is disabled, by two thirds when two hashboards are disabled - compared to the set value, see attachment. The only anomaly is that after turning off or on a hashboard, all boards shutdown for about two minutes. After this time, however, the hashboards that are on will boot to full power.
On the point of fan settings - I don't know about other firmware, but I'm running the S19 with BOS where the fans are still set to 100% speed - manually, as recommended by Braiins experts. This is because the chips increase power consumption with increasing temperature. For this reason I don't use automatic fan speed control. But if the BOS is not running, it would be great to at least lower the fans, say at least to 30%, this would eliminate possible overheating of the chips by accumulated heat after the BOS is turned off.


Support Spring Beans

When use spring with lombok there is no such features support like

Spring Beans Dependencies Diagram
Navigation to autowired dependencies

https://www.jetbrains.com/help/idea/spring-support.html
Is it possible to support them?

Yes please! It would be so useful

Yes, for me especially when I create private final fields for the beans and use a @RequiredArgsConstructor it would be nice to have the bean link on the field.