Moves to mapper interface

Borrowing a design element I used in UDTube, I decompose the dataset object into two pieces:

a Mapper interface which knows how to map between lists of strings and tensors (to decode and encode)
DataSet, as before.

There was no particular reason for the mapper functions to live inside the dataset. This separates the two pieces and uses the mapper object for prediction.
Closes #137. That issue says that the encoding/decoding should be moved to the index, but this actually makes those two even more modular.
(I also imported fix #272 and resolved some merge stuff from #268 etc.)

I don't really have a problem with it, but I don't actually follow the reasoning for needing another class. The "mapping" operations, to me, could intuitively live in the index, the dataset, or the datamodule. Is the ambiguity there around where it should live actually the problem? Or is this just an OOP principle that someone established?

The way to think of the design is this:

The datamodule creates an index.
The datamodule creates datasets.
At the creation of each dataset, a mapper is made from the index and passed to the dataset. (This is necessary because the dataset generates tensors on the fly.)

I am moving it out of the dataset in this PR, because there are a number of places where you want to do what the mapping does but you don't need a reference to the full dataset. (For instance, you don't need the huge list that contains the actual data.) One example of this is in prediction: you need to map but there's no direct reference to the dataset. Putting it in a separate class helps with potential circularity issues.
The other obvious option is to make it part of the index but there are places where we need an index but not a mapper or vice versa. For instance the expert class doesn't need tensors or any of the padding, so it uses the index but not the mapper. I despaired of a way to separate out "string to integer" vs. "integers to tensors" except putting them in separate classes.
The general OOP principle at play here is the Law of Demeter. In the old code (this was part of predict.py) had loader.dataset.decode_target(...); this now reads mapper.decode_target(target).

Thanks for describing your thinking.

there are a number of places where you want to do what the mapping does but you don't need a reference to the full dataset

Yes good point. I think another place this is helpful is for runtime debugging. previously if I wanted to log encoded inputs/predicted outputs in the forward function at runtime, I needed the dataset.


Task/180 upgrade check blueprint files for changes before trying to override them

PR details
What is the purpose of this pull request? (put an "X" next to an item)
Please delete options that are not relevant.

[ ] Documentation update
[ ] Bug fix
[ ] New metadata support
[ ] Enhanced metadata
[ ] Add a CLI option
[ ] Add something to the core
[x] Other, please explain:
enhanced upgrade process

What changes did you make? (Give an overview)

closes #180

Is there anything you'd like reviewers to focus on?
...
Checklist

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] ESLint & Prettier are not outputting errors or warnings
[n/a] README.md updated (if applicable)

i think this will make it easier with #166 because at least we are now defaulting to override instead of to keeping existing files


Make installer for swood

Using nsis for linux and stuff, make a windows installer that adds to path and stuff and auto rebuild on a new release
On other systems I dunno
mac ffmpeg binaries

We should use pynsist for the nsis thing instead of DIY

Done (https://meme.institute/swood)


Alias Delete Objects Via Config

WHAT
Deletes fail when SFDC sends for an object to delete that has a different name in the app. IE. Account has been aliased to Vendor.
WHAT
Added a salesforce_ar_sync.yml option to allow you to specify a map of SFDC objects to app objects.
OTHER
Made a lot of Rubocop changes to the gem... I just had to!

Is this necessary? The way it was designed was that the inbound delete message would indicate the name of the model that the delete is for.

@lnediger This would allow us to configure the map in our app, instead of needing salesforce outbound messages to know which classes exist in our app. @Phillita may be able to speak to this more.

@lnediger It sounds like we would have to make an outbound message in SFDC that knew about the Account to Vendor mapping. Currently, SFDC puts everything into one general deleted object table and then send them to the delete route. It just takes the type of object, in this case account, and send that to the delete without any sort of alias.
What are your thoughts?

Changes:
I'm going to pull the code changes out so that there isn't a mix of Syntax updates with it.
I'll also be updating the readme with the new config file updates.
Hopefully this will help with tests.

Deleting this branch in favour of #19.


download doesn't work for my m4v files on firefox

Any idea why the download fails for this scenario?

If I use the right-click "Save Link As..." context action, I get a browser alert:
The download cannot be saved because an unknown error occurred.
Please try again.
This is what the target link looks like when I use the "Copy Link Location" context action:
https://synapse.isrd.isi.edu/hatrac/Zf/Zf_1-1VD4/Behavior_1-1VDG.m4v:OIPYOU2ANYKFWBBERIYGPJ4NCY?uinit=1&cid=record
I get the feeling this is related to the Content-Type and versioned file naming in the URL.  What I wonder is whether there is any workaround I can use to make it possible to save the data via browser in the way a typical user would expect.

@karlcz
I’m not able to reproduce the error that you described. I tested it both on my machine as well as a Fedora 35 VM. Clicking the link and “Save Link As...”  start the download. Are you still experiencing this issue?

Closing due to inactivity.


docs: Add path field to support-scheduler intervalAction API example

Update support-scheduler API document to add path field to intervalAction address examples.
fix #3828
Signed-off-by: Ginny Guan<EMAIL_ADDRESS>

If your build fails  due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/edgex-go/blob/main/.github/Contributing.md
PR Checklist
Please check if your PR fulfills the following requirements:

[x] I am not introducing a breaking change (if you are, flag in conventional commit message with BREAKING CHANGE: describing the break)
[x] I am not introducing a new dependency (add notes below if you are)
[ ] I have added unit tests for the new feature or bug fix (if not, why?) N/A, fix in API docs only
[ ] I have fully tested (add details below) this the new feature or bug fix (if not, why?) N/A, fix in API docs only
[ ] I have opened a PR for the related docs change (if not, why?) N/A, fix in API docs only

Testing Instructions

N/A
New Dependency Instructions (If applicable)

N/A

Codecov Report

Merging #3832 (8b40f19) into main (b2bbeb2) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #3832   +/-   ##
=======================================
  Coverage   45.77%   45.77%
=======================================
  Files         112      112
  Lines        9640     9640
=======================================
  Hits         4413     4413
  Misses       4844     4844
  Partials      383      383

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b2bbeb2...8b40f19. Read the comment docs.


Cannot follow guest while debugging

Describe what happened:
When my guest (with write access) attaches to my debugging session, they see the current execution point in a read-only copy, not in the "original" file. Both of us can step through the program, but I cannot see my guests cursor in their file (and vice versa). This means, when trying to highlight some code while debugging, we constantly have to refocus on each other. Then, we can see each others cursors, but my guest cannot see the current execution point. This makes co-debugging such a pain that we switched back to the old screensharing approach :(
PS: This happens regardless of whether the guest joins in the browser, in VSCode, or in VS.
What was your system configuration?
Product and Version: VS 2022 17.3.2
OS Version: Windows
Live Share Extension Version: 1.0.5545.1
Target Platform or Language: C++
Steps to Reproduce / Scenario:

Start Live Share Session in VS,
Join Session (e.g. in another instance of VS, or in VSCode)
Verify that both can follow each other normally
Set a breakpoint somewhere and start a debugging session (in my case, debugging a google test with VS Google Test Adapter), guest should attach automatically by default
Both participants be focused on the breakpoint, but cannot see each other's cursor (because they are in different files)

Can anyone reproduce this?

Thanks for filling the issue. We will be taking a look at this in the upcoming quarter.

I am unable to repro the issue mentioned. Given the issue is old, @lursyy can you please confirm if you are still seeing this?

Just give me a week or so, I will try to find the time to reproduce this again. If I can reproduce it using a MWE, I will try to provide a screencast as well.

Ok, so I can still reproduce the issue as described above in the solution that I use with our codebase. For the screencast below, I added a new Console Application Project to that Solution. As you can see in the recording, running the Program leads to a second (locked) version of the file, where the guest can see the execution point but the participants cannot see each other's cursors. The guest can focus on the host's cursor, but then the debugging information is not visible (I guess because it is somehow related to a "different file"?). I also noted that two console windows pop up when executing the program.
When I instead create a fresh solution with a new Console Application inside, I do not experience this problem. Do you have any idea what setting / property could cause this? The depicted solution with the issue was created a few months ago as a "Blank Solution" and projects are added and removed as needed. Is there some way of dumping my settings/config while protecting any potentially sensitive data (file names etc.)?
https://user-images.githubusercontent.com/13033130/225287044-99938f8b-8d38-44c4-bd49-266ad844c290.mp4

Thanks for the detailed scenario @lursyy. I tried the exact scenario you mentioned and I am still not able to repro it. I am not sure what settings might be different. For now, I will keep this issue open for more feedback from users, but we won't be able to prioritize fixing this issue.

Ok, I will let you know if I can come up with a good repro.
Nandini @.***> schrieb am Fr., 31. März 2023, 18:35:

Thanks for the detailed scenario @lursyy https://github.com/lursyy. I
tried the exact scenario you mentioned and I am still not able to repro it.
I am not sure what settings might be different. For now, I will keep this
issue open for more feedback from users, but we won't be able to prioritize
fixing this issue.
—
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/live-share/issues/4757#issuecomment-1492243809,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADDN5KUYI3AQZYK6EGJFDCDW64BWDANCNFSM6AAAAAAQBPLGNA
.
You are receiving this because you were mentioned.Message ID:
@.***>


3D options

Hello.
The 3D options in ZMAP are not activated. I am using ZMAP 7 and Matlab R2019b. Hope someone can help me to figure out this issue.

This is the wrong ZMAP project.


Issue with ViewPropTypes, the deprecated import and stuff. Both iOS and Android apps are crashing in production.

Description
So basically, We had to update our app for new EAS Build on Expo, which meant upgrading Expo SDK to 46.0.0
I managed to do that and after many trials and errors, the build finally passed on Expo. But here's the catch, both iOS and Android apps are crashing in production, straight on startup. You click to open the app and boom, it crashes.
Note: App works fine locally on emulators and tested on a real device with Expo Go. Works like charm there.
I can provide you with crashlogs.crash reports and other stuff you'd need to help me identify the issue. I'm blocked by NDA to share the whole app here, but if you need something, I can ask rest of my team is it okay to share it and I will do so.
Android crash stack trace:
https://paste.pics/1e3297c8b382d597f2d34b8b4b003f18
I can provide other crashlogs at request. Thanks in advance
Version
0.69.5
Output of npx react-native info
System:     OS: Linux 6.0 Pop!_OS 22.04 LTS     CPU: (8) x64 Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz     Memory: 10.36 GB / 15.59 GB     Shell: 5.1.16 - /bin/bash   Binaries:     Node: 16.18.0 - ~/.nvm/versions/node/v16.18.0/bin/node     Yarn: 1.22.19 - ~/.nvm/versions/node/v16.18.0/bin/yarn     npm: 9.3.1 - ~/.nvm/versions/node/v16.18.0/bin/npm     Watchman: Not Found   SDKs:     Android SDK: Not Found   IDEs:     Android Studio: AI-213.7172.25.2113.9123335   Languages:     Java: 11.0.17 - /usr/bin/javac   npmPackages:     @react-native-community/cli: Not Found     react: 18.0.0 => 18.0.0      react-native: ^0.69.5 => 0.69.7    npmGlobalPackages:     *react-native*: Not Found info React Native v0.71.1 is now available (your project is running on v0.69.7). info Changelog: https://github.com/facebook/react-native/releases/tag/v0.71.1. info Diff: https://react-native-community.github.io/upgrade-helper/?from=0.69.7. info To upgrade, run "react-native upgrade"
Steps to reproduce
Live app, can't share it's code.
Snack, code example, screenshot, or link to a repository
I can provide what's necessary if you point me to a part of code you'd like to see. The app is live and NDA is not allowing me to share it.

We can't really help without:

The stacktrace of the crash
A minimal reproducer

Plus you should consider opening this issue against the Expo repo:
https://github.com/expo/expo/issues
as they can help you further over there

Solved it with a resolver library. Thanks, you can close this

Solved it with a resolver library. Thanks, you can close this

can you please share how do you solve this issue.

Solved it with a resolver library. Thanks, you can close this

can you please share how do you solve this issue.

plugins: [
			"@babel/plugin-proposal-export-namespace-from",
			"react-native-reanimated/plugin",
			[
				"module-resolver",
				{
					root: ["."],
					resolvePath(sourcePath, currentFile, opts) {
						if (
							sourcePath === "react-native" &&
							!(
								(
									currentFile.includes("node_modules/react-native/") || // macos/linux paths
									currentFile.includes("node_modules\\react-native\\")
								) // windows path
							) &&
							!(currentFile.includes("resolver/react-native/") || currentFile.includes("resolver\\react-native\\"))
						) {
							return path.resolve(__dirname, "resolver/react-native")
						}
						/**
						 * The `opts` argument is the options object that is passed through the Babel config.
						 * opts = {
						 *   extensions: [".js"],
						 *   resolvePath: ...,
						 * }
						 */
						return undefined
					},
				},
			],
		],

Pasted this in babel.config.js
And installed these two libraries:
"@babel/plugin-proposal-export-namespace-from": "^7.18.9",
"deprecated-react-native-prop-types": "^4.0.0",

Found it somewhere on GitHub in issues, can't remember exactly where. If this doesn't fix your issue, I'll dig the solution out. Cheers!

Solved it with a resolver library. Thanks, you can close this

can you please share how do you solve this issue.

plugins: [
			"@babel/plugin-proposal-export-namespace-from",
			"react-native-reanimated/plugin",
			[
				"module-resolver",
				{
					root: ["."],
					resolvePath(sourcePath, currentFile, opts) {
						if (
							sourcePath === "react-native" &&
							!(
								(
									currentFile.includes("node_modules/react-native/") || // macos/linux paths
									currentFile.includes("node_modules\\react-native\\")
								) // windows path
							) &&
							!(currentFile.includes("resolver/react-native/") || currentFile.includes("resolver\\react-native\\"))
						) {
							return path.resolve(__dirname, "resolver/react-native")
						}
						/**
						 * The `opts` argument is the options object that is passed through the Babel config.
						 * opts = {
						 *   extensions: [".js"],
						 *   resolvePath: ...,
						 * }
						 */
						return undefined
					},
				},
			],
		],

Pasted this in babel.config.js
And installed these two libraries:
"@babel/plugin-proposal-export-namespace-from": "^7.18.9",
"deprecated-react-native-prop-types": "^4.0.0",

Found it somewhere on GitHub in issues, can't remember exactly where. If this doesn't fix your issue, I'll dig the solution out. Cheers!

Still the same anything else needs to be added here?

Solved it with a resolver library. Thanks, you can close this

can you please share how do you solve this issue.

plugins: [
			"@babel/plugin-proposal-export-namespace-from",
			"react-native-reanimated/plugin",
			[
				"module-resolver",
				{
					root: ["."],
					resolvePath(sourcePath, currentFile, opts) {
						if (
							sourcePath === "react-native" &&
							!(
								(
									currentFile.includes("node_modules/react-native/") || // macos/linux paths
									currentFile.includes("node_modules\\react-native\\")
								) // windows path
							) &&
							!(currentFile.includes("resolver/react-native/") || currentFile.includes("resolver\\react-native\\"))
						) {
							return path.resolve(__dirname, "resolver/react-native")
						}
						/**
						 * The `opts` argument is the options object that is passed through the Babel config.
						 * opts = {
						 *   extensions: [".js"],
						 *   resolvePath: ...,
						 * }
						 */
						return undefined
					},
				},
			],
		],

Pasted this in babel.config.js
And installed these two libraries:
"@babel/plugin-proposal-export-namespace-from": "^7.18.9",
"deprecated-react-native-prop-types": "^4.0.0",

Found it somewhere on GitHub in issues, can't remember exactly where. If this doesn't fix your issue, I'll dig the solution out. Cheers!

Still the same anything else needs to be added here?

Woops, my bad. I forgot that you need another file. In the meantime, I found the repo which's instructions I've followed.
Here's the link, you have it step by step
https://github.com/psycheangel/deprecated-with-module-resolver


Nether portal link problem 1.18

Information

Server version: This server is running Paper version git-Paper-19 (MC: 1.18) (Implementing API version 1.18-R0.1-SNAPSHOT) (Git: 892c292)

Full output of /mv version -p: https://paste.gg/p/anonymous/d8942cb352ef4621abd4d05fc3bd38ba

Server log: https://gist.github.com/GroGaloraGran/5fe9d7665c76d61c180d038776f6b543

Details
I can reproduce this issue when all plugins are enabled. The problem does not appear when the Multiverse-NetherPortals v4.2.1 plugin is disabled, but then another problem appears - in the second world the portal to the Nether of the second world does not work and leads to the nether of the first world.
description
In our upper world, a gold farm was created on Nether portals, for which a "service" portal was left in Nether, which was Linked with the farm portals. Below in Nether and a little to the side there is a metro line with a portal through which you can get to the farm.
If we return from the upper world to Nether through the correct portal, we find ourselves in the service portal in Nether (this is dangerous due to the aggressive zombie pig)
here is no problem with the disabled plugin Multiverse-NetherPortals v4.2.1, but we have an additional world with our own Nether world, and if we try to get from the additional world (let it be World2) and its world2_Nether with the plugin disabled, then for some reason we find ourselves in world_nether ((
Steps to reproduce
farm created in 1.16.5 portals are connected

Exit from Nether to the upper world - to approach the farm, aside.
Portals of the farm with a separate Portal in Nether (by coordinates)
after the "transition" to 1.18 with the plugin enabled:
When leaving Nether to the upper world, we will find ourselves several levels below the previously created portal.
If you go to Nether from the previously created portal, we get to the "service" portal tied to the farm

Expected behavior
It is expected that the snapping of portals by coordinates will be preserved, That is, the output of a particular portal will lead to where it was snapped earlier.
Screenshots
do not take a screenshot for this (

Hey
to confirm the anomaly, I made an additional video
Logs and videos using the Multiverse-NetherPortals plugin
https://youtu.be/OzTZgKUTZc4
https://gist.github.com/GroGaloraGran/75e7c5648cc5d3f7ef4bef29240b484c
Logs and videos with disabled plugin
https://youtu.be/DQRweOETx5s
https://gist.github.com/GroGaloraGran/78330ebcbe7425fe40277e0d9663551e
this is what is on our assembly at the moment, with the latest test core Paper 1.18.1 # 71

Hello. Happy Holidays. Can I get feedback from you? This is an important plugin for us and without it we cannot properly update our server to 1.18.1 thanks

I've watched the videos you uploaded and it helps explain the issue, thanks for that.
What's the value of level-name in server.properties?

Hello
level-name = world

I'm closing this in favour of #244.


Make all units immutable

Related to #1 - an idea I would like to throw out:
The API will be cleaner, bugs harder to introduce and maintenance a lot less of a burden if every type defined in unitmath is immutable*.
The major downside would be a possible performance cost but I would argue against any premature optimisations without benchmarks that show cloning units is a bottleneck.

Related: when configuring the global options, any changes made in one file should be independent from changes made in other files. Currently
unit.config(options)

updates the global unit configuration. A better API might be
newUnit = oldUnit.config(options)

where the config() function returns a brand new instance.

*: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze#Browser_compatibility

Thanks for your ideas, @harrysarson!

every type defined in unitmath is immutable

I absolutely agree. Good idea to use Object.freeze(), I hadn't thought of that. All of the public API functions will return a new, frozen unit.

the config() function returns a brand new instance

What exactly do you mean by that? Do you mean that the factory function unit and its configurable options would be cloned? So you would do:
const unit = require('unitmath')

const newUnit = unit.config(options)

// Use newUnit factory to build units
newUnit('3 kg')  // and so on

Then you could also do:
const unit = require('unitmath').config(options)

Is that what you had in mind?

Yes, exactly what I have in mind!

I've updated the README with your suggestions, thanks again!

Looks perfect!


Fix for spotify/ramlfications#85

Current coverage is 96.87%

Branch #88 has no coverage reports uploaded yet.

Powered by Codecov. Updated on successful CI builds.

woah 13 commits! I like a clean commit history (or as clean as possible) :smile:  Please rebase/squash into one commit.

Sorry, will do. I'm a bit new to this stuff, so it might take me a while to figure out :)

@bpowell65536 here's a good resource that I point people to on rebasing/squashing commits.

@bpowell65536 once you squash everything (FYI it will require a force-push since you essentially re-write history when squashing) and address the little nitpicks, I'll merge this. This looks good, thank you for catching this bug and giving a patch!

@econchick Thanks! I have squashed everything now and pushed to my remote repository, but I'm not sure how to make the squashed commit appear instead of all the individual ones. Do I need to make a PR for the squashed commit instead?

@bpowell65536 if you appropriately squashed everything, and (force) pushed to the same remote branch as before, then the pull request should automatically update. So I have a feeling it wasn't done correctly.


Make a function for adding lab custom losses

📝 Description of the feature
To add a custom loss in lab from scripting requires modifying array length variables and appending to arrays. Create 2 functions to make this easier e.g.

💡 Steps for implementing the feature

implement for internal
implement for external
Functions should include add/remove/clear

Functions can live in src/ansys/motorcad/core/methods/rpc_methods_utility.py
🔗 Useful links and references
No response

good first task for Shreya


check os compatibility and dependencies_version in prepare_env.sh

What do these changes do?
check os compatibility and dependencies_version in prepare_env.sh to improve "Getting Started" experience
Related issue number
Fixes #93

cc/  @yecol @sighingnow @wenyuanyu

cc/  @yecol @sighingnow @wenyuanyu

Maybe also checking the platform's version? like Ubuntu 14, 16, 18?

Maybe also checking the platform's version? like Ubuntu 14, 16, 18?

LGTM.
Not part of this task, but I'm wondering if we suggest users kind rather than minikube on Linux and MacOS as well. Test is needed, though. (not in high pri).

thanks for the suggestion. this script can include MacOS prepare environment though kind i think, i will add this feature in next related pr after test.

LGTM.
Not part of this task, but I'm wondering if we suggest users kind rather than minikube on Linux and MacOS as well. Test is needed, though. (not in high pri).

thanks for the suggestion. this script can include MacOS prepare environment though kind i think, i will add this feature in next related pr after test.


Adapters loading

Right now to add in adapters once the server has started, we need to restart the server. I just minimally changed the load function and the do_POST function in server.py, adding in an optional argument to load so that the user can add in an adapter during runtime. I tested it with by starting a server with Mistral and then changing the model to Phi-3B w/ LoRa adapters with a POST request.
**I did not add the restriction that adapters need to be in the current working directory **

Hi @awni , I think the commit you added caused some failed tests?


Allow saving other image formats

#2411
@dictoon I'm unable to save jpg successfully and the bmp images o not appear right. Could you please tell me what changes I might need to make to get it right?

We don't want to explicit handle some image formats, instead we want to automatically support all formats supported by OIIO.

Superseded by #2445.


NVlink for tensorflow2.5, linux, 3090 GPU

OS: Linux Ubuntu18.04
TF building method: TF docker with Nvidia Docker,  CUDA version: 11.1
Version: Tensorflow 2.5.0
GPUs: RTX3090 x2
Hi there.
I noticed that the NVlink for 3090 is published. I got a workstation with 2 3090 GPU cards. I wish to know is it necessary to got a NVlink to accelerate data transfer between these 2 cards?
For tensorflow building via docker, should i install some extra toolkit? And when I train model with MirrowedStrategy, what should I do to activate the NVlink?

@FavorLionQY Sorry for the late response! Could you please refer to the link to build from source and have a look on the similar issue link1, link2 , link3 for reference .Please let us know if it helps?Thank you!


deprecated "net.tunnels_s" pattern still exists in test code

Title: "net.tunnels_s" is the signature of code using a pre-v1.0.0 version of the sdk.  It must be replaced with "net.tunnels" to function.

beautiful. more of this!

Fixed.


allow static tasks to run without a bundle.wrangler.config.* file.

Taken care of.  See https://github.com/elycruz/gulpw/releases/tag/0.2.66-beta


Portal 10.3.1 support

Hi there
I have come across an issue that Portal 10.3.1 content does not display.
Is this expected with the new TLS requirements?

Hi @jplunkett1  I have not see that as we don't have a 10.3.1 Portal available to test against. Have you tried several different browsers? Are there any console errors to point to? Or failed network requests?

Ill ask my customer to see if he can use a different browser and get those console errors.
However he signs in successfully - it looks like we are getting CORS errors.
This is an internal portal as well


[5.5] Remove now unneeded set of mb_internal_encoding

Follow-up to #6891.
I did some tests on PHP 5.6, and mb_internal_encoding() appears to give 'UTF-8' by default. Even if default_charset INI setting has been set to a different charset.
So the only way to have a different charset is to call mb_internal_encoding('other charset') beforehand.

Will just leave it explicit.


Adding svelte animation with transition:draw

Is it possible to use svelte's transition:draw with the Icon component? If so, how would you recommend going about it?
I tried looking through the source code, but still can't really figure out how to inject transition:draw into the path element. Thank you!

@ambiguous48
I think for that case you would need to write your own Icon component like this:
https://svelte.dev/repl/843e711ee337430f852b17f2cf351e1e?version=3.46.6


lock free implementation

I wanted to give a high-level overview of how this is intended to work:
def reserve(limiter, now):
    now_micros = unix_micros(now)
    while True:
        state = atomic_load(limiter.state)
        last_updated_micros, tokens, ok = unpack_state(state)
        # basic sanity checking to ensure state is properly initialized
        if !ok:
            limiter.reinit()
            continue

        # if state was recently updated (especially by a concurrent goroutine racing
        # us for the CAS below), it is very likely this will be true and we can exit quickly.
        if last_updated_micros == now_micros:
            if tokens <= 0:
                return False
            else:
                _, tokens, ok = atomic_decrement(limiter.state)
                # if tokens was _positive_ at the time the decrement was applied,
                # it will be 0 or greater here (and we should let the request through)
                return ok && tokens >= 0
        else:
            new_updated_micros, tokens = limiter.advance(now_micros)
            if tokens < 1:
                return False
            tokens -= 1

            # attempt to update limiter state with a Compare-And-Set operation.  If we fail,
            # we begin the loop again.  However if we failed it means a concurrent goroutine's
            # CAS succeeded and made progress, so with high probability
            # `last_updated_micros == now_micros` and we will enter the if condition above
            # (which always exits the loop + function)
            if atomic_cas(limiter.state, pack_state(new_updated_micros, tokens), state):
                return True

benchmarks on a m5d.4xl (old is the go ratelimter, new is this version):
$ benchstat old.txt new.txt
name          old time/op    new time/op    delta
100000RPS        108ns ± 0%      82ns ± 0%  -24.62%  (p=0.002 n=6+6)
100000RPS-2      122ns ± 1%      41ns ± 1%  -65.93%  (p=0.002 n=6+6)
100000RPS-4      161ns ± 1%      21ns ± 1%  -86.86%  (p=0.002 n=6+6)
100000RPS-8      208ns ± 2%      11ns ± 2%  -94.52%  (p=0.002 n=6+6)
100000RPS-16     251ns ± 2%       7ns ± 3%  -97.31%  (p=0.002 n=6+6)
100000RPS-32     273ns ± 3%       7ns ±14%  -97.35%  (p=0.002 n=6+6)

@ryderlewis on my macbook it sometimes takes over 100 iterations when using 8 cores, but that is with a 10M RPS limit:
https://gist.github.com/bpowers/ea826c6db580d6d7bbcfaed7c5fd4e0b
at 10k (with test modified for 10k burst) we see 47 iterations:
https://gist.github.com/bpowers/3ec61a6afeabd12354494ca9eb18dd3f
at 1k (with test modified for 1k burst) we see 15 iterations:
https://gist.github.com/bpowers/e2abb91520e3e69955f3becd4f6a25af

it would be great to see what it looks like on a 30+ core ec2 instance.  Also -- I believe those multiple loops are almost entirely from right at the beginning when eating through the burst, because for RPS limits under 1M after the burst has been worked through, we only accrue 1 token per microsecond time increment (so all threads that lose the race should dump into the first return false of the loop)


Goldstone Complains about Making Migrations

This issues is being filed at the request of @jxstanford
Expected Behavior

Current Behavior

The log output from Goldstone shows the following error condition. The Services Dashboard does not display data in GS
Jun 20 16:41:00 localhost docker/gstask[917]: [2016-06-20 20:41:00,288: INFO/MainProcess] Received task: goldstone.core.tasks.process_alerts[e58319fc-07bb-49e7-81ee-d9f54e0a5002]
Jun 20 16:41:01 localhost docker/gsapp[917]:  Your models have changes that are not yet reflected in a migration, and so won't be applied.
Jun 20 16:41:01 localhost docker/gsapp[917]:  Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
Jun 20 16:41:10 localhost docker/gstask[917]: [2016-06-20 20:41:10,938: INFO/Worker-2] Starting new HTTP connection (1): <IP_ADDRESS>
Jun 20 16:41:12 localhost docker/gstask[917]: [2016-06-20 20:41:12,403: INFO/Worker-2] Starting new HTTP connection (1): <IP_ADDRESS>

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

Version used: 1.1.2
Operating System and version (desktop or mobile): Desktop, W7

Paging @lexjacobs


Fixing a test failure caused by Firestore 0.14.1 release

projectId field is no longer reliably exposed by the Firestore client -- it seems to be lazy initialized.

FYI @schmidt-sebastian


Multipart uploads don't work in Riak S2 (CS) v2.1.1 [JIRA: RCS-378]

The problem is that multipart upload identifier cannot be found after it has been successfully initialized. This issue can be easily reproduced:
## This project can be used for a quick roll up of development environment
$ git clone https://github.com/manifest/riak-s2-erlang-client.git
$ cd riak-s2-erlang-client
$ git co feature/multipart-uploads

## The build script will install Riak S2 within Docker container,
## create a user for you and store all information required for
## connection to the .docker.env.config file.
$ ./run-docker.sh

## Running Riak S2 client
$ make app shell

%% Initializing connection to Riak S2
{ok, Conf} = file:consult(".docker.env.config"),
{_, Opts} = lists:keyfind(user, 1, Conf),
{_, #{host := Host, port := Port}} = lists:keyfind(httpc_options, 1, Conf),
{ok, Pid} = gun:open(Host, Port, #{protocols => [http]}).

%% Creating a bucket for the testcase
riaks2c_bucket:put(Pid, <<"test-bucket">>, Opts).

%% Initializing multipart upload
#'InitiateMultipartUploadResult'{'UploadId' = Id} = riaks2c_object_multipart:init(Pid, <<"test-bucket">>, <<"test_file">>, #{}, Opts).
%% #'InitiateMultipartUploadResult'{
%%   anyAttribs = [],
%%   'Bucket' = <<"test-bucket">>,'Key' = <<"test_file">>,
%%   'UploadId' = <<"cj-EXDu-T-q7N6ifBOux9A==">>}

%% Trying to upload a part, we get the error with the code: "NoSuchUpload".
riaks2c_object_multipart:put(Pid, <<"test-bucket">>, <<"test_multipart">>, <<"1">>, Id, 1, #{}, Opts).
%% ** exception error: {bad_upload_id,<<"test-bucket">>,<<"test_multipart">>,<<"cj-EXDu-T-q7N6ifBOux9A==">>}

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  NoSuchUpload
  <Message>The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.</Message>
  <UploadId>cj-EXDu-T-q7N6ifBOux9A==</UploadId>
  <HostId>host-id</HostId>
</Error>

We can see that our upload "cj-EXDu-T-q7N6ifBOux9A==" is exist using erlang-s2-erlang-client.
riaks2c_object_multipart:list(Pid, <<"test-bucket">>, #{}, Opts).
%% #'ListMultipartUploadsResult'{
%%   anyAttribs = [],'Bucket' = <<"test-bucket">>,
%%   'KeyMarker' = [],'UploadIdMarker' = undefined,
%%   'NextKeyMarker' = [],'NextUploadIdMarker' = [],
%%   'Delimiter' = [],'Prefix' = [],'MaxUploads' = 1000,
%%   'IsTruncated' = false,
%%   'Upload' =
%%     [#'Upload'{
%%       anyAttribs = [],'Key' = <<"test_file">>,
%%       'UploadId' = <<"cj-EXDu-T-q7N6ifBOux9A==">>,
%%       'Initiator' =
%%         #'CanonicalUser'{
%%           anyAttribs = [],'ID' = <<"HTFXXZNN8WD1JG7NQ-G_">>,
%%           'DisplayName' = <<"user">>},
%%           'Owner' =
%%             #'CanonicalUser'{
%%               anyAttribs = [],'ID' = <<"HTFXXZNN8WD1JG7NQ-G_">>,
%%               'DisplayName' = <<"user">>},
%%           'StorageClass' = <<"STANDARD">>,
%%           'Initiated' = <<"2016-11-06T10:55:13.000Z">>}],
%%   'CommonPrefixes' = undefined}

<?xml version="1.0" encoding="UTF-8"?>
<ListMultipartUploadsResult
  xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <Bucket>test-bucket</Bucket>
  <KeyMarker></KeyMarker>
  <NextKeyMarker/>
  <NextUploadIdMarker></NextUploadIdMarker>
  <Delimiter></Delimiter>
  <Prefix></Prefix>
  <MaxUploads>1000</MaxUploads>
  <IsTruncated>false</IsTruncated>
  <Upload>
    <Key>test_file</Key>
    <UploadId>cj-EXDu-T-q7N6ifBOux9A==</UploadId>
    <Initiator>
      <ID>HTFXXZNN8WD1JG7NQ-G_</ID>
      <DisplayName>user</DisplayName>
    </Initiator>
    <Owner>
      <ID>HTFXXZNN8WD1JG7NQ-G_</ID>
      <DisplayName>user</DisplayName>
    </Owner>
    <StorageClass>STANDARD</StorageClass>
    <Initiated>2016-11-06T10:55:13.000Z</Initiated>
  </Upload>
</ListMultipartUploadsResult>

The same using s3cmd client:
$ s3cmd --signature-v2 -c s3cmd.cfg multipart s3://test-bucket
s3://test-bucket/
Initiated	Path	Id
2016-11-06T10:55:13.000Z	s3://test-bucket/test_file	cj-EXDu-T-q7N6ifBOux9A==

The same "NoSuchUpload" error, trying to abort upload:
$ s3cmd --signature-v2 -c s3cmd.cfg abortmp s3://test-bucket/test_multipart cj-EXDu-T-q7N6ifBOux9A==
ERROR: S3 error: 404 (NoSuchUpload): The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.

Or trying to list parts of our upload.
$ s3cmd --signature-v2 -c s3cmd.cfg listmp s3://test-bucket/test_multipart cj-EXDu-T-q7N6ifBOux9A==
ERROR: S3 error: 404 (NoSuchUpload): The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.

s3cmd.cfg that is used in the testcase
[default]
access_key = HTFXXZNN8WD1JG7NQ-G_
bucket_location = US
cloudfront_host = cloudfront.amazonaws.com
cloudfront_resource = /2010-07-15/distribution
default_mime_type = binary/octet-stream
delete_removed = False
dry_run = False
enable_multipart = False
encoding = UTF-8
encrypt = False
follow_symlinks = False
force = False
get_continue = False
gpg_command = /usr/local/bin/gpg
gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s
gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s
gpg_passphrase = password
guess_mime_type = True
host_base = s3.amazonaws.com
host_bucket = %(bucket)s.s3.amazonaws.com
human_readable_sizes = False
list_md5 = False
log_target_prefix =
preserve_attrs = True
progress_meter = True
proxy_host = <IP_ADDRESS>
proxy_port = 8080
recursive = False
recv_chunk = 4096
reduced_redundancy = False
secret_key = Fl-QLrrjZVgWrdBnlLA-eW_gGRljWLw9zTOE8g==
send_chunk = 4096
simpledb_host = sdb.amazonaws.com
skip_existing = False
socket_timeout = 300
urlencoding_mode = normal
use_https = False
verbosity = INFO

Are there any comments on this issue?

It seems you used the different keys between Initiate Multipart Upload (test_file) and Part Upload (test_multipart). Using the same key over a series of Multipart Upload requests should work.

@ksauzz you're right! My fault. Thanks.


UINavigationbar is under  status bar

sometimes the UINavigationbar is under  status bar  like  the photo

Are you a developer from TaiWan?

yes, from Taiwan.  This issue is other tool issue, please close it.

【iOS9】我的应用也出现这个，页面起点在状态栏下隐藏。但是导航到下一个子页面，返回来，页面显示正常。请问有什么建议么？


No ads with version 3.4.0

Hello. I just upgraded from 3.3.0 to 3.4.0 and I am no longer getting any ads during my testing.
For my upgrade process, I did the following:

Delete my old MobFoxSDKCore.framework and replace it with the new one.
Delete my old MATMoatMobileAppKit.framework and replace it with the new one.

Here's a log from my app with the 3.3.0 SDK:

2018-03-07 14:10:08.788683+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin >> showBanner
2018-03-07 14:10:08.788733+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin >> createAndLoadBanner(5a4c16612c0a289489e91b9c37a2cbc1)
2018-03-07 14:10:08.788744+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin: position: bottom-center
2018-03-07 14:10:08.788753+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin: bannerWidth: 320
2018-03-07 14:10:08.788760+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin: bannerHeight: 50
2018-03-07 14:10:09.220293+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin >> loadBannerContent
2018-03-07 14:10:09.221357+0100 LifeTotal[704:313477] Starting refresh timer (30secs)
2018-03-07 14:10:09.430807+0100 LifeTotal[704:313711] Metal GPU Frame Capture Enabled
2018-03-07 14:10:09.434579+0100 LifeTotal[704:313711] Metal API Validation Enabled
2018-03-07 14:10:10.331590+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin >> unhideBanner
2018-03-07 14:10:12.310732+0100 LifeTotal[704:313972] WF: === Starting WebFilter logging for process LifeTotal
2018-03-07 14:10:12.311382+0100 LifeTotal[704:313972] WF: _userSettingsForUser mobile: {
filterBlacklist =     (
);
filterWhitelist =     (
);
restrictWeb = 1;
useContentFilter = 0;
useContentFilterOverrides = 0;
whitelistEnabled = 0;
}
2018-03-07 14:10:12.311991+0100 LifeTotal[704:313972] WF: _WebFilterIsActive returning: NO
2018-03-07 14:10:12.562930+0100 LifeTotal[704:313477] MobFox Ad Request >>> : {
adFormat = banner;
"adspace_height" = 50;
"adspace_strict" = 0;
"adspace_width" = 320;
autoplay = true;
"banner_pos" = 1;
"dev_js" = 1;
"hide_mute" = true;
"o_iosadvid" = "D07DFB30-3252-4273-BCF6-4597AA759BC1";
rt = "iphone_app";
s = 5a4c16612c0a289489e91b9c37a2cbc1;
skip = "";
"sub_bundle_id" = "com.snowleopardgames.lifetotal";
type = waterfall;
v = "Core_3.3.0_static_core";
}
2018-03-07 14:10:13.351238+0100 LifeTotal[704:313477] MobFox ad response: {
customEvents =     (
{
class = Amazon;
id = stack393247;
parameter = b2ce9c95c3f94900a98802f3d47ad6e1;
pixel = "http://nvirginia-callbacks.mobfox.com/customevent.pixel.php?h=5c974f47e7415a43d467defece9c617f&custom_event_id=stack393247";
},
{
class = Conversant;
id = stack393249;
parameter = "675e31fc-3465-11e4-b31e-15f4fced7073";
pixel = "http://nvirginia-callbacks.mobfox.com/customevent.pixel.php?h=5c974f47e7415a43d467defece9c617f&custom_event_id=stack393249";
},
{
class = Facebook;
id = stack393248;
parameter = "1379767115623780_1607430786190744";
pixel = "http://nvirginia-callbacks.mobfox.com/customevent.pixel.php?h=5c974f47e7415a43d467defece9c617f&custom_event_id=stack393248";
},
{
class = Conversant;
id = stack393250;
parameter = "675e31fc-3465-11e4-b31e-15f4fced7073";
pixel = "http://nvirginia-callbacks.mobfox.com/customevent.pixel.php?h=5c974f47e7415a43d467defece9c617f&custom_event_id=stack393250";
},
{
class = AdMob;
id = stack393209;
parameter = "ca-app-pub-9390556562087047/4857138571";
pixel = "http://nvirginia-callbacks.mobfox.com/customevent.pixel.php?h=5c974f47e7415a43d467defece9c617f&custom_event_id=stack393209";
}
);
}
2018-03-07 14:10:13.356656+0100 LifeTotal[704:313477] dbg: ### Amazon: loadAd ###
2018-03-07 14:10:13.356739+0100 LifeTotal[704:313477] dbg: ### Amazon: networkID: b2ce9c95c3f94900a98802f3d47ad6e1
2018-03-07 14:10:15.274614+0100 LifeTotal[704:313477] dbg: ### AMAZON: >>> BANNER AMAZON: adViewDidLoad <<<
2018-03-07 14:10:15.278101+0100 LifeTotal[704:313477] dbg: ### MobFoxPlugin >> MobFoxAdDidLoad

You can clearly see the network response from the waterfall, it choosing the Amazon adapter and loading a (test) add successfully.
Now here's the equivalent log after the upgrade to 3.4.0:

2018-03-07 14:28:41.432179+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin >> showBanner
2018-03-07 14:28:41.432246+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin >> createAndLoadBanner(5a4c16612c0a289489e91b9c37a2cbc1)
2018-03-07 14:28:41.432258+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin: position: bottom-center
2018-03-07 14:28:41.432266+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin: bannerWidth: 320
2018-03-07 14:28:41.432274+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin: bannerHeight: 50
2018-03-07 14:28:42.293722+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin >> loadBannerContent
2018-03-07 14:28:42.909536+0100 LifeTotal[724:320150] Starting refresh timer (30secs)
2018-03-07 14:28:43.154044+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin >> unhideBanner
2018-03-07 14:28:43.920712+0100 LifeTotal[724:320150] dbg: ### MobFoxPlugin >> MobFoxAdDidFailToReceiveAdWithError: Error Domain=MobFoxJSContext Code=1 "No Ad Available" UserInfo={NSLocalizedRecoverySuggestion=No Ad Available, NSLocalizedDescription=No Ad Available, NSLocalizedFailureReason=No Ad Available}
2018-03-07 14:28:43.923206+0100 LifeTotal[724:320467] MobFox wf request >> dealloc

No request/response for the waterfall, no attempt to hit any of my custom event adapters. If I modify MobFoxPlugin.m to use the test inventory hash, it will serve a test MobFox banner. Modifying any custom event adapter to try to serve test ads for that network (ie. Amazon or AdMob) will have no effect.
What's the deal? Is there a more involved upgrade process needed to get 3.4.0 to work? Are there any official release notes to help us out?

Hi,
It should work, we didn't change the interface or anything.
Our tech support will test it with your hash (5a4c16612c0a289489e91b9c37a2cbc1) and see what's the problem.
Thanks

Hi,
Managed to fix this remotely. Please give it one hour to propagate through our system and try again.
Thanks

Thanks for the quick response! Your changes did enable me to receive a test ad for Amazon, but the app then crashed with the following call stack:

2018-03-07 18:42:53.734389+0100 LifeTotal[817:369170] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSPlaceholderArray initWithObjects:count:]: attempt to insert nil object from objects[1]'
*** First throw call stack:
(0x185c7f164 0x184ec8528 0x185c17c9c 0x185b4a8a0 0x185b5687c 0x100d4e954 0x186209d68 0x1862226bc 0x186655ba0 0x186595894 0x1865854c4 0x10194128c 0x10194d9e4 0x10194128c 0x10194d9e4 0x10194d8a4 0x186657878 0x10194128c 0x10194e678 0x10194cf08 0x1019529d0 0x1019526f4 0x1858a706c 0x1858a6b6c)
libc++abi.dylib: terminating with uncaught exception of type NSException

Unfortunately not much to go on there. Here's a bit more of the log from before the crash, in case it helps:

3-07 18:42:49.801235+0100 LifeTotal[817:368956] dbg: ### MobFoxPlugin >> createAndLoadBanner(5a4c16612c0a289489e91b9c37a2cbc1)
2018-03-07 18:42:49.801245+0100 LifeTotal[817:368956] dbg: ### MobFoxPlugin: position: bottom-center
2018-03-07 18:42:49.801253+0100 LifeTotal[817:368956] dbg: ### MobFoxPlugin: bannerWidth: 320
2018-03-07 18:42:49.801260+0100 LifeTotal[817:368956] dbg: ### MobFoxPlugin: bannerHeight: 50
2018-03-07 18:42:50.250494+0100 LifeTotal[817:368956] dbg: ### MobFoxPlugin >> loadBannerContent
2018-03-07 18:42:51.180257+0100 LifeTotal[817:368956] dbg: ### MobFoxPlugin >> unhideBanner
2018-03-07 18:42:51.791089+0100 LifeTotal[817:369170] MobFox wf request >> dealloc
2018-03-07 18:42:51.853541+0100 LifeTotal[817:368956] dbg: ### Amazon: loadAd ###
2018-03-07 18:42:51.853606+0100 LifeTotal[817:368956] dbg: ### Amazon: networkID: b2ce9c95c3f94900a98802f3d47ad6e1
2018-03-07 18:42:52.777363+0100 LifeTotal[817:369169] TIC Read Status [4:0x0]: 1:57
2018-03-07 18:42:52.777437+0100 LifeTotal[817:369169] TIC Read Status [4:0x0]: 1:57
2018-03-07 18:42:52.779562+0100 LifeTotal[817:369273] NSURLConnection finished with error - code -1002
2018-03-07 18:42:52.988765+0100 LifeTotal[817:368956] dbg: ### AMAZON: >>> BANNER AMAZON: adViewDidLoad <<<
2018-03-07 18:42:53.023916+0100 LifeTotal[817:368956] dbg: ### MobFoxPlugin >> MobFoxAdDidLoad
2018-03-07 18:42:53.158109+0100 LifeTotal[817:369170] Metal GPU Frame Capture Enabled
2018-03-07 18:42:53.181239+0100 LifeTotal[817:369170] Metal API Validation Enabled
2018-03-07 18:42:53.734389+0100 LifeTotal[817:369170] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSPlaceholderArray initWithObjects:count:]: attempt to insert nil object from objects[1]'
*** First throw call stack:
(0x185c7f164 0x184ec8528 0x185c17c9c 0x185b4a8a0 0x185b5687c 0x100d4e954 0x186209d68 0x1862226bc 0x186655ba0 0x186595894 0x1865854c4 0x10194128c 0x10194d9e4 0x10194128c 0x10194d9e4 0x10194d8a4 0x186657878 0x10194128c 0x10194e678 0x10194cf08 0x1019529d0 0x1019526f4 0x1858a706c 0x1858a6b6c)
libc++abi.dylib: terminating with uncaught exception of type NSException

Thinking there might be other changes, I browsed the commits between 3.3.0 and 3.4.0. The only thing I noticed that might break is in commit 9f5fc84, where one of the entries in the customEventInfo dictionary passed to requestAdWithSize was renamed from viewcontroller_parent to viewcontroller. I made that same change in my custom event adapters but it had no effect on the crash.
(I also noticed that in the same commit, the GPS-related info was commented out in your adapters. Why is that?)
Any other suggestions for where I could look, or is this on your end?
Thanks!

Not sure, where this is. Do you know what is the last line in your code or the custom event that gets executed before the crash?
We can try to reproduce it here with your hash and the Amazon custom event.

Hi again,
With your hash I'm getting AdMob custom events and with my Amazon app id I'm not getting any demand yet (and this case works).
Do you have any MobFox hash configured to Amazon only that will get me an ad?
Thanks

Right, Amazon only serves ads in a select few countries. You need to enable Amazon test ads like this:
  AmazonAdOptions *options = [AmazonAdOptions options];
  options.isTestRequest = YES; // Turn on isTestRequest to load a test ad
  [self.amazonAdView loadAd:options];

In reference to your previous question, as far as I can tell, the last of my / custom event code executed before the crash is adViewDidLoad:
- (void)adViewDidLoad:(AmazonAdView *)view
{
  // NSLog(@"dbg: ### AMAZON: >>> BANNER AMAZON: adViewDidLoad <<<");
  [self.delegate MFCustomEventAd:self didLoad:view];
}

This doesn't seem to be present in the crash call stack though, so I think the crash occurs in another thread sometime later.

I'm using the "test request" option but to no avail. Maybe because my Amazon app is not approved.
Are you getting it by using this hash: 5a4c16612c0a289489e91b9c37a2cbc1? if so I will try to disable AdMob.

Yep, that's the app hash I'm using. Its mediation stack should always try to serve an Amazon ad first.

So I'm running the app again with literally the same build last night, and now it's not crashing. It was 100% repro before, now everything is running great.
Were any more back-end changes made?

For the sake of history and completeness, I'm attaching the full core dump from one of the crashes.
Amazon Test Ad Crash.txt

ok, thanks. No, there weren't any other back end changes.
I still want to get an Amazon ad :)
Will update If I get this crash. Maybe it has to do with the specific demand returned by Amazon.

Could be - thanks for continuing to follow up, it's much appreciated.
Do you have any comments on the modified contents of the customEventInfo dictionary in commit 9f5fc84, or the commenting-out of the use of the location-specific members of said dictionary in your custom event adapter code?

Hi,
Yes, the modified contents needs to be updated in all custom events (it's a bug it wasn't). This will be updated next week.
Commenting the location specific data was done on purpose as we expect each SDK to take the location by itself if it's available. We may change this again in the future.

Ok, thanks for the info! In the future, changes to customEventInfo would be worth mentioning in the release notes, for those of us with our own custom event adapters.
Closing this issue since everything is now working properly.

Agreed, this will be done from now on starting from our next release.


ensure stdout is flushed, not buffered

Hopefully, this resolves #97

Looking at the test repo's output (see this workflow run), I think this does indeed solve #97. I'm going to merging this and move it up the release chain for more testing.


Jump to definition without opening new tab

When I run xd to jump to a definition Vlime will focus the buffer's window if one currently exists, otherwise it opens a new tab page.  I never use tabs in Vim and would ideally like it to open in the buffer I just left like Vim's <ctrl-]>, e.g.:

I'm in a window and have my cursor on something.
I press <localleader>xd.
It opens the xref window thing.
I press <cr> to pick one.
It opens the location in the same window from step 1.

Ideally I'd also like two extra things:

Close the xref window after I jump, to reduce clutter.
If there's only one result, don't bother opening the xref window and making me press return, just jump to that one result.

Is there some way I can implement this stuff myself, or would it require changing Vlime?

As of 0bc01d9, when jumping to source locations, Vlime will use existing windows by default. The xref window will still appear even though there's only one match, to allow the user to choose a preferred way of opening the source file (see :help vlime-mappings-xref).


asBase64Decoded does not work with URL-safe base64 string

Describe the bug
Certain Strings containing "~" when base64 ended end up with a "+" in the output base64, however this is not ideal when used in URLs, so there's a specific variant of Base64 that is URL safe, which results in a "-" in the output instead. The JDK includes Base64.getUrlEncoder() / Base64.getUrlDecoder() for this usecase.
When using AssertJ's asBase64Decoded, it fails since it's not allowing the URL safe variant.

assertj core version: 3.24.1
java version: 11, 19
test framework version: junit 5

Test case reproducing the bug
String base64data = Base64.getUrlEncoder().encodeToString("fo~o".getBytes());
assertThat(base64data).asBase64Decoded().asString().isEqualTo("fo~o");

Expected: Assertion pass
Actual:
java.lang.AssertionError:
Expecting "Zm9-bw==" to be a valid Base64 encoded string

Being the URL encoding different from the basic one, what if we would add URL-specific methods?

isBase64Url
asBase64UrlDecoded
asBase64UrlEncoded

@scordio I like your suggestion.

Thanks @bjansen for your feedback 🙂 I marked this issue as ideal for contribution.

OK, I'll try opening a PR then.

Thanks!


Default naming scheme, behavior for all concepts

Default names
This specification outlines the default naming convention for all concepts in composer.
All triggers, actions start with a default name but is user editable.
Bot
There is no default name for a Bot. User must explictly provide a name.
Dialog
There is no default name for a dialog. User must explicitly provide a name.
Trigger

Only Intent trigger has a name property.
Default name for a trigger will be its type name - e.g. IntentTrigger
In case there already exists a trigger with the name, add a number increment as suffix - e.g. IntentTrigger, IntentTrigger1, IntentTrigger2, ...
All trigger names within a specific dialog must be unique.
In case of intent trigger, the intent name will default to the trigger name.
If user renames the trigger in the trigger's property editor, then the intent name is also updated.
If user changes the intent name of a trigger in all-up view and/ or if this intent is used as a reference in another intent, then ask the user for the intended behavior -

Update all references to the new name
Keep reference as is

Input/ prompt actions

Default name for an input action will be its type name - e.g. NumberInput, TextInput, ChoiceInput, ConfirmInput, DateTimeInput, OAuthInput, AttachmentInput
In case there already exists an input action with that name, add a number increment as suffix - e.g. NumberInput, NumberInput1, NumberInput2, ...
All input names tied to a specific dialog must be unique.
The intent name for an input default to the input name.
The prompt LG template name defaults to the input name.
The unrecognized prompt LG template name defaults to <input-name>.unrecognized.prompt
The invalid prompt LG template name defaults to <input-name>.invalid.prompt
If user renames the input in the input's property editor, then the intent name is also updated, LG template names are also updated.
If user changes the intent name of an input in the all-up view or changes the LG template name tied to an input in the all-up view and/ or if the intent or lG template is used as a reference in another intent/ LG template, then ask the user for the intended behavior -

Update all references to the new name
Keep reference as is

SendActivity

Default name is sendActivity, add an increment as suffix to de-dupe - e.g. SendActivity, SendActivity1, SendActivity2, ...
The default LG template name is the name of the SendActivity action
If user renames the send activity in its property editor, then the LG template name is also updated.
If user changes the LG template name in all-up view and/ or if this LG template is used as a reference in other intents or templates, then ask the user for the intended behavior -

Update all references to the new name
Keep reference as is

All other actions

Name is available, defaults to ActionType, add a number increment suffix to de-dupe
These names are just for organizational purposes and readaibility and do not have any runtime implications.

Note: This impacts the current naming scheme used for LG templates in send activity action. With this change, we will no longer use our current naming scheme bfd-<ID>

@cwhitten
As i go over this again, i don't see this as a P0, even if, probably we should delivery lg\lu copy\paste\delete first and then get back to this. Here is the reason

No feature is blocked on this naming convention. Current id link between dialog node to lg\lu sections gave us a robust way to correlate already.

One of the design purpose when this spec is delivered to help correlate dialog nodes and lg\lu sections, which is not a problem now.

Readability.
This do helps readability, for sure.  But while adding readability, this spec will introduce a dependency of dialog context when doing id generation. For example, the lu intent name will be numbered and prefixed with the trigger name.

This kind of dependency  and tight relationship are
a)  unstable, and require lots of efforts to maintain, think about move action up and down, or copy\paste new actions in, or deletion, it requires lots of effort to make it right, or we are aligned to spec and become confusing
b) fragile. Even if we either be smart enough to make it right in a), if any bug cause a little mistach, then the entire bot could be in an inconsistent state, and hard to recover automatically.
So i would strongly prefer we keep a unique id that is not contextual and can correlate easily. If we want more readability , we can add prefex, which is actually no much difference with we did today, TextInput-123456.

If we really like the simplicity of HelloTrigger.TextInput1, I would suggest we do this after we support all lg\lu related change when copy\paste\delete first. Then this becomes a pure refactor.

If we bring this in immediately, it's refactor and ship features toghether, which is always hard and error-prone.

hey @boydc2014 I've asked @zhixzhan to handle the R9 changes we want to make associated with this here: https://github.com/microsoft/BotFramework-Composer/pull/2597
Moving this to R10. What is mossing is the renaming behavior, which may or may not make it in.

hey @boydc2014 I've asked @zhixzhan to handle the R9 changes we want to make associated with this here: #2597
Moving this to R10. What is missing is the renaming behavior, which may or may not make it in.

I think what this spec described is largely implemented, except two minor things

we didn't really named input using number like 1,2,3, we use an random id as suffix.
we didn't auto rename intents when user renaming trigger, which we triaged as too tricky to do.

so i think this one can claimed as complete in r10, let's create new issue in r11, if we still think something can be done in this area


improved cypress testing to cover most of use cases

@rexalex Is this ready? As I can see cypress tests are failing.

The kitkat tests should pass after this gets in https://github.com/eea/volto-eea-kitkat/tree/develop


Feat: Navigating to Search Bar using Cmd/Ctrl + L

As mentioned in the Readme, probably cmd + L would be a better thing to navigate yo the search bar
Even I faced the issue of navigating to the search bar corrrectly as different browsers have different location for their seach bar (maybe)
https://github.com/OthersideAI/self-operating-computer/issues/39#issuecomment-1836316268
If everyone approves, I would go ahead and implement this?

As far as I can think, there might be 2 implementations for this -

Change the promt asking it to use cmd + L for navigating to the search bar directly...
Change the promt to detect if it is a browser, and if it is a browser.. use pyautogui to press cmd + L

Whichever would be more accurate, idk..

@shubhexists Based on the wording of this section in the README:
We recognize that some operating system functions may be more efficiently executed with hotkeys such as entering the Browser Address bar using command + L rather than by simulating a mouse click at the correct XY location. We plan to make these improvements over time. However, it's important to note that many actions require the accurate selection of visual elements on the screen, necessitating precise XY mouse click locations. A primary focus of this project is to refine the accuracy of determining these click locations. We believe this is essential for achieving a fully self-operating computer in the current technological landscape.
It sounds like the primary vision of the project at the moment is to improve click accuracy. Something that the cursor will likely be doing a lot in this program is moving to the navigation bar in the browser. That is likely why cmd+L / ctrl+L hasn't yet been implemented.

Fine, Makes sense :/ We can not run away from the fact that accuracy is more important. These features can be implemented later...

I think that #8 essentially handles this by create a "command" key system for the prompt. I think this makes sense long term. The goal of this project is to allow multi-modal models to most exactly emulate the humans interaction with the computer. I still need to review #8

@shubhexists I'll close this now that this is implemented as a standard part of the project


FIX: TreeDropdownField Folder expansion

When viewing a Folder tree, an expansion icon was shown if the Folder had any children, but it should be restricted to children that are Folders. This has annoyed me for a long time :)
The Hierarchy::getChildrenAsUL() function requires a separate method for counting children, but this hasn't been accommodated in TreeDropdownField so numChildren calculations were wrong. I'm guessing the requirement for the separate counting function predates lazy-loading as now it should be quite efficient to just do $SomeListMethod()->count().
Propose that for SS4.0 the $numChildrenMethod argument in Hierarchy::getChildrenAsUL() is deprecated, and the total can then be simply calculated using $childrenMethod()->count(). Or maybe we could even sneak this in to 3.2?
Companion fix for CMS at https://github.com/silverstripe/silverstripe-cms/pull/1205

Hey core devs here's a demonstration of the bug:
Before: (nasty!)

After:

Github seems to be cutting out the loops so you may need to click through and reload to take it in.

@tractorcow any chance of a review on this one? I think it's a reasonably important UX bug and makes the CMS seem 'dodgy'.

thank @jonom for your fix. Really good work. :)

Thanks mate!

@tractorcow there is a related CMS PR for this at https://github.com/silverstripe/silverstripe-cms/pull/1205


Some tests misusing assertTrue for comparisons

assertTrue is not for comparing arguments, should use assertEqual for that.
The developer's intent of the test was to compare argument 1 with argument 2, which is not happening. Really what is happening is the test is passing because first argument is truthy. The correct method to use is assertEqual. more details
https://github.com/django-oscar/django-oscar/blob/c05353a76dfbb7ce0df744e97cf0e282c54cae56/tests/integration/catalogue/test_options.py#L35
I found this issue automatically, see other issues here

Not that is not what is happening at all, in fact this test just has a really bad and invalid failure message. Thanks for pointing out a fault in the tests though.

I've sent you a pull request to help you with this issue at: https://github.com/code-review-doctor/django-oscar/pull/6


Google change will break GAM authentication

The Google changes at:
https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html
are going to break GAM's default OAuth flow which is OOB. Desktop localhost auth is NOT deprecated and is the flow we should move to using by default. A few things:

on/after Feb 28, 2022 new GAM installs and projects will break when trying to use OOB. Existing projects, including the project GAM itself uses to create projects will break for OOB login flows in (Sept 2022?) about 6 months.
it's possible to tell existing GAM versions today to use localhost. You just need to create a file called oauthbrowser.txt in the GAM folder.
the current localhost implementation does NOT support headless machines like Google Cloud Shell where no browser is present on the client. It won't be possible to authenticate on those machines  The only workaround for those machines and current GAM versions will be to create the oauth2.txt credentials elsewhere and move the file(s) onto the headless machine.
I'm working on an update to GAM that will default to localhost and starting a web server / browser to pass the auth token. At the same time and for headless installations, GAM will allow you to copy the auth URL, perform the auth on a "headed" machine which has a browser and then copy the auth code back to the headless machine. There are going to be a few challenges with such a flow:

GAM used to use the localhost flow as the default. We stopped doing so because more and more users were running GAM on a headless Linux server or such AND because we found even on "headed" Windows laptops, it wasn't uncommon for a firewall on the device to block the localhost connection. These will continue to be a problem for us.
The current python implementation of localhost flow in run_local_server() starts the web server and (tries) to start a web browser then it waits forever for a HTTP request from the web browser. No browser on the device means we're stuck.

I'm currently working on a replacement to both OOB run_console() and localhost run_local_server() flows. This flow will attempt to start the web server and browser but will also prompt for the auth code to be entered directly into the GAM console. That way user can copy the auth URL to another "headed" machine, complete the auth there and paste the resulting auth code back into GAM console. However, in such a scenario we'll still need a web server running on the "headed" client to handle the redirect back to localhost Google will make and present the user with the auth code to be copied back to GAM. We can do two things here:

have a GAM command like gam oauth justlisten that simply starts a web server on localhost and handles the browser redirect at the end of the auth flow. It can then present the auth code to the user in the browser and GAM client for an easy copy/paste back to the headed server. This command should not do ANYTHING special with OAuth, client secrets, etc since it won't know any of the project details being setup on the headless server.
above solution may be challenging for some users to run. Corp policy may prevent running GAM binaries on the local machine or it may be a Chromebook without Crostini where GAM can't run. In those cases, we can direct the user that at the end of the auth flow they will be redirected to a localhost URL and loading that page will fail. They should copy that full URL and paste it back into GAM. GAM can then parse the redirect URL and extract the auth code from it. Not pretty but it removes the requirement from running GAM on the browser machine.

@taers232c FYI
Fun times. :-/

Still needs to be done:

catch bad auth codes / urls and reprompt user rather than giving up
catch potential errors
provide better messaging to users on auth process and what they can do when localhost does not respond in browser (that's expected for headless)
write a standalone command that will only start the web server and respond with code in browser to be copied to headless device. This should not depend on ANY local config files.

These updates were released with 6.17. Closing this.


Today vs Upcoming

Feature that sorts between events/tasks scheduled for today and upcoming events/task in the next 7 days

I think we would be doing this based on the date that is stored with each task

We are doing this as a today vs next 10 task list regardless of day.  This will show the date of a task with it and sort them, but not group them


Check on the processor which is also used by default Unicorn to fill …

The current code now checks if the automated publish trigger is enabled or not. But the unicorn code has another processor which fills the publish queue, the AddSyncedItemsToPublishQueue. So it should check this one to be consistent with Unicorn default behavior.

No longer relevant.


[Experiment] Add global test timeout to avoid regression.

This set a global timeout for each test to 5s, Individual test can be
marked with a longer timeout instead of marking with with
@pytest.mark.timeout(new_value), I introduce a @slow decorator and
marker.

it makes test easy to disable with "-m 'not slow'" on pytest CLI,
and
introduce the NAPARI_TESTING_TIMEOUT_SCALING env variable, so if you
want to double all timeouts as you are on a slower machine, you can set
it to '2'.

I expect for example to set it on macos and windows which on CI may be
quite slow.
Old description:
This set a global timeout for each test to 5s,
Individual test can be marked with a longer timeout with
@pytest.mark.timeout(new_value), but this should ensure that there is
no surprised regression of test ballooning to 15 to 30s like the theme
ones.
See #3404 for example.

I'm pretty sure some test are going to fail on CI, and will need to be marked as slow with higher timeout... but let's see.

I'm pretty sure some test are going to fail on CI, and will need to be marked as slow with higher timeout... but let's see.

🤞🏼

Updated description:
This set a global timeout for each test to 5s, Individual test can be
marked with a longer timeout instead of marking with with
`@pytest.mark.timeout(new_value)`, I introduce a `@slow` decorator and
marker.

1) it makes test easy to disable with "-m 'not slow'" on pytest CLI,
and
2) introduce the NAPARI_TESTING_TIMEOUT_SCALING env variable, so if you
want to double all timeouts as you are on a slower machine, you can set
it to '2'.

I expect for example to set it on macos and windows which on CI may be
quite slow.

That's a significant change, so i'm going to dismiss prior approval.

I'm wondering if it's worth it, there are many test that abort in teardown, and there is no way to tell pytest-timeout to let the teardown takes all its time.

How about using https://pytest-timeouts.readthedocs.io/en/latest/usage/

How about using https://pytest-timeouts.readthedocs.io/en/latest/usage/

Yeah that's one I'm looking into, but pytest-timeout (no s), was already a dependency,
plus pytest-timeout is maintained, pytest-timeouts is not anymore (last commit 2019).

plus pytest-timeout is maintained, pytest-timeouts is not anymore (last commit 2019).

hmmm true, well it does seem to provide what we need here.

#3416 tries pytest-timeouts .... and it seem to be incompatible with windows... so I think that put the lid on the coffin.


Look at more than just Trump

E.g. other specific people or topics or everything.

One idea is to create a script to loop through all twitter accounts (we may narrow down to important ones only, based on some kind of popularity metric) and try to figure out the impact or correlation of their tweets with stock prices. The end goal would be trying to find pattern similar to what you have observed with Trump's account.

I think it would be better to filter social media accounts for just politicians. You could also do some language analysis on news articles.

It could also look into tweets of people reacting to Trump's tweets, especially people who always respond to them. A nice example would be if Bloomberg were to respond to Trump's tweets.
Another idea could be to pick some articles based on keywords on his Tweets, search articles based on them, and base decisions on the results.
I'm new to this, so any way to contribute would be a great start for me! :)

It's been a while since this bot was created and I intend to revive it as a general social media trading bot. There are paid Discord servers as well as paid Twitter profiles which send out buy/sell orders to subscribers. I want to attach bots to these accounts to automate their trades. In terms of political figures, as waseem mentioned, I don't think using their Twitter profiles would be adequate. There is an account called @unusual_whales who gathers information on politicians' stock market trades. Accessing his database would be more accurate and much easier.


Deprecate distutils

Replace:
distutils.spawn.find_executable with shutil.which
distutils.version.StrictVersion with packaging.version.Version
distutils.util.strtobool with codechecker_common.util.strtobool
as distutils is not available in Ubuntu noble.

Thank you for the patch, this is a very useful change!
Sorry, another patch had been merged which caused some merge conflicts. If you fix those, then this PR can merged. Please, also remove the "pylint disable" and "TODO" comments above the changed lines.
Thanks again!


Trigger kuberhealthy checks as part of a CI pipeline

Thanks for the nice project, I also think that those kind of synthetic tests will grow more and more to as they cannot really be replaced by simple metrics and as k8s admins need to know when some a basic use case is broken.
Describe the feature you would like and why you want it
I was thinking on how we could include kuberhealthy in our CI pipeline for our Kubernetes clusters which deploys almost everything including namespaces with rolebindings, the nvidia GPU operator or also a CSI plugin.
My goal is to have an automated rollout after a merged pull request from INT to PROD with end-to-end tests in-between as a gatekeeper for the PROD deployment.
Additional context
The simplified setup consists of a DEV, an INT and a PROD Kubernetes cluster.
The development flow would look like this:

A feature branch in our version control system is created
All changes on that branch are directly deployed in our DEV environment
Development is done including tests on DEV
Pull request from feature branch to master is opened
Pull request compares and applies the changes in a dry run to INT but doesn't deploy on INT
Pull request is reviewed, approved and merged to master
CI sees new changes on master and triggers INT deployment
CI INT deployment runs, finishes applying and triggers all kuberhealthy checks blocking (fails when kuberhealthy fails)
Depending on the kuberhealthy check result the PROD rollout is triggered

This is only a simplified idea, I would also have a pretty clean DEV environment also running kuberhealthy, but the important points for me are to

deploy automatically to PROD with no manual intervention
and have kuberhealthy as a gatekeeper for this.

Any ideas if this could be done already now (didn't find anything for triggering all checks)?
Thanks!

Related to #528

I am closing this one as a duplicate.  Please watch #528 for details because @joshulyne is working on this one soon I think.


Callback when video has loaded

I'm trying to display my own loading indicator while BigPicture is loading an embedded video. To do this I'd like to be able to define a callback which is called when BigPicture displays the viewer (i.e. the embed has finished loading). This could either be a simple callback or a promise.
Ideally I'd also like an option to disable the built in loading spinner.

Hi Cameron, I've just added options to the script that should allow you to do this. Please take a look at the readme, update to the latest version, and let me know if you run into any trouble.

Brilliant - looks good to me! Thanks @henrygd 🥇


Error writing to databricks UC volume

Environment
Databricks Runtime 14.3 LTS ML
databricks-automl-runtime==0.2.20
databricks-cli==0.18.0
databricks-feature-engineering==0.2.1
databricks-sdk==0.28.0
mosaicml-streaming==0.7.6
To reproduce

import numpy as np
from PIL import Image
from shutil import rmtree
from uuid import uuid4
from streaming import MDSWriter

# Local or remote directory path to store the output compressed files.
out_root = "dbfs:/Volumes/**name.schema/volume_name**/test"
#out_root = '/dbfs/FileStore/test'
os.environ['DATABRICKS_HOST'] = 'https://**address**.azuredatabricks.net'
os.environ['DATABRICKS_TOKEN'] = dbutils.secrets.get(..... )
# A dictionary of input fields to an Encoder/Decoder type
columns = {
    'uuid': 'str',
    'img': 'jpeg',
    'clf': 'int'
}

# Compression algorithm name
compression = 'zstd'

# Generate random images and classes
samples = [
    {
        'uuid': str(uuid4()),
        'img': Image.fromarray(np.random.randint(0, 256, (32, 48, 3), np.uint8)),
        'clf': np.random.randint(10),
    }
    for _ in range(1000)
]

# Use `MDSWriter` to iterate through the input data and write to a collection of `.mds` files.
with MDSWriter(out=out_root, columns=columns, compression=compression) as out:
    for sample in samples:
        out.write(sample)

Error trace:

exception calling callback for <Future at 0x7faba572cbb0 state=finished raised InvalidParameterValue>
Traceback (most recent call last):
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 342, in _invoke_callbacks
    callback(self)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/format/base/writer.py", line 330, in exception_callback
    raise exception
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/storage/upload.py", line 877, in upload_file
    _upload_file()
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/util.py", line 525, in new_func
    raise e
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/util.py", line 521, in new_func
    return func(*args, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/storage/upload.py", line 875, in _upload_file
    self.client.files.upload(remote_filename_wo_prefix, f)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/service/files.py", line 968, in upload
    self._api.do('PUT',
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/core.py", line 131, in do
    response = retryable(self._perform)(method,
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper
    raise err
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper
    return func(*args, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/core.py", line 242, in _perform
    raise self._make_nicer_error(response=response, **payload) from None
databricks.sdk.errors.platform.InvalidParameterValue: GetVolume full_name_arg is not a valid name
exception calling callback for <Future at 0x7faba5783d90 state=finished raised InvalidParameterValue>
Traceback (most recent call last):
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 342, in _invoke_callbacks
    callback(self)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/format/base/writer.py", line 330, in exception_callback
    raise exception
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/storage/upload.py", line 877, in upload_file
    _upload_file()
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/util.py", line 525, in new_func
    raise e
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/util.py", line 521, in new_func
    return func(*args, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/streaming/base/storage/upload.py", line 875, in _upload_file
    self.client.files.upload(remote_filename_wo_prefix, f)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/service/files.py", line 968, in upload
    self._api.do('PUT',
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/core.py", line 131, in do
    response = retryable(self._perform)(method,
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper
    raise err
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper
    return func(*args, **kwargs)
  File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/databricks/sdk/core.py", line 242, in _perform
    raise self._make_nicer_error(response=response, **payload) from None
databricks.sdk.errors.platform.InvalidParameterValue: GetVolume full_name_arg is not a valid name

Writing to dbfs works fine.

Thanks for raising this @JK87iab! @XiaohanZhangCMU could you take a look?

Thank you @JK87iab !
I tried to reproduce the error on a single-assigned cluster with 14.3 LTS ML using the script you posted above. But it turns out working fine, without any error message.
I wonder if it is something related to your dbutils credentials, are you able to run a file-uploading script like below?
from streaming.base.storage import CloudUploader
cu = CloudUploader.get( out=("/tmp/", "dbfs:/Volumes/main/mosaic_hackathon/managed-volume/test_issue703"), exist_ok=True)
!echo "Some text here." > /tmp/myfile.txt
cu.upload_file("myfile.txt")

This is the content of my notebook:
%pip install databricks-automl-runtime==0.2.20
%pip install databricks-cli==0.18.0
%pip install databricks-feature-engineering==0.2.1
%pip install databricks-sdk==0.28.0
%pip install mosaicml-streaming==0.7.6
dbutils.library.restartPython()

import numpy as np
from PIL import Image
from shutil import rmtree
from uuid import uuid4
from streaming import MDSWriter

# Local or remote directory path to store the output compressed files.
out_root = "dbfs:/Volumes/main/mosaic_hackathon/managed-volume/test_issue703"
#out_root = '/dbfs/FileStore/test'
# os.environ['DATABRICKS_HOST'] = 'https://**address**.azuredatabricks.net'
# os.environ['DATABRICKS_TOKEN'] = dbutils.secrets.get(..... )
# A dictionary of input fields to an Encoder/Decoder type
columns = {
    'uuid': 'str',
    'img': 'jpeg',
    'clf': 'int'
}

# Compression algorithm name
compression = 'zstd'

# Generate random images and classes
samples = [
    {
        'uuid': str(uuid4()),
        'img': Image.fromarray(np.random.randint(0, 256, (32, 48, 3), np.uint8)),
        'clf': np.random.randint(10),
    }
    for _ in range(1000)
]

# Use `MDSWriter` to iterate through the input data and write to a collection of `.mds` files.
with MDSWriter(out=out_root, columns=columns, compression=compression) as out:
    for sample in samples:
        out.write(sample)

Hey thx for the quick response. Works on my end now too.
I think I was the bug. Used a dot (.) instead of / to write the path to the volume in UC.
thx

Thanks! closing this issue.


Incorporate Jaclyn's edits (suggestions and clarifications) for 2019

Simply ran out of time to get to them in 2018. I have them on a printout.

Any chance you can scan them and attach them to this issue?

Good idea, though they're just on a printout of the whole assessment, so not really practical. They weren't major, mainly for clarity; I can easily do them for next time (especially as she's reviewing again).

Thanks @andrew-edwards that will be great.

I started looking at these a few weeks ago. Mainly minor (think I did some), but one question was why is the spawning biomass females only (in herring they do males and females)?
Similarly, should probably clarify somewhere that it's a one-sex model because [presumably] there is little difference between male and female growth? A reference would be nice.

Yes on sex model - does our reference to the growth work done in the 2011 assessment say anything about growth by sex (or only through time)?  Below is what it says and their justification.  I think this stuff got slowly taken out year after year as the document grew due to other things.  We cite this growth work, but that's about it.

[In both 2011 assessment models and in models used for management prior to the 2006
stock assessment, variability in length-at-age was included in stock assessments via the
calculation of empirical weight-at-age. In the 2006 and subsequent assessments that attempted to
estimate the parameters describing a parametric growth curve, strong patterns have been
identified in the observed data indicating sexually dimorphic and temporally variable growth.
Synthesis models in recent years have not explicitly accounted for sex-specific patterns (although
they have been documented repeatedly) but have allowed for the dramatic decline in maximum
size and corresponding increase in growth rate observed in the data (Figure 7). Parametric
growth models fit externally to data collected prior to 1990 and afterward show the same
dramatically different rates of growth for both sexes that has been estimated inside the SS model
in recent years (Figure 14). Hake show very rapid growth at younger ages, clearly evident in
data partitioned into seasons within each year (Figures 15 and 16). The trajectories of individual
cohorts also vary greatly, as has been documented in previous assessments.
In aggregate, these patterns result in a great amount of process error for length at age
relative to commonly employed parametric growth models. This means that even complex
approaches to modeling growth (and therefore fitting to length or age-at-length data explicitly)
will have great difficulty in making predictions that mimic the observed data. This has been
particularly evident in the residuals to the length-frequency data from recent SS models. We
investigated models that allow for a high degree of complexity in the growth process and fit to
length and age-at-length data in preparation for this assessment, but poor residual patterns
persisted in all cases (Figure 17).]

We typically do SSB based on just females, the only time most folks consider SSB based on males and females (for us anyway) is with rockfish or other species where's males nest guard.  To do a male and female SSB we would need a male maturity curve, and the NWFSC just started collecting some testes in 2018.

Do @kellijohnson or @aaronmberger want to write a sentence or two summarising the above, thanks!
I'll skim through the rest of Jaclyn's edits if time, but that was the only major one (and I did some a while back), and she said there was nothing major....

I can. What section or line will it go in?

The executive summary says we retain the structural from of the 2018 model so I don't think we need more specifics there. So I was thinking just a couple of sentences in section 3.2 - Description of the base model, covering that it is a single sex model.
RE: "because [presumably] there is little difference between male and female growth? A reference would be nice." - it actually is the opposite, the 2011 assessment found significant variation in growth through time and by sex, but they reference the incredible amount of variation (and time and time again bad residual patterns for all sorts of growth parameterizations) as why the wt-at-age approach was used (parsimony really) - so could cite the 2011 assessment document.  - much of this is already in section 2.3.4 though.  So maybe just add the single sex model part....

maybe just after first sentence of 3.2.  - mention that the model is a gender and fleet aggregated, single-area statistical catch-at-age model - or something like that and call it good.

Okay I am going to add the following to 3.2 after the first sentence:
The statistical-catch-at-age model assumes that the Pacific Hake population is a single coast-wide stock subject to one fleet with similar male and female population dynamics.

Changed two words.... we must be getting close to the deadline b/c getting nit picky...
The statistical-catch-at-age model assumes that the Pacific Hake population is a single coast-wide stock subject to one aggregated fleet with combined male and female population dynamics.

Thanks Kelli.
Couple of Jaclyn's points for clarification (someone want to pick this off as I'm not quite sure of the answers, thanks):

page 7, line 6:  "Discard from all fisheries is estimated to be less than 1% of landings in recent years." Clarify which fisheries (esp. in context of preceding sentence).

page 10: " The subsequent decline is from the 2010 year class surpassing the age at which gains in
weight from growth are greater than the loss in weight from natural mortality." Clarify sentence, and delete 'natural'.  I agree... think deleting natural is enough.

Actually, that's it. The others (based on 2018 assesment) are done anyway during the updating...

Re the equilibrium question, I just talked to Ian and he confirmed my thoughts on this.  It is really quite convoluted (not easy to describe) due to early year (1946-1969) rec devs.  SS defines equilibrium virgin biomass as 'true' equilibrium (i.e., not adjustments using early year rec devs) despite putting those values in year 1964.  So let's not address the equilibrium thing now in the document, but we (or I) can speak to it at the meeting if someone brings it up.

I'll work on #1.
RE #2, can we do "The subsequent decline from 2014 to 2016 is primarily from the 2010 year class surpassing the age at which gains in weight from growth are greater than the loss in weight from mortality." I agree with removing natural mortality, though from Table 25 M is the most important but F in there too of course.

Regarding # 1, can't we just change it to
"Discard from all fisheries, including those that don't target hake, is estimated to be less than 1% of landings in recent years."

Thanks - I think both are good. @aaronmberger want to do them. (Should 'loss' be losses'?).

changed to: "The subsequent decline from 2014 to 2016 is primarily
from the 2010 year class surpassing the age at which gains in weight from growth
are greater than the loss in weight from mortality."  so I think loss works here

Andy said that's it so closing...


OWLS-101056 - fixed log msg to avoid test failure in the case of REST call failure

https://build.weblogick8s.org:8443/job/weblogic-kubernetes-operator-kind-new/11588/
Running entire kind parallel run
https://build.weblogick8s.org:8443/job/weblogic-kubernetes-operator-kind-new/11589

parallel run results are intermittent and not related to this fix


Can't mockk Javalin Context in 5.x

With Javalin 4.x following code compiled and good:
    val ctx = Context(
      mockk { every { getHeader(AUTHORIZATION) } returns null },
      mockk {}
    )

With Javalin 5.x, I'm getting error like Interface Context does not have constructors.
After changing to:
    val ctx = JavalinServletContext(
      mockk { every { header(AUTHORIZATION) } returns null },
      mockk {}
    )

I get "header" is unresolved.
Please advise, how to implement the same thing with Javalin 5.x.

Did you try mocking Context rather than JavalinServletContext ? The latter is an internal class.

Seems to be able to solved as follows:
    val context = JavalinServletContext(
      cfg = JavalinConfig(),
      req = mockk { every { getHeader(AUTHORIZATION) } returns null },
      res = mockk()
    )

No, I didn't try mocking Context further. Probably worth to try a bit more.

In Javalin 5.x context is an interface to hide internal implementation and simplify mocking, you should definitely go for it instead of the servlet implementation.


Please update

Error: The delayMs option was removed from express-rate-limit v3.
Also I can see mtr.sh is resolving domains but you haven't updated it here.

Repo will be brought up to date with the currently running live version on mtr.sh in the next few days

Code base has been updated with the live version from https://mtr.sh/, update of express-rate-limit to v3 and other dependencies will be tracked in #25.


Don't hard code cluster dns domain!

Please add clusterDomain: xxx.local to your values.yaml and use it to setup kamaji-etcd and the certificates definition.
Change this to certmanager_certificate.yaml
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  labels:
    {{- $data := . | mustMergeOverwrite (dict "component" "issuer") -}}
    {{- include "kamaji.labels" $data | nindent 4 }}
  name: kamaji-selfsigned-issuer
  namespace: {{ .Release.Namespace }}
spec:
  selfSigned: {}
root@k8s-02:~/kamaji/kamaji/charts/kamaji/templates# more certmanager_certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  labels:
    {{- $data := . | mustMergeOverwrite (dict "component" "certificate") -}}
    {{- include "kamaji.labels" $data | nindent 4 }}
  name: {{ include "kamaji.certificateName" . }}
  namespace: {{ .Release.Namespace }}
spec:
  dnsNames:
    - {{ include "kamaji.webhookServiceName" . }}.{{ .Release.Namespace }}.svc
    - {{ include "kamaji.webhookServiceName" . }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}
  issuerRef:
    kind: Issuer
    name: kamaji-selfsigned-issuer
  secretName: {{ include "kamaji.webhookSecretName" . }}

Thanks for the report @rossbachp, are you up to work on this?

git diff templates/certmanager_certificate.yaml
diff --git a/charts/kamaji/templates/certmanager_certificate.yaml b/charts/kamaji/templates/certmanager_certificate.yaml
index 2f310b1..9c3300c 100644
--- a/charts/kamaji/templates/certmanager_certificate.yaml
+++ b/charts/kamaji/templates/certmanager_certificate.yaml
@@ -9,8 +9,8 @@ metadata:
spec:
dnsNames:
- {{ include "kamaji.webhookServiceName" . }}.{{ .Release.Namespace }}.svc

{{ include "kamaji.webhookServiceName" . }}.{{ .Release.Namespace }}.svc.cluster.local

{{ include "kamaji.webhookServiceName" . }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}
issuerRef:
kind: Issuer
name: kamaji-selfsigned-issuer

secretName: {{ include "kamaji.webhookSecretName" . }}
\ No newline at end of file

secretName: {{ include "kamaji.webhookSecretName" . }}

git diff values.yaml
diff --git a/charts/kamaji/values.yaml b/charts/kamaji/values.yaml
index 21b529e..e0046e4 100644
--- a/charts/kamaji/values.yaml
+++ b/charts/kamaji/values.yaml
@@ -98,6 +98,7 @@ loggingDevel:
-- Specify the default DataStore name for the Kamaji instance.
defaultDatastoreName: default
+clusterDomain: cluster.local
kamaji-etcd:
deploy: true
fullnameOverride: kamaji-etcd
@@ -108,4 +109,4 @@ kamaji-etcd:
-- Disable the analytics traces collection
telemetry:
disabled: false
\ No newline at end of file
+

I must install the Crds manually:
k apply -f crds
helm install kamaji . --namespace kamaji-system --create-namespace --values kamaji-values.yaml
helm template -s templates/certmanager_issuer.yaml kamaji . --namespace kamaji-system --create-namespace --values kamaji-values.yaml | kubectl -n kamaji-system apply -f -

Also must create the DataStore manually:
helm template -s charts/kamaji-etcd/templates/etcd_datastore.yaml kamaji . --namespace kamaji-system --create-namespace --values kamaji-values.yaml | kubectl apply -f -

@rossbachp please, don't use the issue messages as a sort of memo or DMs, we need to decrease them as much as we can the spam and report back just relevant information.
Upon Kamaji uninstallation some chicken/egg problems arise with finalizers, validating/mutating webhooks, etc.
You can remove manually the finalizers from the objects unable to get deleted since Kamaji is not running anymore, as well as delete CRDs since Helm installs them only on the Install event (it's a well-known limitation).
Furthermore, I would suggest giving a try removing all the cluster.local suffix from the Certificate SAN, I don't remember the internal quirks of Kubernetes Dynamic Admission Controllers but resolution should start first with the third level domain name (${svc_name}.${svc_namespace}.svc) rather than the fully qualified one.

@rossbachp just for reference, when a broken installation of Kamaji: https://kamaji.clastix.io/getting-started/#cleanup

I miss a the information how I delete the datastore manifest. Only the deletion of CRDS are explained.

@rossbachp any update on this contribution, besides the bug report?

Unfortunately, we were not able to reproduce the situation, and the bug can be closed. Sorry, but we had to change the datastore. Can I clean the etcd datastore of an failed tenant?

Yes, you can, although these actions must be performed in order:

Delete all the TCPs using the Data store, and if deletion stucks, remove the finalizers manually
Uninstall the Helm release for the Datastore, if managed via Helm
In case of stuck deletion, delete all the etcd objects, such as STS, PVC, PV, etc, and finally Datastore instance

I give it a try!
Many thanks.


Remove install instructions for Z under the RHEL and SLES

Release notes contain this: As of EE 2.1, Docker has deprecated support for IBM Z (s390x). Refer to the Docker Compatibility Matrix for detailed compatibility information.