x/tools/cmd/gomvpkg: package not changed in files with different OS build tags

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.10.2 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
GOHOSTARCH="amd64"
GOHOSTOS="darwin"

What did you do?
Ran gomvpkg -from github.com/hashicorp/consul/agent/proxy -to github.com/hashicorp/consul/agent/proxyprocess -vcs_mv_cmd "git mv {{.Src}} {{.Dst}}"
If you checkout the current head of Consul 4d658f34cfcb5c2e0b29ae5103e923872bddcaa7 and run the same command you should see the same issue.
What did you expect to see?
All files moved to the new package name and their package declarations updates
What did you see instead?
All files moved but two files with build tags that didn't match my host OS were not modified and so kept the old package proxy declaration.
The two files are https://github.com/hashicorp/consul/blob/4d658f34cfcb5c2e0b29ae5103e923872bddcaa7/agent/proxy/process_windows.go#L1-L3
// +build windows

package proxy

and
https://github.com/hashicorp/consul/blob/4d658f34cfcb5c2e0b29ae5103e923872bddcaa7/agent/proxy/exitstatus_other.go#L1-L3
// +build !darwin,!linux,!windows

package proxy

This is unpleasant as build and test pass locally still naturally!
This may well be a "known" issue caused by something in the tool chain that is hard to work around, however I don't see an open issue for it and it's certainly surprising to me that this wouldn't work for files with os-dependent build tags. Even if the tooling makes it hard to fix those for some reason, it would be better to at least detect that happening and warn the user to update manually.
Thanks!

I notice on reading the help docs more closely there is a caveat which appears to apply here:

gomvpkg will not always be able to rename imports when a package's name is changed.
Import statements may want further cleanup.

It feels like this case is common enough that we could at least be more specific about it there?


H6147 Works As Expected

Just wanted to leave this here for anyone else!

Nice

Also works with H617E Thanks!


textAlign "center" doesn't center the text

Looks to be because of this line
https://github.com/dengyin2000/dynamic_widget/blob/7bf660acf23551175f92e53975fd376b56454e7b/lib/dynamic_widget/utils.dart#L15

Looks to be because of this line
https://github.com/dengyin2000/dynamic_widget/blob/7bf660acf23551175f92e53975fd376b56454e7b/lib/dynamic_widget/utils.dart#L15

@KieranLafferty   thanks.  the bug was fixed.


Make AbstractIterable serializable (in the java.io sense)

Iterable enumerations and comprehensions are compiled into anonymous subclasses of AbstractIterable. Of course those classes do have names really and this can make them hard to deserialize (using java.io serialization), because it can introduce a non-obvious dependency on a  module which adds an iterable to a graph of objects to be serialized. For example in
serialize({1, 2, 3})

we end up with a class defined in the module that's calling serialize being part of the serialized stream. That's not obvious (especially in the case of a stream of constants, for example).
This situation could be avoided if such streams were materialized at serialization time, into a Sequential, for example.

I think this just boils down to adding
Object writeReplace() throws ObjectStreamException {
    return sequence();
}

to AbstractIterable, because I don't think the compiler ever generates a reference to the anonymous class name.


Reference documentation for what are the possible contents of an unmanaged cluster creation configuration file

Description
Discussed in this thread in Slack
https://kubernetes.slack.com/archives/C02GY94A8KT/p1645319008410969
The CLI says you can pass in a config file on unmanaged cluster creation but there is no documentation on what can be in the config file
 -f, --config string                        A config file describing how to create the Tanzu environment
According to @stmcginnis you can use:
tanzu uc config
But this did not work for me. We need some sort of doc for what can go in that configuration file

Did you find this, and are you saying it did not create a config file for you?
❯ tanzu uc configure -h

Generate a configuration file that can be used when running:
tanzu unmanaged-cluster create -f <config-file-name>.yaml. Configure generates
a config file injected with default values. When flags are specified
(e.g. --cni) the flag value is respected in the overridden config.

Usage:
  tanzu unmanaged-cluster configure <cluster name> [flags]

Flags:
  -c, --cni string            The CNI to deploy. Default is 'antrea'
  -f, --config string         Configuration file for unmanaged cluster creation
  -h, --help                  help for configure
      --pod-cidr string       The CIDR to use for Pod IP addresses. Default and format is '<IP_ADDRESS>/16'
      --provider string       The infrastructure provider to use for cluster creation. Default is 'kind'
      --service-cidr string   The CIDR to use for Service IP addresses. Default and format is '<IP_ADDRESS>/16'
  -t, --tkr string            The Tanzu Kubernetes Release location.
      --tty-disable           Disable log stylization and emojis

Global Flags:
      --log-file string   Log file path
  -v, --verbose int32     Number for the log level verbosity(0-9)

Can you please review https://tanzucommunityedition.io/docs/latest/ref-unmanaged-cluster/#custom-configuration, which talks about how to create a custom config file (same as @stmcginnis mentioned above)

No sorry
That does not work because it doesn't list all the possible configuration values, their effects, and their defaults. It does generate some of this but I am looking for all the reference documentation and possible values.
For example, here is all the documentation for what can go in a postgresql.conf file
https://www.postgresql.org/docs/current/runtime-config.html
And here is the documentation for the values for just the resource values section of the configuration file
https://www.postgresql.org/docs/current/runtime-config-resource.html
While I realize we won't have as many variables in our file - I can not find any link for what are the possible values.
The link above (and the page in the current doc) only says it will generate a config file. That doesn't explain what should actually go in the config file and what changing the values mean.
I am asking for doc to explain what is actually IN the file. Does that make sense?

@stmcginnis please reopen - just saw the notification on messages for this

@thesteve0 can you give an example of what's missing from the unmanaged-cluster reference documentation here: https://github.com/vmware-tanzu/community-edition/blob/main/docs/site/content/docs/ref-unmanaged-cluster.md
Currently, we do note the different fields (including a full config file), flags, and what a user could enter to achieve different goals. In my opinion, this is sufficient for users needs.


Initial migration from Egeria core

Initial migration of connector code from Egeria core

@jmertic can we setup the same controls we have on Egeria core on this repository as well?
Things like:

automated Jenkins build
review as optional

Thanks!

@cmgrote Added the DCO check
@schannamallu Can you help with getting this integrated into the Jenkins build?

@jmertic, I will integrate them. Will update once done.

Recheck

Recheck

Recheck

Recheck

FYI for future changes to build infrastructure we should use the odpi/ci-management project
Note - we'll need the igc docker image created somewhere, so this project may also need that element added to, which can then be consumed by our helm charts

Note - we'll need the igc docker image created somewhere, so this project may also need that element added to, which can then be consumed by our helm charts

The IGC docker image cannot be created through ODPi build infrastructure — it requires commercially licensed software and entitlements.  We could move the helm template for it here, but the image itself would always need to remain external.

I wasn't thinking about IGC itself (which is all commercial) but the proxy we use as part of the VDC chart, ie server chassis with the igc connector/event mapper loaded pointed to a remote repo.
The VDC chart will be dependent on that image - albeit only when IGC is being used.
If we think that includes proprietary code too, then that can't be built here, but I had thought the proxy would be open source only? Also if it is proprietary it would be great if we could persuade the proprotery owners to support the required images....

Gotcha -- indeed the proxy will be fine: but my intention would be to just re-use the Egeria core (OMAG Server Platform) image and inject the connector at startup; rather than needing to maintain different images for every potential combination of connectors...  I've added an issue to the other repo (for now) to track that migration: all dependent on us agreeing the ZIP vs JAR approach first, though, so probably worth discussing on the Tuesday call 😄

@cmgrote, @jmertic - Integrated with the jenkins builds. Can you check and confirm.

Thanks @schannamallu !  I saw it working on an earlier PR 👍


Possible bug in interpolation.cpp?

here in lch lerp (src/interpolation.cpp, line 69)
shouldn't it be 360.f (instead of 380.f)?
thank you.
lch_t lerp( const lch_t& lch1, const lch_t& lch2, const float t )
{
    col_t delta = lch2 - lch1;

    //  move along shortest path (wrap [0; 360])
    if ( delta.z > 180.f ) {
        delta.z -= 360.f;
    } else if ( delta.z < - 180.f ) {
        delta.z += 380.f;  // <---------- should this be 360?
    }

    auto interp = lch1 + t * delta;

    //  project back to [0; 360]
    interp.z = std::fmod( interp.z + 360.f, 360.f );
    return static_cast<lch_t>( interp );
}

What a great catch, fascinating how long this was allowed to stay in there! ;) Thanks for reporting the issue. Will fix in upcoming patch.

Fixed with #17/[v3.0.1].


npm install error

Hi, I'm using vite.js with react and I have an issue with installing this package, this is the error:
# npm resolution error report

2022-09-05T09:50:55.491Z

While resolving<EMAIL_ADDRESS>Found<EMAIL_ADDRESS>node_modules/react
<EMAIL_ADDRESS>from the root project

Could not resolve dependency:
peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>node_modules/react-darkreader
  react-darkreader@"*" from the root project

Fix the upstream dependency conflict, or retry
this command with --force, or --legacy-peer-deps
to accept an incorrect (and potentially broken) dependency resolution.

Raw JSON explanation object:

{
  "code": "ERESOLVE",
  "current": {
    "name": "react",
    "version": "18.2.0",
    "whileInstalling": {
      "name": "my-vue-app",
      "version": "0.0.0",
      "path": "/home/jckpt/js/league/my-vue-app"
    },
    "location": "node_modules/react",
    "isWorkspace": false,
    "dependents": [
      {
        "type": "prod",
        "name": "react",
        "spec": "^18.2.0",
        "from": {
          "location": "/home/jckpt/js/league/my-vue-app"
        }
      }
    ]
  },
  "currentEdge": {
    "type": "prod",
    "name": "react",
    "spec": "^18.2.0",
    "from": {
      "location": "/home/jckpt/js/league/my-vue-app"
    }
  },
  "edge": {
    "type": "peer",
    "name": "react",
    "spec": "^16.8.6",
    "error": "INVALID",
    "from": {
      "name": "react-darkreader",
      "version": "1.5.6",
      "whileInstalling": {
        "name": "my-vue-app",
        "version": "0.0.0",
        "path": "/home/jckpt/js/league/my-vue-app"
      },
      "location": "node_modules/react-darkreader",
      "isWorkspace": false,
      "dependents": [
        {
          "type": "prod",
          "name": "react-darkreader",
          "spec": "*",
          "from": {
            "location": "/home/jckpt/js/league/my-vue-app"
          }
        }
      ]
    }
  },
  "strictPeerDeps": false,
  "force": false
}

I've used npm i react-darkreader --force and the package works correctly


ci: track latest lima tag

Issue #, if available:
Description of changes:

Switch back to latest Lima now that upstream commit got released

Testing done:

[x] I've reviewed the guidance in CONTRIBUTING.md

License Acceptance
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Overriding merge since this change doesn't get tested by these checks anyway


Cant install lua-requests because xml dependency wont install in lua 5.4

i cant install lua-requests because xml cant install in lua5.4
running sudo luarocks install lua-requests gives the following error:
Installing https://luarocks.org/lua-requests-1.2-2.src.rock
Missing dependencies for lua-requests 1.2-2:
   xml (not installed)
   luasec >= 0.5.1, < 0.8-1 (not installed)

lua-requests 1.2-2 depends on lua >= 5.1 (5.4-1 provided by VM)
lua-requests 1.2-2 depends on luasocket (3.0rc1-2 installed)
lua-requests 1.2-2 depends on md5 (1.3-1 installed)
lua-requests 1.2-2 depends on lua-cjson (<IP_ADDRESS>-1 installed)
lua-requests 1.2-2 depends on xml (not installed)

Error: Could not satisfy dependency xml: No results matching query were found for Lua 5.4.

OS : Arch linux

Same prob here :(

I also have same problem on Mac OS.

@BugAlpha  @Lanttcat @amirhossein-ka  Hi, if you do not need handle with xml , just copy the src/requests.lua with your code. and then comment out https://github.com/JakobGreen/lua-requests/blob/master/src/requests.lua#L8 and https://github.com/JakobGreen/lua-requests/blob/master/src/requests.lua#L114. of course, you need install the dependency manually.
FYI.
luarocks install luasocket
luarocks install luasec
luarocks install lua-cjson
luarocks install md5


example docker images would be great :-)

FROM python:2.7
ENV PYTHONBUFFERED 1
RUN pip install ncolony & ncolony add "python -m SimpleHTTPServer"
ENTRYPOINT ['/usr/bin/python', 'ncolony']

or similar that actually works -- if you have an easy copy+paste example it will make it a lot easier for people to grab this instead of dumb-init when they are in a rush

Sounds good!
A little experimentation shows this works -- is this what you mean?
  I will use that right away :-)
(our shiny new gitlab-ci worker needs to scale up now that everybody is using it and I'd like to use ncolony)


Disable the TTY for CircleCI so it does not print progress bars

This didn't work so I'm closing it.


Fix ros2-master

fix compilation error due to https://github.com/rst-tu-dortmund/teb_local_planner/pull/280

Yes, my mistake. I forgot to test compilation after changing directory name. But maybe we should not just comment out this line
install(DIRECTORY launch DESTINATION share/${PROJECT_NAME})

But instead just change it to
install(DIRECTORY params DESTINATION share/${PROJECT_NAME})

Changes for install(DIRECTORY params DESTINATION share/${PROJECT_NAME}) in last commit

Redundant with (included in) : https://github.com/rst-tu-dortmund/teb_local_planner/pull/282


Sound in background

You can't transmit or receive sound when the app is minimized.

This has already been fixed, a new build will be out soon.


Load a scene0, the training only generates a gt video, no reconstruction video is generated

Load a scene0, and then train for 2 hours. When it is finally generated, only a gt video is generated, and no reconstructed video is generated.
The error message is "invail buffer size, packet size 65536 < expected frame_size 3231453"

The command to run is this：
python tools/train.py --config_file configs/omnire.yaml dataset=nuscenes/3cams data.scene_idx=0 data.start_timestep=0 data.end_timestep=-1

Hi @qiuqc1, I've tested the exact same command recently and it worked well. I think this issue mainly occurs because there's not enough space in RAM (cpu memory) to cache the results. All rendered results (images, masks, etc.) are cached in CPU, and if it reaches the limit, the program gets killed. I encountered this issue before when trying to reconstruct super long sequences (e.g., 7 cameras X 300 frames).
My recommendation for this situation would be: select fewer keys for rendering, only choose the ones you actually want to render. This way, fewer images will be cached, reducing memory usage.
Try to comment out "Dynamic_rgbs", "RigidNodes_rgbs" ... and keep only "rgbs", it should work
https://github.com/ziyc/drivestudio/blob/a1fe5d29162e5c4da81bb281c8223b457964a693/tools/train.py#L146-L161

Hi @qiuqc1, I've tested the exact same command recently and it worked well. I think this issue mainly occurs because there's not enough space in RAM (cpu memory) to cache the results. All rendered results (images, masks, etc.) are cached in CPU, and if it reaches the limit, the program gets killed. I encountered this issue before when trying to reconstruct super long sequences (e.g., 7 cameras X 300 frames).
My recommendation for this situation would be: select fewer keys for rendering, only choose the ones you actually want to render. This way, fewer images will be cached, reducing memory usage.
Try to comment out "Dynamic_rgbs", "RigidNodes_rgbs" ... and keep only "rgbs", it should work
https://github.com/ziyc/drivestudio/blob/a1fe5d29162e5c4da81bb281c8223b457964a693/tools/train.py#L146-L161

Thank you for your quick reply. I just tried eval again and it has been successfully generated.


Default font-size 18px.

Went with 18px because the objective was to have a bigger font size and 14px seemed too small.

 Use font size 14pt for text

Coverage remained the same at 42.508% when pulling ba6c45d23e01ecadbbd15b99103a68e2759d10db on ynurmahomed:font-size into 030f75e2e7c93ca1d7c7a6673b42abd5b8bfe562 on esaude:master.


can't create user in Linux centos 6

I have a Linux Centos 6.5 server and install node.js and sqlpad and I trying in to app but I cant create user because show error:

It should be menthioned that also I have awindows 10 instalation and also install the node.js and sqlpad and here is working Ok.
Thanks in advance

Sorry, I could solve it, it was my mistake, on server I was opening with mozilla firefox and I never set ip for open from remote client, with mozilla this doesn't work show error mentioned  and the follow error with email field require it although was provided.


docs(global): 📝 update documentation

Description
Update documentation
Ref: #231
Related Issues
https://github.com/wrappid/core/issues/332 and https://github.com/wrappid/native-web/issues/114 are need to resolve before this

@anantakumarghosh please guide him with the pr template what to do and what not

Sure.


Implement Database Changes Identified in #206/#236

Implement the data model changes recommended by Data Architects
@rpardofigueroa has worked with the data architects to identify some key improvements to our existing data model.  These are described in the following two tickets:

https://github.com/bcgov/nr-forests-access-management/issues/206 - Add Column Comments
https://github.com/bcgov/nr-forests-access-management/issues/236 - Misc Fixes

This ticket describes the implementation of those changes.
The sql file described here includes comments that describe different types of changes that need to be implemented.  Each section identified by a comment should have its own migration file.
Acceptance Criteria

[ ] Changes made to the orm model (server/backend/api/app/models/model.py)
[ ] All tests pass after changes implemented
[ ] Alembic migrations created and verified
[ ] Corresponding flyway migrations are created
[ ] Changes have been tested with local postgres instance
[ ] Changes implemented in the AWS database.

#206 - Add Column Comments  and #236 - Misc Fixes needs to be reviewed to determine MVP tasks on the ticket.

#206 has been completed. (Even if there was anything outstanding, it wouldn't be MVP.)

Here's my assessment of what's outstanding and what needs to be done:
Pre MVP:

Change timestamp columns to include time zone (TIMESTAMPTZ)

Post MVP:

Rename column group.purpose to group.group_purpose (I don't like the naming standard but might as well be consistent).
Add comments for: user.cognito_user_id, user.user_guid, user.user_name, fam_application_client.cognito_client_id,

Never:

Change case of table names

Already Done:

add user_type_code table
update most table comments


Monster Hunter World randomly freezes

Monster Hunter World (with proton) randomly freezes.
This usually happens in between after 10min to 4hours, so a long random time period.
As the DXVK_HUD (with memory) was enabled, at the time of the freeze, around ~3.9gb (assuming this is vram) of 6gb were used.
Most noticeable the dmesg output:
NVRM: Xid (PCI:0000:09:00): 31, Ch 0000004b, intr 10000000. MMU Fault: ENGINE GRAPHICS GPCCLIENT_T1_4 faulted @ 0x0_00000000. Fault is of type FAULT_PDE ACCESS_TYPE_READ

Xid 31, the addr 0x0_00000000, intr 10000000 and ACCESS_TYPE_READ are always constant.
To me, it looks like a simple nullptr access, as it is always the 0x0 addr, but I don't know how to investigate this problem further. I can not let the game run with
VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_standard_validation or apitrace for hours, as this makes it very unplayable.
PROTON_USE_WINED3D=1 just results in a black screen.
Allow flipping (in nvidia-setting) on/off does not change anything.
Please let me know how to make this report more useful, I am out of ideas.
Software information

Monster Hunter World
vsync: off
30fps lock (getting weird input lag otherwise sometimes)
Steam / Proton 3.16-beta5

System information

GPU: Nvidia Geforce 1060gtx 6gb
Driver: nvidia-drivers-415.23
Wine version: Proton 3.16-beta5 (???)
DXVK version: Proton 3.16-beta5 (dxvk 0.93)
Kernel: 4.19.10
Ram: 16gb
CPU: Ryzen 2700X

Log files
(with DXVK_LOG_LEVEL=debug and DXVK_HUD=devinfo,fps,memory)

d3d11.log: MonsterHunterWorld_d3d11.log
dxgi.log: MonsterHunterWorld_dxgi.log

EDIT:
The game overall runs pretty well, just the random freezes are a pretty frustrating problem.
EDIT2:
The screen freezes but the game background music is still running.

I'm aware of this, but I cannot fix this. I don't even know what the dmesg message means exactly and what could possibly cause it.
If apitrace doesn't work then I would still ask you to run the game with VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_standard_validation set. Righ now I have nothing to work with at all.

Hm ok, I will try to make something happen with
VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_standard_validation
but please tell me where/how/in which file should I see some output for this?
In d3d11.log? in dxgi.log? in proton log? on the console/terminal?
So that my attempts will not be in vain.

it will write to stdout, so capturing console output should work. It will not appear in the DXVK log files, and I don't know whether the Proton log captures it.

only that dxvk passes a null pointer to the gpu somehow

GPU pointers are hidden behind abstractions, so no, at least not directly.

Just a brief test already shows some errors like:
VUID-VkRenderPassCreateInfo-pDependencies-00837(ERROR / SPEC): msgNum: 0 - Dependency 1 specifies a source stage mask that contains stages not in the GRAPHICS pipeline as used by the source subpass 0. The Vulkan spec states: For any element of pDependencies, if the srcSubpass is not VK_SUBPASS_EXTERNAL, all stage flags included in the srcStageMask member of that dependency must be a pipeline stage supported by the pipeline identified by the pipelineBindPoint member of the source subpass. (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-VkRenderPassCreateInfo-pDependencies-00837)
    Objects: 1
       [0] 0x0, type: 0, name: (null)
Validation(ERROR): msg_code: 0:  [ VUID-VkRenderPassCreateInfo-pDependencies-00837 ] Object: VK_NULL_HANDLE (Type = 0) | Dependency 1 specifies a source stage mask that contains stages not in the GRAPHICS pipeline as used by the source subpass 0. The Vulkan spec states: For any element of pDependencies, if the srcSubpass is not VK_SUBPASS_EXTERNAL, all stage flags included in the srcStageMask member of that dependency must be a pipeline stage supported by the pipeline identified by the pipelineBindPoint member of the source subpass. (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-VkRenderPassCreateInfo-pDependencies-00837)
VUID-VkSubpassDependency-srcAccessMask-00868(ERROR / SPEC): msgNum: 0 - vkCreateRenderPass(): pDependencies[3].srcAccessMask (0xa000540) is not supported by srcStageMask (0x8000). The Vulkan spec states: Any access flag included in srcAccessMask must be supported by one of the pipeline stages in srcStageMask, as specified in the table of supported access types (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-VkSubpassDependency-srcAccessMask-00868)
    Objects: 1
       [0] 0x0, type: 0, name: (null)

but NOTE the game did not freeze yet! They might not be related to the freeze itself.
MonsterHunterWorld_VK_LAYER_LUNARG_standard_validation.log
Tomorrow I will try to make it until the game freezes.

Those are harmless and occur in every game because of some transform feedback issue.

Log with game frozen:
MonsterHunterWorld_VK_LAYER_LUNARG_standard_validation_long.log
VUID-vkResetDescriptorPool-descriptorPool-00313(ERROR / SPEC): msgNum: 0 - It is invalid to call vkResetDescriptorPool() with descriptor sets in use by a command buffer. The Vulkan spec states: All uses of descriptorPool (via any allocated descriptor sets) must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkResetDescriptorPool-descriptorPool-00313)
    Objects: 1
       [0] 0xce03, type: 22, name: (null)
Validation(ERROR): msg_code: 0:  [ VUID-vkResetDescriptorPool-descriptorPool-00313 ] Object: 0xce03 (Type = 22) | It is invalid to call vkResetDescriptorPool() with descriptor sets in use by a command buffer. The Vulkan spec states: All uses of descriptorPool (via any allocated descriptor sets) must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkResetDescriptorPool-descriptorPool-00313)

I hope this helps. :)
The rest
VUID-vkDestroyFramebuffer-framebuffer-00892(ERROR / SPEC): msgNum: 0 - Cannot call vkDestroyFramebuffer on Framebuffer 0x4bf461f that is currently in use by a command buffer. The Vulkan spec states: All submitted commands that refer to framebuffer must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyFramebuffer-framebuffer-00892)
    Objects: 1
       [0] 0x4bf461f, type: 24, name: (null)
Validation(ERROR): msg_code: 0:  [ VUID-vkDestroyFramebuffer-framebuffer-00892 ] Object: 0x4bf461f (Type = 24) | Cannot call vkDestroyFramebuffer on Framebuffer 0x4bf461f that is currently in use by a command buffer. The Vulkan spec states: All submitted commands that refer to framebuffer must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyFramebuffer-framebuffer-00892)

...

VUID-vkDestroyBufferView-bufferView-00936(ERROR / SPEC): msgNum: 0 - Cannot call vkDestroyBufferView on BufferView 0x4bf419a that is currently in use by a command buffer. The Vulkan spec states: All submitted commands that refer to bufferView must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyBufferView-bufferView-00936)
    Objects: 1
       [0] 0x4bf419a, type: 13, name: (null)
Validation(ERROR): msg_code: 0:  [ VUID-vkDestroyBufferView-bufferView-00936 ] Object: 0x4bf419a (Type = 13) | Cannot call vkDestroyBufferView on BufferView 0x4bf419a that is currently in use by a command buffer. The Vulkan spec states: All submitted commands that refer to bufferView must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyBufferView-bufferView-00936)

...

VUID-vkDestroyBufferView-bufferView-00936(ERROR / SPEC): msgNum: 0 - Cannot call vkDestroyBufferView on BufferView 0x4bf419a that is currently in use by a command buffer. The Vulkan spec states: All submitted commands that refer to bufferView must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyBufferView-bufferView-00936)
    Objects: 1
       [0] 0x4bf419a, type: 13, name: (null)
Validation(ERROR): msg_code: 0:  [ VUID-vkDestroyBufferView-bufferView-00936 ] Object: 0x4bf419a (Type = 13) | Cannot call vkDestroyBufferView on BufferView 0x4bf419a that is currently in use by a command buffer. The Vulkan spec states: All submitted commands that refer to bufferView must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyBufferView-bufferView-00936)

I guess is only because I killed the process (kill -9 pid) after it was stuck for 1+min.

I honestly can't tell, there are no timestamps on this logs. I happens too quickly so I guess around the same time.
Any way to add timestamp? or prove it? Or want me to test something else?

Further observation:

DXVK_STATE_CACHE=0 seems to make it worse, tried it 3 times and it froze within the first 15min.
(But take it with a grain of salt, might be unrelated)
using DXVK_HUD, stats as it froze

Geforce GTX 1060 6GB
Driver: 415.23.0
Vulkan: 1.1.84
FPS: 30.0
min: 9.7 max: 57.0
Queue submissions: 5
Draw calls: 1310
Dispatch calls: 149
Render passes: 127
Graphics pipelines: 536
Compute pipelines: 140
Memory allocated: 2948 MB
Memory used: 2779 MB

also tried VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_api_dump but can not get it working

But I found another bug, which might taint some of my reports so far.
The Game crashes when going to an F1 terminal.
Reproduce:

Open the Game
Wait for it to load until you are in the Menu
Press "Ctrl + Alt + F1"
-> crash in
=>0 0x00007f0d31a42e09 in libnvidia-glcore.so.415.25 (+0x11a0e09) (0x00007f0d2c84b4a0)
steam-582010_dump_v5.log

This might invalidate my "after the freeze nvidia-smi reports 0% GPU usage" comment.
But it also could be caused by the new:

DXVK version: Proton 3.16-beta6 (dxvk 0.94)

or

Driver: nvidia-drivers-415.25

I also had this bug, and here is my log (with proton 3.16-6-beta) until the next proton release arrives.
on this one I played for few hours and then went to take a small nap and let game run (+ "login") to check if this bug would occur even doing nothing...and it did.
can't say if that log can help you but there it. if there is anything I can do give you better report I will.
I will try to replicate the bug on next proton update...whenever it arrives

my system spec:
inxi -b
System:    Host: linux Kernel: 4.20.6-1-default x86_64 bits: 64 Desktop: KDE Plasma 5.14.5
           Distro: openSUSE Tumbleweed 20190209
Machine:   Type: Desktop Mobo: ASUSTeK model: Z170 PRO GAMING v: Rev X.0x serial: <root required> UEFI: American Megatrends
           v: 3805 date: 05/16/2018
CPU:       Quad Core: Intel Core i5-6600K type: MCP speed: 4374 MHz min/max: 800/4400 MHz
Graphics:  Device-1: NVIDIA GM204 [GeForce GTX 970] driver: nvidia v: 410.93
           Display: x11 server: X.Org 1.20.3 driver: nvidia resolution: 1920x1080~60Hz, 1920x1080~60Hz
           OpenGL: renderer: GeForce GTX 970/PCIe/SSE2 v: 4.6.0 NVIDIA 410.93
Network:   Device-1: Intel Ethernet I219-V driver: e1000e
Info:      Processes: 443 Uptime: 02:47:03  up 6 days  3:15,  3 users,  load average: 0.70, 1.07, 1.39 Memory: 15.60 GiB
           used: 6.94 GiB (44.5%) Shell: bash inxi: 3.0.30

Just came back to the game, I'll see what I can do to produce some logs, can confirm that the same error occurs on the latest stable proton. Nvidia's website recommends cuda-memcheck or cuda-gdb, but I haven't had any luck getting cuda-gdb to attach properly (granted, that's my first attempt at gdb, so I might be missing something there).
I'll see what I can come up with for VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_api_dump and VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_vktrace tomorrow too

This is a note for the xid message. As documented here, xid 31 is GPU memory page fault, usually invalid memory access. In this case, it looks like a null pointer read.
For debugging with cuda-gdb, it depends on the number of gpus... Please see this.
It is probably possible to dump the core as instructed here
BTW, I am not an expert in GPU/DXVK... I am still learning from nvidia's manual. Feel free to correct me if I am wrong.

Managed to get cuda-gdb attached to it, fingers crossed!

@Xaenalt Could you please test disabling the nvapi hack and see if things improve? Put "dxgi.nvapiHack = False" in dxvk.conf, and add DXVK_CONFIG_FILE=/path/to/dxvk.conf to the launcher command line.

Will do!

No change with dxgi.nvapiHack = False, crash still occurs

Regular log from the crash while I attempt to get the SDK set up
steam-582010.log

Is this a driver bug? Has a bug report been submitted to Nvidia?

We'll know soon I hope. I got the api_trace working, and a 2TB drive to hold the log. Worst case I'm gonna leave it overnight and hope the error is in there

AHA, gotcha! Log uploading now! :D
(it'll be pretty big, I'm gzipping it to try to reduce size, but expect a long gunzip)
We lose the GPU on frame 13485 with the VK_ERROR_DEVICE_LOST (-4) error

33G to 1G, wow, that compressed really well o.O
It's too big to upload to github directly, but I threw it into my gdrive, let me know if you have any issues pulling it: https://drive.google.com/file/d/1SHxowR6NZlSlUC4sm40o6m5WPk89OoXg/view?usp=sharing

I may be wrong, I'm no expert, but it seems like the semaphore at 0x16d4fc10 might be getting overwritten? It looks like some buffer copies target that area too. That might be expected behavior, idk

Hopefully @doitsujin will be able to take a peep at it? :)

Fingers crossed that the error is in plain sight in there :)

@Emanem that error occurs because of the lost device, it doesn't cause it.

Any way to help track down what's causing the lost device?

Does this still happen with the fixed shader and the latest driver? There's an admittedly small chance that the bad shader caused these hangs in the first place.
There doesn't really seem to be anything wrong apart from it, and these hangs do seem to be specific to Nvidia.

Testing now with that patch you just provided in the other bug. Should I grab another API trace if it encounters the bug?

@doitsujin I'm sorry to say, it just happened with the version you sent me

It seems to hang less frequently though, in the past 2 tests, it seems like it took a lot longer for the hang to happen. I'm going to keep testing

Still get a few early on, I'll try to get the api trace from one. In the meantime, here's a shader dump
https://drive.google.com/file/d/19MIcBdoZp6V8PPWstbbQvnJoXwSw3hR6/view?usp=sharing

Just doing some additional testing, setting d3d11.zeroWorkgroupMemory = True will cause the lockup to not lock up the entire system, still occurs though, and the fps takes a big hit

Just doing some additional testing, setting d3d11.zeroWorkgroupMemory = True will cause the lockup to not lock up the entire system, still occurs though, and the fps takes a big hit

Can confirm that  d3d11.zeroWorkgroupMemory = True also prevents a full lockup, possibly implying it's related to a compute shader?

Just on a hunch, also trying with dxvk.numCompilerThreads = 1 just in case, will report if it crashes with that

I tried it already. It will crash.
On Tue, Feb 26, 2019, 18:50 Sean Pryor<EMAIL_ADDRESS>wrote:

Just on a hunch, also trying with dxvk.numCompilerThreads = 1 just in
case, will report if it crashes with that
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/doitsujin/dxvk/issues/816#issuecomment-467663470, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABY5npcoS0JkYIelBf2n9kdJ4S_WEyj8ks5vRcgvgaJpZM4ZW911
.

Last time I tried using the validation layers I didn't get anything interesting, this time when it crashed I received

VUID-vkDestroyCommandPool-commandPool-00041(ERROR / SPEC): msgNum: 0 - Attempt to destroy command pool with command buffer (0x7d41d750) which is in use. The Vulkan spec states: All VkCommandBuffer objects allocated from commandPool must not be in the pending state. (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyCommandPool-commandPool-00041)
Objects: 1
[0] 0x7d41d750, type: 6, name: NULL
VUID-vkDestroyFence-fence-01120(ERROR / SPEC): msgNum: 0 - Fence 0x23b804 is in use. The Vulkan spec states: All queue submission commands that refer to fence must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkDestroyFence-fence-01120)
Objects: 1
[0] 0x23b804, type: 7, name: NULL

Followed by a flood of

VUID-vkResetDescriptorPool-descriptorPool-00313(ERROR / SPEC): msgNum: 0 - It is invalid to call vkResetDescriptorPool() with descriptor sets in use by a command buffer. The Vulkan spec states: All uses of descriptorPool (via any allocated descriptor sets) must have completed execution (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkResetDescriptorPool-descriptorPool-00313)
Objects: 1
[0] 0xd3, type: 22, name: NULL

Yep, does indeed still crash with the same issue. I wonder if there's any extra debugging info we can add to track it down further, I put the API dump in an earlier comment, which it looks like didn't have any clues

@rsw0x that happens because of the error, but doesn't cause it. DXVK still doesn't handle DEVICE_LOST errors properly.

Are device lost errors something recoverable?

Device lost might just mean that the Driver crashed.
Most games don't handle this well. Mostly benchmarks do that because they know either users are trying unstable OC or unstable drivers.

I had this problem previously. Just updated to the latest nvidia-drivers ebuild on Gentoo:
x11-drivers/nvidia-drivers-418.43::gentoo was built with the following: USE="X acpi driver gtk3 kms multilib tools -compat -static-libs -uvm -wayland" ABI_X86="32 (64) (-x32)"
Running this kernel:
Linux localhost 4.20.13-gentoo #2 SMP Thu Feb 28 20:13:14 EST 2019 x86_64 Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz GenuineIntel GNU/Linux
Latest Steam proton beta (3.16-7).
nvidia-smi output:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.43       Driver Version: 418.43       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1070    Off  | 00000000:01:00.0  On |                  N/A |
| 37%   59C    P0   130W / 180W |   3164MiB /  8119MiB |     99%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 970     Off  | 00000000:02:00.0 Off |                  N/A |
|  0%   27C    P8    12W / 201W |      1MiB /  4043MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      3273      G   /usr/libexec/Xorg                            291MiB |
|    0      3516      G   /usr/bin/gnome-shell                         171MiB |
|    0      4169      G   ...uest-channel-token=17976434344080092270    51MiB |
|    0     19152      G   ...in/.local/share/Steam/ubuntu12_32/steam    33MiB |
|    0     19161      G   ./steamwebhelper                               3MiB |
|    0     20860    C+G   ...ter Hunter World\MonsterHunterWorld.exe  2495MiB |
|    0     22146      G   ...quest-channel-token=1348227650674135017   113MiB |
+-----------------------------------------------------------------------------+

I've had MHW running for an hour and a half now without any freezes. I'm going to let it run overnight and see if it freezes up, just looking out over the ocean.
If it DOES freeze up eventually, I have two nvidia cards I can test with on identical hardware otherwise. This is running on a GTX 1070, but I also have a GTX 970 I was using previously.
The GTX 970 usually froze up within 45 minutes to an hour of starting.

Looks like it froze just after the 4 hour mark overnight. Same Xid dmesg log as buscher above.
Is there anything I can do to help diagnose further?

latest vulkan driver changelog:

Fixed a bug which could cause the compiler to crash in some Vulkan games

https://developer.nvidia.com/vulkan-driver
Any changes?

latest vulkan driver changelog:

Fixed a bug which could cause the compiler to crash in some Vulkan games

https://developer.nvidia.com/vulkan-driver
Any changes?

Not sure if I'm not understanding the way nvidia does version numbering, or if I already have that update? My nvidia-smi shows I'm running 418.43, whereas the latest version on that page is 418.42.02. I'll switch to that version and test again overnight, given the ~4 hour run time until it freezes it pretty much has to be an overnight test for me.

Same result with 418.42.02. Started at<PHONE_NUMBER>.835805, froze at<PHONE_NUMBER>.174271, just under 6 hours of runtime.

I have this problem with Battlefield 1. System freezes randomly, sometimes after many hours, and I have to kill X to get the frozen image away.
Running KDE Neon 18.04 with nvidia-driver-418-418.43, with Wine 4.3 and DXVK 1.0.
My dmesg messages are [ 2980.508578] NVRM: GPU at PCI:0000:01:00: GPU-a919130d-9a04-dbf1-19c0-c827155af29b [ 2980.508582] NVRM: Xid (PCI:0000:01:00): 31, Ch 00000054, intr 10000000. MMU Fault: ENGINE GRAPHICS GPCCLIENT_T1_0 faulted @ 0x1_5d1ee000. Fault is of type FAULT_PTE ACCESS_TYPE_READ
Let me know if there's more info I can provide

Something about the Kulve Taroth fight makes it crash much more frequently @_@

Something about the Kulve Taroth fight makes it crash much more frequently @_@

Since I updated drivers to 418 and have a new videocard (2080 Ti), I never experienced this infamous crash once.

That's interesting, I'm on the 1080ti, with driver 418.43, I wonder if there's some ray tracing function that is being used that avoids this issue
Can you give us an inxi -b to tell us a bit more about the system?

I don't have any real proof one way or another, but this feels like a memory/handle issue. Is memory fragmentation a thing for GPU memory, could there be 2GB of VRAM free but no large contiguous chunks to allocate for a given texture/shader/etc, causing that allocation to fail and return null?
This would explain why more VRAM seems to let the game last longer before freezing, even though VRAM usage doesn't actually seem to leak.

Hmmm, if that were the case, I'd sort of expect to see a GPU out of memory error, or for that to be revealed in the trace I posted
I wish I knew more to help pin down the specifics of the error

Well at least some things do require contiguous memory. See for example https://vulkan.lunarg.com/doc/view/<IP_ADDRESS>/linux/vkspec.chunked/ch10s02.html

Note
vkMapMemory will fail if the implementation is unable to allocate an appropriately sized contiguous virtual address range, e.g. due to virtual address space fragmentation or platform limits. In such cases, vkMapMemory must return VK_ERROR_MEMORY_MAP_FAILED. The application can improve the likelihood of success by reducing the size of the mapped range and/or removing unneeded mappings using VkUnmapMemory.

Yeah, if that were the case, I'd expect to see that error rather than a null pointer dereference. We'd get the VK_ERROR_MEMORY_MAP_FAILED despite having GPU memory usage not being at 100%

@Xaenalt My understanding is that the game does not support Ray tracing (as that is Dx12 feature) nor does GTX gpus, you need RTX for that

Ah, I figured that would be the only explanation if the 2080rtx wasn't crashing while the 1080gtx did

Again, I think I have generally 10.9 GiB free ram (reminder of 11.264 is used by the OS/gnome).
I'm not sure if it's ram related, but that could be a good hint. Still, with new drivers I had less crashes on my 1080 too (please not it was not a Ti).

Since this MAY be somewhat memory related from your comments above, is https://github.com/doitsujin/dxvk/issues/267#issuecomment-475299958 possibly related? Even tho you are not using PRIME...

Since this MAY be somewhat memory related from your comments above, is #267 (comment) possibly related? Even tho you are not using PRIME...

I've got 418.43 currently...

Using
nvidia-drivers-418.56
and I got no freeze for at least 6h of game play.
it maybe is just a coincident but maybe the changelog entry (for 418.56)

Fixed a bug which could sometimes cause Vulkan applications to lock up the GPU when freeing large chunks of memory on systems with PRIME enabled.

even thou I am not using PRIME.
FTR I got several crashes & freezes with 418.43 or older, with my gtx1060 6gb ram OC. I will report back once I got a freeze.

Still freezing on 418.56 unfortunately.
Played with some friends for ~4 hours last night, then saved to title screen and loaded back in (to force a save). Let it idle overnight.
22:10AM was the start time of the MonsterHunterWorld.exe process, froze at 6:18 (8h 8m of runtime).

Still a problem for me as well. Nvidia 418.56, Proton 4.2, Linux Mint 19.1, Nvidia 970. Crashes randomly, in-between a 1 - 4 hours range it seems.
Thanks for all your work :green_heart: - let me know how I can help debug this.

I can confidently report that since I moved to a RTX 2080 Ti, the issue has completely gone.
I wonder if it's an Nvidia driver issue which appears with less performant cards (it did happen frequently with my GTX 1080).

I can confidently report that since I moved to a RTX 2080 Ti, the issue has completely gone.
I wonder if it's an Nvidia driver issue which appears with less performant cards (it did happen frequently with my GTX 1080).

How long did you test for? I was able to get 6-8 hours on average before having the issue on my 1070, and my 970 had the issues after 45-50 minutes on average.
Based on increased specs from the 1070 to the 2080 ti, I'd expect to see probably 14-20 hours of runtime before you ran into the issue.
I'm curious to see if the new HD texture pack DLC they released impacts the runtime before encountering the issue, as well, but haven't had a chance to test yet.

Just had a freeze after about 2h 15m when using HD texture DLC, as opposed to 6h 30m normally. there's an in-game VRAM usage estimator now that shows 4.4GB of VRAM with my normal settings, 6.2GB with HD textures.

People here are still experiencing some freeze with current MHW build these days?

Are people here still experiencing some freeze with current MHW build these days?

Yes. I've played few days ago and had some freezes. I felt like they now happen less frequently. It's probably just random.
I was so annoyed with this I wrote a script wich detect if game froze and kill it. Had to do it, because when game freezes, everyting freezes with it, cant open terminal or tty.

I haven't been experiencing the freeze since I updated from 1080 GTX to 2080 Ti RTX.

Since it seems this problem is Nvidia GPU driver-related, anyone cared to reach out to Nvidia?

sadly, this error is still happening. Interestingly enough, it used to freeze my whole X Session and i had to Ctrl-Alt-F2 to get into another shell and kill X. Since a fews days it's no longer freezing X but only the game itself, and i can alt-tab out into another terminal to kill only the MHW process. Sadly i'm not sure which version update led to that change. Right now i'm using Proton 4.2.3 from Steam with 418.74 as Nvidia driver on Arch Linux.
I also found this thread by googling the error message: https://forums.geforce.com/default/topic/1096146/linux-monster-hunter-world-drivers-crash/
but there's not much too it.
I play the game a lot so if i can help debug this please tell me. Would it help to run the game with strace to see which systemcall is failing? Is there even any way to workaround this issue without nvidia's help?

Reached out to Nvidia to see if they can help debug it

Try disable Z-Prepass in game graphics settings, give me a really stable gameplay (but performance not as good as previous

@Misairu-G this is only for Optimus right?

@Misairu-G this is only for Optimus right?

@huberb I don't have other system setup, so I can't answer your question. But you're right, I'm using bumblebee and PrimusVK.
My freezing only appear when I switch to Driver version higher than 396, or proton 4.x. FYI, I'm running Ubuntu 16.04 with 4.15.0 kernel.

Sadly, it's the same problem with proton 3.16. Random freezes with FAULT_PDE ACCESS_TYPE_READ in dmesg. I'm really hoping Nvidia will eventually do something about this.

Perhaps this is related to #1169. There is a fix in development by nvidia for that one. When it is available, it might be a good idea to test it here too.

Tested the latest vulkan beta drivers (435.19.03) and the issue persists (freeze after ~3h 50m), so this wasn't fixed by the fix for #1169 unfortunately.
I was running nvidia-smi dmon -s pucm -o DT -i 0 during the testing and this was towards the end of the output, fifth line being where the freeze occurred:
 20190909   16:00:15      0   131    63     -    99    47     0     0  4006  1936  3191     9
 20190909   16:00:16      0   132    63     -    99    47     0     0  4006  1936  3191     9
 20190909   16:00:17      0   131    63     -    99    47     0     0  4006  1936  3191     9
 20190909   16:00:18      0   133    63     -    99    47     0     0  4006  1936  3191     9
 20190909   16:00:19      0    49    62     -     0     0     0     0  4006  1936  3191     9
 20190909   16:00:20      0    49    61     -     0     0     0     0  4006  1936  3191     9
 20190909   16:00:21      0    49    61     -     0     0     0     0  4006  1936  3191     9
 20190909   16:00:22      0    33    60     -     0     1     0     0  3802   999  3191     9
#Date       Time        gpu   pwr gtemp mtemp    sm   mem   enc   dec  mclk  pclk    fb  bar1
#YYYYMMDD   HH:MM:SS    Idx     W     C     C     %     %     %     %   MHz   MHz    MB    MB
 20190909   16:00:23      0    33    59     -     0     1     0     0  3802   999  3191     9
 20190909   16:00:24      0    24    59     -     0     2     0     0   810   797  3191     9
 20190909   16:00:25      0    17    58     -     0     3     0     0   810   797  3191     9
 20190909   16:00:26      0    17    58     -     0     3     0     0   810   797  3191     9
 20190909   16:00:27      0    15    57     -     1     5     0     0   405   227  3191     9
 20190909   16:00:28      0    15    57     -     1     6     0     0   405   227  3191     9

At the start, the line listed sm % as 99 and mem % as 46. Normal usage shows sm % of 0 or 1, and mem % of 1.
Game window froze, but terminal on other monitor continued to output every second up until I tried to alt+tab out, then everything froze.  MMU Fault: ENGINE GRAPHICS GPCCLIENT_T1_5 faulted @ 0x0_00000000. Fault is of type FAULT_PDE ACCESS_TYPE_READ```

If an API Trace or something else can help with diagnosing the cause of this issue, I can generate one provided someone can provide me documentation on how to do so. I have no issue letting the game run overnight to get a valid dump if needed.

Happy new year ;-)
What should we do with this bug? MHW runs generally fine for me. There are a few issues left but none of them are (as far as I can tell) related to DXVK.
Close it?

Agreed - let's close it.


failed to find rejected notification with sequence number 0

I get this error when calling shutdown(). Calling shutdown() also resends all the push notifications that I had sent during that session!
PushManager-connection-0 failed to find rejected notification with sequence number 0 (buffer has range 1 to 11); this may mean the sent notification buffer is too small. Please report this as a bug.

Here is the bug again with trace logging enabled. In the below case, after I called shutdown(), it sent me the same message over and over again (I received it on my phone constantly) until I killed the process with "kill -9"
INFO  2015-02-18 18:26:33,254 [main] - OurcastPushManager starting.
DEBUG 2015-02-18 18:26:33,342 [main] - OurcastPushManager-connection-0 beginning connection process.
DEBUG 2015-02-18 18:26:34,092 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-0 connected; waiting for TLS handshake.
DEBUG 2015-02-18 18:26:34,815 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-0 successfully completed TLS handshake.
TRACE 2015-02-18 18:26:34,815 [nioEventLoopGroup-2-1] - Connection succeeded: ApnsConnection [name=OurcastPushManager-connection-0]
INFO  2015-02-18 18:27:00,848 [pool-1-thread-4] - Sent push #20 to 39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2 with message Hello test
TRACE 2015-02-18 18:27:00,857 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-0 sending SendableApnsPushNotification [sequenceNumber=1, token=39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2, payload={"aps":{"alert":"Hello test","sound":"default"}}, deliveryInvalidation=Wed Feb 18 18:29:00 EST 2015]
TRACE 2015-02-18 18:27:00,860 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-0 successfully wrote notification 1
INFO  2015-02-18 18:27:26,075 [ShutdownHook] - APNS daemon got shutdown signal
INFO  2015-02-18 18:27:28,866 [main] - OurcastPushManager shutting down.
DEBUG 2015-02-18 18:27:28,867 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-0 sending known-bad notification to shut down.
TRACE 2015-02-18 18:27:28,869 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-0 successfully wrote known-bad notification 2
DEBUG 2015-02-18 18:27:29,113 [nioEventLoopGroup-2-1] - APNs gateway rejected notification with sequence number 0 from OurcastPushManager-connection-0 (MISSING_TOKEN).
ERROR 2015-02-18 18:27:29,113 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-0 failed to find rejected notification with sequence number 0 (buffer has range 1 to 1); this may mean the sent notification buffer is too small. Please report this as a bug.
TRACE 2015-02-18 18:27:29,113 [nioEventLoopGroup-2-1] - ApnsConnection [name=OurcastPushManager-connection-0] returned 1 unprocessed notifications
TRACE 2015-02-18 18:27:29,116 [nioEventLoopGroup-2-1] - Connection closed: ApnsConnection [name=OurcastPushManager-connection-0]
DEBUG 2015-02-18 18:27:29,117 [pool-3-thread-1] - OurcastPushManager-connection-1 beginning connection process.
DEBUG 2015-02-18 18:27:29,372 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-1 connected; waiting for TLS handshake.
DEBUG 2015-02-18 18:27:29,930 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-1 successfully completed TLS handshake.
TRACE 2015-02-18 18:27:29,930 [nioEventLoopGroup-2-1] - Connection succeeded: ApnsConnection [name=OurcastPushManager-connection-1]
TRACE 2015-02-18 18:27:29,930 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-1 sending SendableApnsPushNotification [sequenceNumber=1, token=39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2, payload={"aps":{"alert":"Hello test","sound":"default"}}, deliveryInvalidation=Wed Feb 18 18:29:00 EST 2015]
TRACE 2015-02-18 18:27:29,931 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-1 successfully wrote notification 1
DEBUG 2015-02-18 18:27:29,931 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-1 sending known-bad notification to shut down.
TRACE 2015-02-18 18:27:29,931 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-1 successfully wrote known-bad notification 2
DEBUG 2015-02-18 18:27:30,186 [nioEventLoopGroup-2-1] - APNs gateway rejected notification with sequence number 0 from OurcastPushManager-connection-1 (MISSING_TOKEN).
ERROR 2015-02-18 18:27:30,186 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-1 failed to find rejected notification with sequence number 0 (buffer has range 1 to 1); this may mean the sent notification buffer is too small. Please report this as a bug.
TRACE 2015-02-18 18:27:30,186 [nioEventLoopGroup-2-1] - ApnsConnection [name=OurcastPushManager-connection-1] returned 1 unprocessed notifications
TRACE 2015-02-18 18:27:30,187 [nioEventLoopGroup-2-1] - Connection closed: ApnsConnection [name=OurcastPushManager-connection-1]
DEBUG 2015-02-18 18:27:30,188 [pool-3-thread-1] - OurcastPushManager-connection-2 beginning connection process.
DEBUG 2015-02-18 18:27:30,469 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-2 connected; waiting for TLS handshake.
DEBUG 2015-02-18 18:27:31,074 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-2 successfully completed TLS handshake.
TRACE 2015-02-18 18:27:31,074 [nioEventLoopGroup-2-1] - Connection succeeded: ApnsConnection [name=OurcastPushManager-connection-2]
TRACE 2015-02-18 18:27:31,075 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-2 sending SendableApnsPushNotification [sequenceNumber=1, token=39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2, payload={"aps":{"alert":"Hello test","sound":"default"}}, deliveryInvalidation=Wed Feb 18 18:29:00 EST 2015]
TRACE 2015-02-18 18:27:31,075 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-2 successfully wrote notification 1
DEBUG 2015-02-18 18:27:31,075 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-2 sending known-bad notification to shut down.
TRACE 2015-02-18 18:27:31,076 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-2 successfully wrote known-bad notification 2
DEBUG 2015-02-18 18:27:31,357 [nioEventLoopGroup-2-1] - APNs gateway rejected notification with sequence number 0 from OurcastPushManager-connection-2 (MISSING_TOKEN).
ERROR 2015-02-18 18:27:31,358 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-2 failed to find rejected notification with sequence number 0 (buffer has range 1 to 1); this may mean the sent notification buffer is too small. Please report this as a bug.
TRACE 2015-02-18 18:27:31,358 [nioEventLoopGroup-2-1] - ApnsConnection [name=OurcastPushManager-connection-2] returned 1 unprocessed notifications
TRACE 2015-02-18 18:27:31,358 [nioEventLoopGroup-2-1] - Connection closed: ApnsConnection [name=OurcastPushManager-connection-2]
DEBUG 2015-02-18 18:27:31,359 [pool-3-thread-1] - OurcastPushManager-connection-3 beginning connection process.
DEBUG 2015-02-18 18:27:31,607 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-3 connected; waiting for TLS handshake.
DEBUG 2015-02-18 18:27:32,149 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-3 successfully completed TLS handshake.
TRACE 2015-02-18 18:27:32,149 [nioEventLoopGroup-2-1] - Connection succeeded: ApnsConnection [name=OurcastPushManager-connection-3]
TRACE 2015-02-18 18:27:32,150 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-3 sending SendableApnsPushNotification [sequenceNumber=1, token=39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2, payload={"aps":{"alert":"Hello test","sound":"default"}}, deliveryInvalidation=Wed Feb 18 18:29:00 EST 2015]
TRACE 2015-02-18 18:27:32,150 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-3 successfully wrote notification 1
DEBUG 2015-02-18 18:27:32,150 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-3 sending known-bad notification to shut down.
TRACE 2015-02-18 18:27:32,151 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-3 successfully wrote known-bad notification 2
DEBUG 2015-02-18 18:27:32,400 [nioEventLoopGroup-2-1] - APNs gateway rejected notification with sequence number 0 from OurcastPushManager-connection-3 (MISSING_TOKEN).
ERROR 2015-02-18 18:27:32,401 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-3 failed to find rejected notification with sequence number 0 (buffer has range 1 to 1); this may mean the sent notification buffer is too small. Please report this as a bug.
TRACE 2015-02-18 18:27:32,401 [nioEventLoopGroup-2-1] - ApnsConnection [name=OurcastPushManager-connection-3] returned 1 unprocessed notifications
TRACE 2015-02-18 18:27:32,401 [nioEventLoopGroup-2-1] - Connection closed: ApnsConnection [name=OurcastPushManager-connection-3]
DEBUG 2015-02-18 18:27:32,402 [pool-3-thread-1] - OurcastPushManager-connection-4 beginning connection process.
DEBUG 2015-02-18 18:27:32,652 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-4 connected; waiting for TLS handshake.
DEBUG 2015-02-18 18:27:33,192 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-4 successfully completed TLS handshake.
TRACE 2015-02-18 18:27:33,192 [nioEventLoopGroup-2-1] - Connection succeeded: ApnsConnection [name=OurcastPushManager-connection-4]
TRACE 2015-02-18 18:27:33,193 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-4 sending SendableApnsPushNotification [sequenceNumber=1, token=39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2, payload={"aps":{"alert":"Hello test","sound":"default"}}, deliveryInvalidation=Wed Feb 18 18:29:00 EST 2015]
TRACE 2015-02-18 18:27:33,193 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-4 successfully wrote notification 1
DEBUG 2015-02-18 18:27:33,193 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-4 sending known-bad notification to shut down.
TRACE 2015-02-18 18:27:33,193 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-4 successfully wrote known-bad notification 2
DEBUG 2015-02-18 18:27:33,445 [nioEventLoopGroup-2-1] - APNs gateway rejected notification with sequence number 0 from OurcastPushManager-connection-4 (MISSING_TOKEN).
ERROR 2015-02-18 18:27:33,445 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-4 failed to find rejected notification with sequence number 0 (buffer has range 1 to 1); this may mean the sent notification buffer is too small. Please report this as a bug.
TRACE 2015-02-18 18:27:33,445 [nioEventLoopGroup-2-1] - ApnsConnection [name=OurcastPushManager-connection-4] returned 1 unprocessed notifications
TRACE 2015-02-18 18:27:33,446 [nioEventLoopGroup-2-1] - Connection closed: ApnsConnection [name=OurcastPushManager-connection-4]
DEBUG 2015-02-18 18:27:33,446 [pool-3-thread-1] - OurcastPushManager-connection-5 beginning connection process.
DEBUG 2015-02-18 18:27:33,713 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-5 connected; waiting for TLS handshake.
DEBUG 2015-02-18 18:27:34,252 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-5 successfully completed TLS handshake.
TRACE 2015-02-18 18:27:34,253 [nioEventLoopGroup-2-1] - Connection succeeded: ApnsConnection [name=OurcastPushManager-connection-5]
TRACE 2015-02-18 18:27:34,253 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-5 sending SendableApnsPushNotification [sequenceNumber=1, token=39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2, payload={"aps":{"alert":"Hello test","sound":"default"}}, deliveryInvalidation=Wed Feb 18 18:29:00 EST 2015]
TRACE 2015-02-18 18:27:34,254 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-5 successfully wrote notification 1
DEBUG 2015-02-18 18:27:34,254 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-5 sending known-bad notification to shut down.
TRACE 2015-02-18 18:27:34,255 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-5 successfully wrote known-bad notification 2
DEBUG 2015-02-18 18:27:34,506 [nioEventLoopGroup-2-1] - APNs gateway rejected notification with sequence number 0 from OurcastPushManager-connection-5 (MISSING_TOKEN).
ERROR 2015-02-18 18:27:34,506 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-5 failed to find rejected notification with sequence number 0 (buffer has range 1 to 1); this may mean the sent notification buffer is too small. Please report this as a bug.
TRACE 2015-02-18 18:27:34,506 [nioEventLoopGroup-2-1] - ApnsConnection [name=OurcastPushManager-connection-5] returned 1 unprocessed notifications
TRACE 2015-02-18 18:27:34,507 [nioEventLoopGroup-2-1] - Connection closed: ApnsConnection [name=OurcastPushManager-connection-5]
DEBUG 2015-02-18 18:27:34,507 [pool-3-thread-1] - OurcastPushManager-connection-6 beginning connection process.
DEBUG 2015-02-18 18:27:34,751 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-6 connected; waiting for TLS handshake.
DEBUG 2015-02-18 18:27:35,283 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-6 successfully completed TLS handshake.
TRACE 2015-02-18 18:27:35,283 [nioEventLoopGroup-2-1] - Connection succeeded: ApnsConnection [name=OurcastPushManager-connection-6]
TRACE 2015-02-18 18:27:35,284 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-6 sending SendableApnsPushNotification [sequenceNumber=1, token=39490a6c7717092c6b667c969f207f8dce43bd8de030bc316db80472125c71a2, payload={"aps":{"alert":"Hello test","sound":"default"}}, deliveryInvalidation=Wed Feb 18 18:29:00 EST 2015]
TRACE 2015-02-18 18:27:35,285 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-6 successfully wrote notification 1
DEBUG 2015-02-18 18:27:35,285 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-6 sending known-bad notification to shut down.
TRACE 2015-02-18 18:27:35,285 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-6 successfully wrote known-bad notification 2
DEBUG 2015-02-18 18:27:35,530 [nioEventLoopGroup-2-1] - APNs gateway rejected notification with sequence number 0 from OurcastPushManager-connection-6 (MISSING_TOKEN).
ERROR 2015-02-18 18:27:35,531 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-6 failed to find rejected notification with sequence number 0 (buffer has range 1 to 1); this may mean the sent notification buffer is too small. Please report this as a bug.
TRACE 2015-02-18 18:27:35,531 [nioEventLoopGroup-2-1] - ApnsConnection [name=OurcastPushManager-connection-6] returned 1 unprocessed notifications
TRACE 2015-02-18 18:27:35,531 [nioEventLoopGroup-2-1] - Connection closed: ApnsConnection [name=OurcastPushManager-connection-6]
DEBUG 2015-02-18 18:27:35,531 [pool-3-thread-1] - OurcastPushManager-connection-7 beginning connection process.
DEBUG 2015-02-18 18:27:35,788 [nioEventLoopGroup-2-1] - OurcastPushManager-connection-7 connected; waiting for TLS handshake.

Having never seen this before today, I'm now getting what looks like the same problem.  Seems to happen intermittently.  When it does occur, I get the notification sent multiple times, even though the PushManager reports it failed to connect to the service.

Bizarre. This sounds like a serious problem, and possibly something originating upstream. Thank you, everybody, for the detailed information. We'll do our best to get to the bottom of this as quickly as possible.

…although this does appear to be a duplicate of #149. Closing as such.


Removed interface EmitterOpRandomStartEndConfig

This PR

Updates the Documentation

Describe the changes below:

The interface EmitterOpRandomStartEndConfig, even though it is another path to configure a random emit value, does not fully describe that all the options for the EmitterOpEaseConfig are available. Instead, it is better to add to the description of the EmitterOpEaseConfig that there is a random start value option available.

👍


hansenlaw vectorization with np.power()

Using the magic of np.power() to replace pow() allows some vectorization improvement to the hansenlaw algorithm.  I imagine/hope these types of code changes may be taken further ...

removes 'k' for loop

for k in range(K):  # Iterate over k, the eigenvectors?
            X[:, k] = pow(Nm, lam[k])*X[:, k] +\
                      h[k]*gamma(Nm, lam[k], n)*gp[:, n]  # Eq. (15 or 17)

(apparently) no longer requires the pseudo 1-column shift - e.g. run examples/example_all_O2.py

 #for some reason shift by 1 pixel aligns better? - FIX ME!
     if direction == "inverse":
         AIM = np.c_[AIM[:, 1:],AIM[:, -1]]

faster execution  ~30% improvement (n=501) cf PR #155

In [2]: abel.benchmark.AbelTiming(n=[301, 501, 1001])
Out[2]:
PyAbel benchmark run on x86_64

time in milliseconds

=========    Basis Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001
----------------------------------------------------------------------------
        basex_bs    6148.8           20052.1               nan
onion_peeling_bs       6.5              26.3             172.5
  three_point_bs      10.8              28.4             135.1
    two_point_bs       4.3              11.8              43.6

=========  Forward Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001
----------------------------------------------------------------------------
   direct_Python      65.0             297.8               nan
       hansenlaw       9.8              21.6              75.6

=========  Inverse Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001
----------------------------------------------------------------------------
           basex      15.0             120.1               nan
   direct_Python      74.8             313.0               nan
       hansenlaw      10.1              21.6              71.6
    onion_bordas     205.2             751.2            3187.9
   onion_peeling       4.2              18.8             140.6
     three_point       4.7              19.3             142.1
       two_point       4.7              19.4             143.7

@stggh, this looks like a nice improvement!
Can you make a comparison of both hansenlaw methods out to some large image size? (Is 5000x5000 feasible?)
I assume that this aggressively-vectorized hansenlaw will surely be faster for small images, but we should just make sure that it also works for larger images as well. (It will take more RAM, so this may be a problem for super-huge images. I'm just not sure what "super huge" is in this case.)
(Those benchmarks are from the Ubuntu computer with the slow Numpy, right? The Dasch methods look a little bit slow, and this is probably a result of us switching them to np.tensordot, which doesn't run quickly when linked with non-optimized linear-algebra libraries.)

Yes, I had also wondered about the image sizes effect.  Also, the direct_C is missing from the above benchmarks, for some reason it goes "missing" on my Ubuntu system.
New hansenlaw benchmarks for large n images:
n<3001
Implementation      n = 501           n = 1001          n = 2001          n = 3001
----------------------------------------------------------------------------------------------
       hansenlaw      20.1              73.5             262.7             570.9
   onion_peeling      18.5             138.5            1099.1            3614.5
     three_point      18.6             139.4            1083.9            3637.9
       two_point      18.7             143.1            1099.4            3649.3

n<10001  (only hansenlaw has a sensible execution time):
Implementation      n = 5001          n = 8001          n = 10001
----------------------------------------------------------------------------
       hansenlaw    1764.2            4079.1            6491.0

Old hansenlaw:
Implementation      n = 1001          n = 2001          n = 3001          n = 5001
----------------------------------------------------------------------------------------------
       hansenlaw      87.2             307.0             663.5            2006.4

=========  Inverse Abel implementations ==========

Implementation      n = 5001          n = 8001          n = 10001
----------------------------------------------------------------------------
       hansenlaw    1827.5            4583.2            7038.6

The relative execution time improvement for larger images is less.

Awesome! Thanks for doing that. Pretty much as expected. At some huge image size, "old" hansenlaw might be faster than new Hansenlaw. But, I think that we can optimize for images less than 10,000x10,000 :)
The improvement with the new Hansenlaw is great in the ~1000x1000 range.

I have not had much free time to play, but moving the k-vector loop outside of the column iteration slightly improves on the execution times from those above.  My gut feeling is that he i column loop may eventually be eliminated. Core hansenlaw.py code:
    # special case lam = 0
    hkgam[:, 0] = h[0]*gammagt(Nm, lam[0], nn)
    pNm[:, 0] = np.ones_like(Nm)

    # lam < 0
    for k in range(1, K):
        hkgam[:, k] = h[k]*gammalt(Nm, lam[k], nn)
        pNm[:, k] = np.power(Nm, lam[k])

    # Eq. (15) or (17)
    for i, n in enumerate(nn):
        X = pNm[i]*X + hkgam[i]*np.expand_dims(gp[:, n], axis=1)
        AIM[:, n] = X.sum(axis=1)

=========  Inverse Abel implementations ==========

Implementation      n = 501           n = 1001          n = 2001          n = 3001
----------------------------------------------------------------------------------------------
       hansenlaw      18.3              64.4             248.2             557.8
   onion_peeling      19.0             142.1            1097.0            3592.8
     three_point      19.3             140.3            1084.3            3864.9
       two_point      19.3             139.1            1088.3            3786.6

Implementation      n = 5001          n = 8001          n = 10001
----------------------------------------------------------------------------
       hansenlaw    1591.8            4009.1            5975.3

This is interesting - it looks like you are pre-computing something that looks like a basis set, and then you are applying this to the image, in a way that is independent of the row and summing over the columns. It seems to me that you are about one step away from re-phrasing the Hansen-Law transform in terms of a dot product, just like the other transforms.
It's interesting how this algorithm still scales much better than the other transforms as we go to huge image sizes.
You can feel free to merge this if you want. Or keep playing with it.

I'll keep playing with it over the weekend. Yes, execution time improvements, ultimately leading to the dot product, is the goal ;-)
@rth it is great to hear from you. PyAbel misses your clever python coding and in-depth discussion.

@DanHickstein

Those benchmarks are from the Ubuntu computer with the slow Numpy, right?

Yes. I love my ubuntu system because the PyAbel benchmark's show the hansenlaw method beating all other methods, once the image size becomes sufficiently large. The reality is, with the correct linear-algebra libraries, the Dasch methods (two_point, three_point, and onion_peeling) will always win.
Here are the comparative benchmark plots from the Ubuntu system and my Macbook pro, which while it has overall slower execution times, the Dasch methods are relatively much faster (~ x4):
a. Ubuntu

b. Macbook pro (note, 2pt, 3pt, and onion drop down due to the faster lin. alg. libraries installed on the Mac)

Your idea of saving basis calculations inspired a basis version of the hansenlaw method. However, it was a kludge to implement, with a different basis for the forward and inverse transforms, and two matrices to store Phi and Gamma. All, in the end, for virtually no gain, since the hansenlaw basis calculation is a trivial time component, as the figures above illustrate.

The basis version of hansenlaw caused substantial code changes to PyAbel, .npz for save(), get_bs_cached() relocated to tools/basis.py. The straw that broke the camel's back was various failures with the unit tests, which I see little reason to persevere with to fix.
Annoyingly, my Macbook has all sorts of issue with legacy installations, particularly, when running ipython.  I was puzzled why direct_C timings turned up in the pyabel-benchmark plots, but not when running ipython benchmark! I resorted to adding self.version = "some label" in benchmark.py (now removed), to ensure I was running the correct version. It would be nice to have something similar to abel.__version__ to be to differentiate between branches. Any ideas?

The bottom line is that I have not improved on Friday's fork hansenlaw-vec. I think the basis stuff discussed above just adds baggage, for no gain.

Due to the difference in timings between my Ubuntu system and my Macbook, I think that my timing improvement estimates, from the old hansenlaw code may have been inflated. However, IMO, the current code is more representative of the algebraic formulae in the Hansen and Law paper. I will run a few more tests tomorrow at work, before pushing the merge button.

I think that the 'black' look from the combined colors of all the O2- photoelectron spectra, indicates that we are doing a pretty good job, with all the algorithms. (Or the look could just be that the last method two-point has a black line, but my initial though was color blending.  ;-) )

(note I am not 100% happy with the alignment of the hansenlawinversion cf the Dasch methods on a spectroscopic scale, but this is unrelated to the current PR, and a matter for another discussion issue).

[OT:  my weekend of coding was not totally wasted, having managed to (legally) unlock my xiaomi redmi note 3 phone (Chinese version). Not easy when the applications have predominantly Chinese instructions, one is forced to make use of a windows PC, and google play is not installed.]

@stggh, thanks for doing all of that work to investigate the HansenLaw basis set open! Too bad it didn't speed things up too much.
Often, I get confused if I have the maser branch of my fork installed (using python setup.py develop). Using abel.__file__ usually resolves this. Does that work for you?
If you really want to get the real git version number, I think that you could use something like this: http://stackoverflow.com/questions/14989858/get-the-current-git-hash-in-a-python-script
The plots you posted in Item 4 are fantastic! The results of the various methods in the 2D image are basically indistinguishable. And the overlap is excellent in the lineout as well. (You could use alpha=0.3) or similar if you really want to see the overlap of the colors :).
Anyway, this is a great achievement of PyAbel, demonstrating that, when one is careful to apply the different transform methods on an equal footing, the results are quite similar. I wonder if there are situations (higher noise?) that could bring out more obvious differences between the methods.
Good work on the phone :)

@DanHickstein   develop and __file__, excellent! Thanks, I should have followed the instructions in our own docs CONTRIBUTING. rst and took the trouble to research the difference between that flag vs install.
alpha=0.3 is less dramatic, but more informative (run with a sampling of dr=0.1):

Expanded over the v'=1 band (2nd transition peak in from the right side):

Your questions are good, and will be examined following the issues of Jacobian=True scaling, and transform alignment, as illustrated in the figure above. hansenlaw has a definite offset, which ~<0.5 pixel. The forward transform, that almost the same code, for a 1d-Gaussian is fine.  I will open a separate issue to discuss this.

Merge PR summary
This PR was inspired by the super fast Dasch inverse Abel transform algorithms in PyAbel (two_point, three_point, and onion_peeling) that exploit Python vector execution using fast linear-algebra libraries. numpy.power() offered the prospect to improve the execution speed of the hansenlaw method, that contains (the source image independent) (N-n/N-n-1)λk terms.
The end result appears to be a factor of ~x2(timing from an arbitrary selected old PyAbel version, 21Feb16), but less of an improvement over more recent versions. However, there has been the added bonus a better understanding of the Hansen & Law algorithm.
Benchmarks on a Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz running Ubuntu 15.10, that unfortunately appears to lack the faster version of the linear-algebra libraries. The Dasch methods are ~4x slower than optimum:
PyAbel 21Feb16
Implementation |    n = 301          |    n = 501
 ----------------------------------------------------------
      HansenLaw |    15.05            |    31.37

cf this PR:
Implementation      n = 301           n = 501
----------------------------------------------------------
           basex      13.8             118.1
        direct_C      21.1              90.8
   direct_Python      69.1             318.4
       hansenlaw       7.3              19.0
    onion_bordas     215.5             661.4
   onion_peeling       4.0              19.0
     three_point       4.0              18.5
       two_point       4.1              18.7

Just as a follow up on the linear algebra libraries.
The stackoverflow [benchmarking-python-vs-c-using-blas-and-numpy](http://stackoverflo Oddly/sadly,  hansenlaw is slower.
w.com/questions/7596612/benchmarking-python-vs-c-using-blas-and-numpy) discussion, suggests the absence of import numpy.core._dotblas indicates that slow internal numpy linear algebra routines will be used.  Athough, additional comments mention that only the atlas distribution is important!

As of Jan. 2014 ATLAS is the recommended library to use as OpenBLAS will deadlock when used in combination with the multiprocessing module (or any use of os.fork) and older versions (<= 0.2.8) tend to crash when used on larger problem sizes.

On my Ubuntu 15.10 distribution I have found a difference between python and python3, the latter having _dotblas, returning the x4 execution time improvement for the basis methods basex, two_point, three_point and onion_peeling.
python3.4
etalon:/home/stg121]$ ipython3
Python 3.4.3+ (default, Oct 14 2015, 16:03:50)

IPython 2.3.0 -- An enhanced Interactive Python.

In [1]: import numpy.core._dotblas

In [2]: numpy.core._dotblas.__file__
Out[2]: '/usr/lib/python3/dist-packages/numpy/core/_dotblas.cpython-34m-x86_64-linux-gnu.so'

In [3]: import abel

In [4]: abel.__file__
Out[4]: '/home/stg121/Software/Python/VMI/PyAbel/abel/__init__.py'

In [5]: abel.benchmark.AbelTiming()
Out[5]:
PyAbel benchmark run on x86_64

time in milliseconds

=========    Basis Abel implementations ==========

Implementation      n = 301           n = 501
----------------------------------------------------------
        basex_bs    6778.4           21364.6
onion_peeling_bs       5.7              14.8
  three_point_bs      21.2              54.5
    two_point_bs       5.8              14.0

=========  Forward Abel implementations ==========

Implementation      n = 301           n = 501
----------------------------------------------------------
        direct_C      17.4              75.5
   direct_Python     130.4             532.2
       hansenlaw      11.9              27.9

=========  Inverse Abel implementations ==========

Implementation      n = 301           n = 501
----------------------------------------------------------
           basex       6.5              43.1
        direct_C      28.5             110.3
   direct_Python     139.5             539.2
       hansenlaw       9.0              21.0
    onion_bordas     399.5            1209.1
   onion_peeling       1.4               6.4
     three_point       2.1               5.4
       two_point       1.2               5.4

python2.7
[etalon:/home/stg121]$ ipython
Python 2.7.10 (default, Oct 14 2015, 16:09:02)

IPython 2.3.0 -- An enhanced Interactive Python.

In [1]: import numpy.core._dotblas
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-6afb7d943759> in <module>()
----> 1 import numpy.core._dotblas

ImportError: No module named _dotblas

In [2]: import abel

In [3]: abel.__file__
Out[3]: '/home/stg121/Software/Python/VMI/PyAbel/abel/__init__.pyc'

In [4]: abel.benchmark.AbelTiming()
Out[4]:
PyAbel benchmark run on x86_64

time in milliseconds

=========    Basis Abel implementations ==========

Implementation      n = 301           n = 501
----------------------------------------------------------
        basex_bs    6099.7           20639.7
onion_peeling_bs      13.5              29.0
  three_point_bs      18.2              47.3
    two_point_bs       6.0              11.2

=========  Forward Abel implementations ==========

Implementation      n = 301           n = 501
----------------------------------------------------------
        direct_C      16.3              73.3
   direct_Python      81.6             295.6
       hansenlaw       6.7              17.2

=========  Inverse Abel implementations ==========

Implementation      n = 301           n = 501
----------------------------------------------------------
           basex      13.7             132.7
        direct_C      21.4             101.2
   direct_Python      69.1             337.9
       hansenlaw       7.8              23.7
    onion_bordas     202.1             763.1
   onion_peeling       5.2              22.3
     three_point       4.0              19.0
       two_point       4.0              21.0

Here's a "big" 10,001x10,001 image comparison, python3.4:
 abel.benchmark.AbelTiming(n=[10001], select=['hansenlaw', 'two_point'])

=========    Basis Abel implementations ==========

Implementation      n = 10001
----------------------------------------
    two_point_bs    6196.0

=========  Forward Abel implementations ==========

Implementation      n = 10001
----------------------------------------
       hansenlaw    4715.5

=========  Inverse Abel implementations ==========

Implementation      n = 10001
----------------------------------------
       hansenlaw    5446.7
       two_point   44428.9

Here is a correct benchmark plot run with python3 that on my system uses the fast linear-algebra routines. For n > 2001 the hansenlaw method pulls ahead.
Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz   Ubuntu 15.10

Selected Abel methods, that will finish in a sensible time for large image sizes:
In [4]: abel.benchmark.AbelTiming(n=[301, 501, 1001, 5001, 10001], select=['hansenlaw', 'onion_peeling', 'two_point', 'three_point'], n_max_bs=10005, n_max_slow=10005)
Out[4]:
PyAbel benchmark run on x86_64

time in milliseconds

=========    Basis Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001          n = 5001          n = 10001
----------------------------------------------------------------------------------------------------------------
onion_peeling_bs       5.4              17.4              78.1            6852.1           49597.6
  three_point_bs      20.0              51.6             143.2            4360.8           18834.6
    two_point_bs       7.4              19.4              81.4            1471.9            6555.7

=========  Forward Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001          n = 5001          n = 10001
----------------------------------------------------------------------------------------------------------------
       hansenlaw      12.5              19.8              59.9            1325.0            4754.0

=========  Inverse Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001          n = 5001          n = 10001
----------------------------------------------------------------------------------------------------------------
       hansenlaw      12.1              27.7              89.4            1510.5            5591.1
   onion_peeling       1.2               5.3              42.0            5491.1           44699.8
     three_point       1.2               5.3              42.0            5544.3           45036.0
       two_point       1.2               5.4              49.3            5606.2           44368.5

@stggh, great work on this!
For diagnosing the peak-shift, perhaps we can generate a single-pixel step function, forward transform, and then look at the shifts of the various transform methods. It might be more sensitive than using the experimental datasets.
That is interesting to see that using Python3 gives you a shortcut to the better linear algebra libraries. Have you tried installing the Anaconda python distribution? The latest version of Anaconda uses MKL by default. MKL is supposedly the fastest math library. I don't think that it matters much on a Mac, since Apple's Accelerate Framework is similarly fast. But, there could be a big difference on Linux.
I think that MKL must support multithreading better than the linear algebra function in Scipy. I say this because, previously, I benchmarked PyAbel on our 32-core (4 cpu x 8 core) cluster and found that all transform methods were slower than on my MacBook. This makes sense for single-thread operation, since the cluster is only 2.4 GHz. However, benchmarks with the latest version of PyAbel (with the numpy functions) running with mkl.set_num_threads(32) show a significant improvement over single-processor operation.
Macbook (threads=2)
Implementation      n = 101           n = 501           n = 1001          n = 2001          n = 5001          n = 10001         n = 20001
----------------------------------------------------------------------------------------------------------------------------------------------------
   onion_peeling       0.1               5.2              25.3             191.2            3200.8           27746.1          314629.1
       two_point       0.1               3.4              25.2             196.2            3066.3           26836.9          298594.1

cluster (threads=1)
Implementation      n = 101           n = 501           n = 1001          n = 2001          n = 5001          n = 10001
----------------------------------------------------------------------------------------------------------------------------------
   onion_peeling       0.2              10.9              92.7             692.6           10825.7           82943.1
       two_point       0.1               9.5              92.4             582.1            9275.7           79939.5

cluster (threads=32)
Implementation      n = 101           n = 501           n = 1001          n = 2001          n = 5001          n = 10001         n = 20001
----------------------------------------------------------------------------------------------------------------------------------------------------
   onion_peeling       0.2               3.2              19.8             103.8            1235.6            9063.7           67490.4
       two_point       0.2               3.3              19.3              87.6            1228.4            9278.4           66719.1

Comparing the cluster running on 1 versus 32 threads, we see a ~10x speedup, which is pretty good! This enables the transform of 20,001x20,001 images in about 60 seconds, which puts the transform of gigapixel images on approachable timescale.

For the peak shift I will run various test functions, as you suggest, and then open an issue based on my findings.

@DanHickstein you certainly know your python. Anaconda is quit impressive, halves the execution time for the basis set transforms, but unfortunately does not do much for hansenlaw.  Use of threading looks interesting.

I also tried, just in case, swapping tensordot() for dot() in hansenlaw.py:
X = np.dot(Phi[i], X) + Gamma[i]*gp[:, n]

and it actually slows it down.
Anaconda python3 benchmarks running on Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz Ubuntu 15.10 compare with above
n [5]: abel.benchmark.AbelTiming(n=[301, 501, 1001], n_max_bs=1002, n_max_slow=1002)
Out[5]:
PyAbel benchmark run on x86_64

time in milliseconds

=========    Basis Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001
----------------------------------------------------------------------------
       basex_bs    6801.9           22526.6          166718.4
onion_peeling_bs       6.4               8.3              47.8
 three_point_bs      15.8              45.0             149.7
   two_point_bs       5.1              14.8              44.6

=========  Forward Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001
----------------------------------------------------------------------------
       direct_C      20.3              89.0             602.3
  direct_Python      82.8             423.2            3828.0
      hansenlaw       8.2              20.5              60.5

=========  Inverse Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001
----------------------------------------------------------------------------
          basex       2.5              22.5             156.1
       direct_C      20.8              85.3             625.0
  direct_Python      83.0             340.7            3761.5
      hansenlaw       9.0              17.3              59.9
   onion_bordas     364.4             769.9            6075.0
  onion_peeling       0.7               3.2              22.6
    three_point       0.6               3.5              22.1
      two_point       0.7               1.7              22.0 I

and here
=========  Inverse Abel implementations ==========

Implementation      n = 301           n = 501           n = 1001          n = 5001          n = 10001
----------------------------------------------------------------------------------------------------------------
      hansenlaw       8.7              19.0              63.1            1324.3            4785.1
  onion_peeling       0.7               2.7              21.4            2207.3           16890.9
    three_point       0.6               2.7              21.3            2332.5           17006.6
      two_point       0.7               2.7              21.8            2152.2           17395.5

The current benchmark plot, note ymin had to be set lower, to 0.01.


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from  to before v4.9.274",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=45a99589a9a3445dce114060f3b52f2df13bc79b"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "45a99589a9a3445dce114060f3b52f2df13bc79b",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "net: caif: fix memory leak in ldisc_open\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v4.9.274 by commit 45a99589a9a3445dce114060f3b52f2df13bc79b. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001099


Fixed NullpointerException when using Microwave.Builder#bake

see https://issues.apache.org/jira/browse/MEECROWAVE-108 for further infos!

Has already been merged. See svn.apache.org


etcdctl : wrong exit codes

Hello, i have problem, that etcdctl returns different exit codes that I assume it will from documentation :
0    Success
1    Malformed etcdctl arguments
2    Failed to connect to host
3    Failed to auth (client cert rejected, ca validation failure, etc)
4    400 error from etcd
5    500 error from etcd

When i query working etcd cluster and non-existing key i get exit code 4, which is okay :
root@redis:/# etcdctl get /runtime/redis
Error:  100: Key not found (/runtime/redis) [4401167]
root@redis:/# echo $?
4

But when I configure ETCDCTL_ENDPOINT to localhost, where not ETCD is running :
 root@redis:/# export ETCDCTL_ENDPOINT=http://<IP_ADDRESS>:2379
... and therefore query this key with wrongly configured etcd cluster connection I also get exit code 4, but from doc I assume it will be exit code 2 - Failed to connect to host, or am I wrong? :)
root@redis:/# etcdctl get /runtime/redis
Error:  dial tcp <IP_ADDRESS>:2379: getsockopt: connection refused
root@redis:/# echo $?
4

I have etcdctl version 2.2.4 :
root@redis2:/# etcdctl -v
etcdctl version 2.2.4

Error code 2 means bad connection. Error code 4 means server error.
So error 4 is expected because it's error from misconfigured cluster.

If you want to look at the code, https://github.com/coreos/etcd/blob/master/etcdctl/command/error.go.
Please let me know if you have any other question.
Thanks!

@arteal it's a bad connection because etcdctl can't establish a connection to etcd because it's being passed an address that doesn't point to an etcd server instance. The client is misconfigured (hence, bad connection), not the sever.

Ok, then text is a bit misleading, at least for me, because I have i see "etcd" as etcd server/cluster and "400 error" as return code, so I will just think that I will never get return code from server that I can not reach, but that kinda my "personal problem" and big thanks for your clarifiation.
Still I want to ask you if, you guys, can help to solve my original problem, i need something like this :
etcdctl get /runtime/redis 2>/dev/null
if [ $? -eq 4 ];then
 echo "var not found, create it"
  if [[ $(etcdctl set /runtime/redis "set") != "set" ]];then
    echo "cannot setup runtime var, exiting"
    exit 1
  fi
elif [ $? -eq 1 ]; then
  echo "error connection to ETCD cluster, exiting"
  exit 1
fi

Is this functionality possible with etcdctl or should I have to use some, for instance, Ruby client?
Thanks very much for patiance and answer!

@arteal Yes, use elif [ $? -eq 2 ]; instead.

@heyitsanthony : yeah, this is typo and there should have been elif [ $? -eq 2 ];, but still, there is problem why i created this issue : when connection to ETCD fail, etcdctl get return exit code 4, so my code is not working, because first if will be true if key is not found and also if connection fails and I need to make difference between these situations.

@arteal Right, I fixed that bug two months ago. I see the fix is in 2.3.0.alpha-1 but not 2.2.5.
@gyuho should 4477ef636e43fa6c8f9e45523a11377764c873df be backported?

Oh we might have forgot to include that patch. We can backport in the next release. Thanks!

This issue is fixed in master. We are doing a 2.3 release. It will come out probably next week.