ocis: init at 5.0.7

Description of changes

Build ownCloud Infinite Scale from source. It can be built successfully, but I do not have enough time to fully test the binary at the moment.
The EULA applies to release tarball. Can we safely mark this package as Apache 2.0 since we are building from source?
Related: https://github.com/NixOS/nixpkgs/issues/230190
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

I'm getting this error on startup:
{"level":"fatal","service":"idp","error":"open identifier/index.html: file does not exist","time":"2024-09-16T19:53:11-07:00","message":"Could not open index template"

This is my configuration:
{ config, inputs, system, ... }:

let
    cfg = config.services.ocis;
in
{
    services = {
        ocis = {
            enable = true;
            package = inputs.ocis.legacyPackages.${system}.ocis;
            environment = {
                NOTIFICATIONS_SMTP_HOST = "smtp.sendgrid.net";
                NOTIFICATIONS_SMTP_PORT = "587";
                NOTIFICATIONS_SMTP_SENDER =<EMAIL_ADDRESS>                NOTIFICATIONS_SMTP_USERNAME = "apikey";
            };
            environmentFile = config.sops.templates.ocis-secrets.path;
        };
        caddy.virtualHosts."files.haddock.cc".extraConfig = ''
            import auth
            reverse_proxy :${toString cfg.port}
        '';
    };

    sops = {
        templates.ocis-secrets = {
            content = # bash
            ''
                # IDs & passwords
                GATEWAY_STORAGE_USERS_MOUNT_ID=${config.sops.placeholder."haddock/ocis/gateway_storage_users_mount_id"}
                GRAPH_APPLICATION_ID=${config.sops.placeholder."haddock/ocis/graph_application_id"}
                IDM_IDPSVC_PASSWORD=${config.sops.placeholder."haddock/ocis/idm_idpsvc_password"}
                IDM_REVASVC_PASSWORD=${config.sops.placeholder."haddock/ocis/idm_revasvc_password"}
                IDM_SVC_PASSWORD=${config.sops.placeholder."haddock/ocis/idm_svc_password"}
                OCIS_JWT_SECRET=${config.sops.placeholder."haddock/ocis/jwt_secret"}
                OCIS_LDAP_BIND_PASSWORD=${config.sops.placeholder."haddock/ocis/ldap_bind_password"}
                OCIS_MACHINE_AUTH_API_KEY=${config.sops.placeholder."haddock/ocis/machine_auth_api_key"}
                OCIS_SERVICE_ACCOUNT_ID=${config.sops.placeholder."haddock/ocis/service_account_id"}
                OCIS_SERVICE_ACCOUNT_SECRET=${config.sops.placeholder."haddock/ocis/service_account_secret"}
                OCIS_SYSTEM_USER_API_KEY=${config.sops.placeholder."haddock/ocis/system_user_api_key"}
                OCIS_SYSTEM_USER_ID=${config.sops.placeholder."haddock/ocis/system_user_id"}
                OCIS_TRANSFER_SECRET=${config.sops.placeholder."haddock/ocis/transfer_secret"}
                STORAGE_USERS_MOUNT_ID=${config.sops.placeholder."haddock/ocis/storage_users_mount_id"}
                # OIDC
                # WEB_OIDC_CLIENT_ID=${config.sops.placeholder.ocis-oidc-client-id_ocis}
                # Mail
                NOTIFICATIONS_SMTP_PASSWORD=${config.sops.placeholder.sendgrid-api-key_ocis}
            '';
            owner = cfg.user;
        };
        secrets = {
            "haddock/ocis/gateway_storage_users_mount_id".owner = cfg.user;
            "haddock/ocis/graph_application_id".owner = cfg.user;
            "haddock/ocis/jwt_secret".owner = cfg.user;
            "haddock/ocis/ldap_bind_password".owner = cfg.user;
            "haddock/ocis/machine_auth_api_key".owner = cfg.user;
            "haddock/ocis/service_account_id".owner = cfg.user;
            "haddock/ocis/service_account_secret".owner = cfg.user;
            "haddock/ocis/storage_users_mount_id".owner = cfg.user;
            "haddock/ocis/system_user_api_key".owner = cfg.user;
            "haddock/ocis/system_user_id".owner = cfg.user;
            "haddock/ocis/transfer_secret".owner = cfg.user;
            "haddock/ocis/idm_idpsvc_password".owner = cfg.user;
            "haddock/ocis/idm_revasvc_password".owner = cfg.user;
            "haddock/ocis/idm_svc_password".owner = cfg.user;
            ocis-oidc-client-id_ocis = {
                key = "haddock/ocis/oidc_client_id";
                owner = cfg.user;
            };
            ocis-oidc-client-id_authelia = {
                key = "haddock/ocis/oidc_client_id";
                owner = "authelia-haddock";
            };
            ocis-oidc-client-secret_authelia = {
                key = "haddock/ocis/oidc_client_secret/authelia";
                owner = "authelia-haddock";
            };
            sendgrid-api-key_ocis = {
                key = "common/sendgrid/api_key";
                owner = cfg.user;
            };
        };
    };
}

Did you forget to copy this file over?

Hi @poperigby,
I've fixed the issue by running pnpm build in the buildPhase. Could you please test it out one more time?

I'm getting a hash mismatch:
ocis-pnpm-Progress: resolved 2049, reused 1316, downloaded 0, added 0
error: hash mismatch in fixed-output derivation '/nix/store/dc359zdj1bvhmgxyqq7s4i8vzf572fml-ocis-pnpm-deps.drv':
        likely URL: (unknown)
         specified: sha256-ojrgoyl+xBsgEsx3d3gv5Wf6ziabHoFkEpAoDZcYavo=
            got:    sha256-Aj7dSSbXvzbj/BQVEmU9VGSmCQPZNZPssdrDDvvUYF8=

Sorry, I'm not familiar with pnpm.configHook. It can built on my side with nixpkgs-review:

That's strange. I can't reproduce even if I remove ocis-pnpm-deps and rebuild. Here is the ocis-pnpm-deps I got:

nix derivation show /nix/store/c81c64ihp4w7ln32ajh8mhvqvx3k25pv-ocis-pnpm-deps

{
  "/nix/store/dc359zdj1bvhmgxyqq7s4i8vzf572fml-ocis-pnpm-deps.drv": {
    "args": [
      "-e",
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh"
    ],
    "builder": "/nix/store/vpvy79k1qq02p1vyqjk6nb89gwhxqvyb-bash-5.2p32/bin/bash",
    "env": {
      "__structuredAttrs": "",
      "buildInputs": "",
      "builder": "/nix/store/vpvy79k1qq02p1vyqjk6nb89gwhxqvyb-bash-5.2p32/bin/bash",
      "cmakeFlags": "",
      "configureFlags": "",
      "depsBuildBuild": "",
      "depsBuildBuildPropagated": "",
      "depsBuildTarget": "",
      "depsBuildTargetPropagated": "",
      "depsHostHost": "",
      "depsHostHostPropagated": "",
      "depsTargetTarget": "",
      "depsTargetTargetPropagated": "",
      "doCheck": "",
      "doInstallCheck": "",
      "dontBuild": "1",
      "dontConfigure": "1",
      "fixupPhase": "runHook preFixup\n\n# Remove timestamp and sort the json files\nrm -rf $out/v3/tmp\nfor f in $(find $out -name \"*.json\"); do\n  jq --sort-keys \"d
el(.. | .checkedAt?)\" $f | sponge $f\ndone\n\nrunHook postFixup\n",
      "installPhase": "runHook preInstall\n\nlockfileVersion=\"$(yq -r .lockfileVersion pnpm-lock.yaml)\"\nif [[ ${lockfileVersion:0:1} -gt 9 ]]; then\n  echo \"ERROR: l
ockfileVersion $lockfileVersion in pnpm-lock.yaml is too new for the provided pnpm version 9!\"\n  exit 1\nfi\n\nexport HOME=$(mktemp -d)\npnpm config set store-dir $out
\n# Some packages produce platform dependent outputs. We do not want to cache those in the global store\npnpm config set side-effects-cache false\n# As we pin pnpm versi
ons, we don't really care about updates\npnpm config set update-notifier false\n# Run any additional pnpm configuration commands that users provide.\n\n# pnpm is going t
o warn us about using --force\n# --force allows us to fetch all dependencies including ones that aren't meant for our host platform\npnpm install \\\n    --force \\\n
 --ignore-scripts \\\n     \\\n    --frozen-lockfile\n\nrunHook postInstall\n",
      "mesonFlags": "",
      "name": "ocis-pnpm-deps",
      "nativeBuildInputs": "/nix/store/q35mwzks79767rz6agab7919c6w22s4l-nss-cacert-3.101.1 /nix/store/rwypdgzzxk9740bd01irnq708p62axah-jq-1.7.1-dev /nix/store/sww15lfpa8
m7m05rcx6fig7visb94yyk-moreutils-0.69 /nix/store/1y9cr6yng82lkfk6yq7w60x45d726ffi-pnpm-9.7.0 /nix/store/ahch83vgag79gsiyvmqh6m1w6wwv9b9s-python3.12-yq-3.4.3",
      "out": "/nix/store/c81c64ihp4w7ln32ajh8mhvqvx3k25pv-ocis-pnpm-deps",
      "outputHash": "sha256-ojrgoyl+xBsgEsx3d3gv5Wf6ziabHoFkEpAoDZcYavo=",
      "outputHashMode": "recursive",
      "outputs": "out",
      "patches": "",
      "propagatedBuildInputs": "",
      "propagatedNativeBuildInputs": "",
      "sourceRoot": "source/services/idp",
      "src": "/nix/store/ccsmdfqdv3qfp7blm16k48j0h9jmpdw9-source",
      "stdenv": "/nix/store/ipjwd2g1kbfra83kd0dglx26xv7kn50n-stdenv-linux",
      "strictDeps": "",
      "system": "x86_64-linux",
      "version": "v5.0.7"
    },
    "inputDrvs": {
      "/nix/store/1yfcpc8qbg790k9jmcr7267ixijl11vq-pnpm-9.7.0.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/6sn8q8ssw50vvski2mifkyl8fqjjhbzs-stdenv-linux.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/7sayjrlawsxwan7pgd6ghxd4ibg3kwzv-python3.12-yq-3.4.3.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/bfl71xj1zs632dydnaf2cagsw4y5i2i2-moreutils-0.69.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/lvnmaggm7nb8nndjps5lkmmcig0spdrr-bash-5.2p32.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/mfgxm2lx7akg6jilz52y2bzbzmb65dwi-source.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/ndrz7pvwjr31nirb2v16h1g5z8ngxvi5-nss-cacert-3.101.1.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/pnmywf05lvzrallwhyim293gv74pcykj-jq-1.7.1.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "dev"
        ]
      }
    },
    "inputSrcs": [
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh"
    ],
    "name": "ocis-pnpm-deps",
    "outputs": {
      "out": {
        "hash": "a23ae0a3297ec41b2012cc7777782fe567face269b1e81641290280d97186afa",
        "hashAlgo": "sha256",
        "method": "nar",
        "path": "/nix/store/c81c64ihp4w7ln32ajh8mhvqvx3k25pv-ocis-pnpm-deps"
      }
    },
    "system": "x86_64-linux"
  }
}

I guess it's a pnpm issue? I will try on another machine.

@ofborg build ocis

Tested on my machine with a minimal config:
{ config, pkgs, ... }:
{
  services.ocis = {
    enable = true;
    package = pkgs.ocis;
    stateDir = "/var/lib/ocis";
    url = "https://drive.xinyang.life:8443";
    address = "<IP_ADDRESS>";
    port = 9200;
    configDir = "/var/lib/ocis/config";
    environment = {
      OCIS_INSECURE = "false";
      PROXY_TLS = "false";
      OCIS_LOG_LEVEL = "debug";
      OCIS_LOG_PRETTY = "true";
      WEB_HTTP_ADDR = "<IP_ADDRESS>:12345";
    };
  };
}

Super weird. The first time I tried building after pulling from your branch, pnpm had a segmentation fault, the second time it worked. No hash mismatch.

@poperigby That's super weird. It's hard to diagnose if it's not reproducible.
I did a quick dive into pnpm.fetchDeps, though. It turns out that it uses proxy environment variables and several native dependencies when calculating the hash. Are you using proxies or have an overlay on these packages? That's my best guess.

No, no overlays or proxies as far as I can tell.

Add updateScript
Bump to 5.0.9 from 5.0.7


Union bySegment queries can get confused about which descriptors to use

When historicals get a union query with multiple datasources, they can get confused and scan segments other than the ones the broker intended. This is because they use UnionTimeLineLookup.findEntry(interval, version) to resolve the descriptors into Segments, but it's possible for segments in different datasources to have the same interval and version. This is actually always going to happen if the datasources were ingested with standalone realtime, since the version is fixed to the start of the hour.
This shows a couple of union queries where one segment is scanned twice and one is not scanned at all (which one is scanned depends on the order of the datasources; see union.json vs union2.json): https://gist.github.com/gianm/fc3a1eba0dbafbf87720

@nishantmonu51

Ah, the reason behind introducing UnionTimelineLookup at the historical was to distribute the merging load on both historical and broker.
It seems the historical does not have enough information to work on multiple DataSources here.
we would either need to add the dataSource info to the MultipleSegmentSpec or revert back to the old way of letting the broker handle breaking the query into individual datasources and send query for individual datasources down to the historical nodes.
I think we could revert back the breaking changes and let the broker send the query for individual DS down to the historical nodes. @fjy @gianm I will make a PR for reverting them if it sounds good to you ?

@nishantmonu51 that approach sounds good to me- I think the other alternative would be to add dataSources to SegmentDescriptors, which seems like a big change for a feature that is not very heavily used.


[Improvement]  Separate flush thread pools for different storage type.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

What would you like to be improved?
In the current MEMORY_LOCALFILE_HDFS mode, the flush thread pool is multiplexed, but for machines with a small number of local disks, the number of configured flush threads is small, for example, only 4 flush threads are configured, which is sufficient for writing to local disks, but for For writing HDFS, the number of threads is too small.
How should we improve?
Maybe we can separate flush thread pools for different storage type.
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

@jerqi  @zuston  @smallzhongfeng  PTAL.

+1 for this feature, please go head.


Support for external fonts

Made the formula parser to have instance constructors
Restructured the parser to have support for external symbols and text style dependent Greek letters.
Created a scheme for using different formula settings and symbols(by zipping and unzipping the packaged file).

@B3zaleel, thank you, this is very cool. How did you generate this .wmpkg file? It includes very interesting XML files I'd like to know about.

I used the fonttools Python module to extract the symbols from the font file and FontForge to read some other information like the general parameters (sup1, bigopspacing1, etc) and to change the location of certain glyphs.


Wrapping subselects in ARRAY()

I have a subselect that returns multiple rows.
To get the values out of the DB, I need to wrap the subselect in ARRAY().
There doesn't seem to be a way to achieve this without writing a raw query.
My use case is only a small variation on example http://vincit.github.io/objection.js/#getting-count-of-relations
e.g
const tweets = await Tweet
  .query()
  .select(
    'Tweet.*',
    raw('ARRAY(?)', Tweet.relatedQuery('likers').select('id')).as('likerIds')
  );

console.log(tweets[4].likeIds);

Any advice @koskimas ?

So the first example doesn't work? What is it that you want? How doesn't it work?

You need to move the alias outside the ARRAY though like this:
const tweets = await Tweet
  .query()
  .select(
    'Tweet.*',
    raw('ARRAY(?) as "likerIds"', Tweet.relatedQuery('likers').select('id'))
  );

console.log(tweets[4].likeIds)

That fixes. Impressive as always, apologies for the lacklustre issue logging ;)
Something like a Model.QueryRelation that lets these subselects be defined as relations and manipulated with relation expressions could be interesting...

Great! 1.2.3 is now out and contains this fix.
That kind of QueryRelation is not possible to implement. It wouldn't work with most of the stuff people do with relations and eager.

Nice work. What kind of stuff do people do, if you can elaborate?

Relations need to work with relatedQuery, eager, joinEager, upsertGraph, insertGraph, composite keys, json keys, etc. There is no way to allow users to provide their own arbitrary query and make all those use cases work.

Ah of course and they all support insert/update/patch.
It feels like we're missing a trick around relation expressions. They give me a standardized way to manipulate what other expensive parts of a model are included.
Sami is there some way a plugin could latch onto the relationexpr infrastructure and provide some way to include/exclude/filter subselects or joined fields?


Move to crdt interpreter

Seems like you're using an odd npm version: it has added 15k of lines to package-lock.json


Building ALCOM locally fails

Description of the bug / バグの説明
I have the issue that I often can't get build to run, only dev.
git clone --recurse-submodules https://github.com/vrc-get/vrc-get.git
in vrc-get-gui -> npm run tauri build
It is always about the licenses, navigating there will crash dev.
What am I doing wrong?
Steps to Reproduce / 再現手順

clone repo "git clone --recurse-submodules https://github.com/vrc-get/vrc-get.git"
navigate to vrc-get-gui
npm run tauri build

Expected Behavior / 想定される挙動
It should build.
Screenshots / スクリーンショット

or

Software Version / バージョン情報
0.1.10-beta
OS
Windows 11
Additional context / 補足
No response

Please try upgrading cargo about

cargo-about isn't part of the Cargo.toml, so updating it apparently wont work, only installing it would.

I ran cargo install --locked cargo-about outside my project directory, and it now seems to build.
How would I have known about this if you didn't tell me?
Won't the project I try to build specify the version it needs if it depends on it?

ALCOM is depending on the latest version of cargo about and rust currently and there's no specific version ALCOM is depending on.
I didn't write about this in any documentation so I'll document this.
I've planned to leave cargo about in the future since many futures in cargo about is not necessary for this project.


chore: commit modified files after npm run build

I'm not sure if this is intended, but locally after running npm run build I see untracked changes?

You need to run npm run format after that. This is the unformatted version of the file TanStack Router generates.

Oh gotcha, thanks!


Using dockerode with self-signed certificate chains

This is an information that should be added to the getting started section in the README. It cost me hours to get it working - so I'm happy to share my experience:
This is how to instantiate dockerode with a self-signed certificate chain:
docker = new Docker({
	protocol: 'https',
	host: my-docker-host.com, // host name must match the cert's common name
	port: 2376,
	ca: [
		fs.readFileSync('ca.pem'),
		fs.readFileSync('intermediateCA.pem')
	],
	cert: fs.readFileSync('cert.pem'),
	key: fs.readFileSync('key.pem')
});

This might also be related to issue #154
Hopefully this will be helpful for others, too.

Hostname must always match the cert's common name, it's a thing related with docker of even dockerode, even if it was a simple web server it must match :)

Well, my real issue was not related to the common name. That's obvious and can be found in every SSL/TLS-related howto.
My real problem with the self-signed certificate chain was that the ca property has to be specified as an array of each and every CA and intermediate CA:
	ca: [  // specify the certificate chain as array of the CA and all intermediate CAs
		fs.readFileSync('ca.pem'),
		fs.readFileSync('intermediateCA.pem')
	]

Such information can be found anywhere in the net. Furthermore, the common procedure to concatenate the CA and the intermediate CAs into a single file doesn't work here.
I've finally found necessary information in a side note in the node.js docs: https://nodejs.org/api/tls.html#tls_tls_connect_options_callback


Add social link for Keybase

Description
Allows users to add Keybase as one of their social links.
How Has This Been Tested?

Added my keybase name to keybase in config .toml
Verified that it rendered correctly in the list of social icons.
Removed my keybase name from keybase in config.toml
Verified that keybase no longer appeared in the list of social icons.

Hugo Version: 0.37
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation, including theme.toml, accordingly.
[x] I have read the Contributing Document.

Thanks for the quick review @pacollins. I agree, using fa-key isn't ideal.  The trouble is that Font Awesome have yet to add a keybase icon.
See https://github.com/FortAwesome/Font-Awesome/issues/4095, where the recommended approach is to use fa-key for now.
A keybase icon was recently added to the fork "Fork-Awesome", but that doesn't help here unfortunately. :(

Closing this pull request now that we have FontAwesome 5 and the real Keybase social icon. 🎉


[WIP] Add AnchorTargetCreator

Merge after #148.

Since it is complicating to handle multiple PRs at same time, the content of this is PR will be merged to #154 for now.


Allow brod_consumer to spawn kafka connection

Prior to this commit, partition leader connections
are managed by brod_client and shared between
partition workers (brod_producer:s and brod_consumer:s)
Now bootstrap hosts (plus connection config) is supported
as first arg to brod_consumer:start_link.
This should allow users to link or supervise brod_consumer
instead of always under brod_client's brod_consumers_sup.

fixed and rebased.

Pull Request Test Coverage Report for Build 1143

33 of 35 (94.29%)  changed or added relevant lines in 1 file are covered.
9 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.03%) to 80.516%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/brod_consumer.erl
33
35
94.29%

Files with Coverage Reduction
New Missed Lines
%

src/brod_client.erl
3
84.7%

src/brod_consumer.erl
6
78.95%

Totals

Change from base Build 1135:
-0.03%

Covered Lines:
2124

Relevant Lines:
2638

💛  - Coveralls


No Enabled in examples

The examples are all missing the Enabled parameter, as well as a password parameter.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1fb61649-0f07-ff16-3fbe-4029b9119c7e
Version Independent ID: 5f3c184f-42bd-176e-5764-5411c3227c6e
Content: New-ADUser (ActiveDirectory)
Content Source: docset/winserver2019-ps/activedirectory/New-ADUser.md
Product: w10
Technology: windows
GitHub Login: @JasonGerend
Microsoft Alias: jgerend

@Multipanda Thank you for your feedback. We have made changes to the document via PR#https://github.com/MicrosoftDocs/windows-powershell-docs/pull/2809
Thanks
Sri

@Multipanda Thank you for your feedback. We have made changes to the document via PR##2809
Thanks Sri

Great to see, thank you.
Though it seems the merger was blocked, though I guess we just have to wait for the right person to see it and confirm the merger, or is it something you can do? Not sure how such things work.

@Multipanda That is correct. The merger was blocked as it needs approval from the author. We have the details in the document now.
@e0i This can be closed.
Thanks
Sri


Access Properties of the Node

Hello Massimo Saccani
Describe the bug
Not a bug but after sending you a request through Paypal, the response you gave was to post my questions here.

The status is displayed below the "Ovest" node in the sample flow.  I would like to access this status and return it to the dashboard so the user has an idea of what is going on in the background.  Where can this property be retrieved?

In the debug window of Node-Red, I see the BANANA PROCESSING object getting refreshed continually, I assume this is normal behavior but wonder if there is a way to stop it when the camera is not in use by my application?

Is it possible to see the live video stream?

The camera I'm currently testing with has audio input, output and alarm capabilities.  It is not clear how to access these features from my Node-Red application that is running on a Raspberry Pi.  Once my application is finished, I would like to have two-way conversations from my browser to the camera.  The application is a doorbell/access control project where I'm assuming hte camera alarm will trigger an MQTT notification at which point the user will visit a webpage that has audio out, a push to talk button to control the mic and if access is ok, the alarm output will release a magnetic door lock.

Any response to these four questions would be greatly appreciated.
Best,
Harvey

Hi

There is a “status” node, that grabs the status of other nodes. It’s a node shipped with node-red 👍🏻
Check wether near the camera’s IP address (in the node configuration) there is a “banana”. If yes, yes remove it. It’s a command to log all things.
No. I suggest you not to do so via node-red. Rather, if exists, use a dashboard image node and link it to the camera stream directly. It’s pretty easy and you won’t overload your raspberry.
It’s nearly impossible to do. Hikvision hasn’t released an ISAPI that allow to do so. I’ve struggled a lot, with the help of some Homeassistant devs, without success. I ended up attaching a shelly input to the alarm output of my Hikvision doorbell and catch the event via the shelly node.


remove underlines from patterns in code blocks

This took a while. Before, patterns would render like this:

See how root pattern is underlined.
The reason textmate grammar was using some rules called string.other.link. These rules are set to render underlines in the github-dark and github-light shiki themes, which is what vitepress use as default.
After this change:

should we make a PR for https://github.com/ocamllabs/vscode-ocaml-platform/blob/master/syntaxes/reason.json? that's where our reason.json was copied from. But I guess this is more a matter of styling, it doesn't really cause issues in VS Code.

I think it depends on the actual implementation of the Theme. VSCode might style links differently?


CSRF on DELETE requests

I can't for the life of me avoid hitting csrf errors when sending a DELETE request via axios. I've tried adding the header, adding the csrf field to the body, and even doing both. As soon as I change it to a POST request, everything's fine
POST /collections/:id/items/delete is really bugging me. I'd like to be able to do DELETEs :package:
Anyone know why this is happening?

Nope. https://github.com/expressjs/csurf

I guess I should pull out fiddler and see if it's axios or csurf that's causing the problem

Axios explicitly ignores request bodies on DELETE requests. I guess this can't be fixed


tarod v0.1.1-alpha Not Launching Ubuntu 20.04 Bitcoin Core 24.99 LND 0.15.4 Testnet

Expected behavior
Followed the video tutorial here:
https://youtu.be/xtklaJHfKIY
And the instructions here:
https://github.com/lightninglabs/taro

Once I launch tarod I expected it to connect to the testnet running on the same machine
Actual behavior
When launching using:
tarod --network=testnet --debuglevel=debug --lnd.host=localhost:10009 --lnd.macaroonpath=~/.lnd/data/chain/bitcoin/testnet/admin.macaroon --lnd.tlspath=.lnd/tls.cert
The output is
2022-11-30 11:39:57.567 [WRN] CONF: open /home/opb/.taro/taro.conf: no such file or directory
2022-11-30 11:39:57.568 [INF] CONF: Opening sqlite3 database at: /home/opb/.taro/data/testnet/taro.db
2022-11-30 11:39:58.528 [INF] CONF: Attempting to establish connection to lnd...
error creating server: unable to connect to lnd node: error checking connected lnd version. at least version "v0.15.99, build tags 'signrpc,walletrpc,chainrpc,invoicesrpc'" is required

To reproduce
Running the lnd version 0.15.4-beta make && make install tags=”signrpc walletrpc routerrpc invoicesrpc”
The lnd and lncli do compile then running lnd –network=testnet getinfo
returns:
"version": "0.15.4-beta commit=v0.15.4-beta"...
"testnet": true
"chains": [
{
"chain": "bitcoin",
"network": "testnet"
}
]
System information
Using Bitcoin Core version 249900
Ubuntu 20.04
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          8
Model name:                      Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
Disk type: sda

You need lnd at commit ca501d92 (that exact version, everything later on the master branch is broken and only fixed by #206). Do git checkout ca501d92 && make && make install tags="signrpc walletrpc routerrpc invoicesrpc"

@guggero thanks for the prompt response.

Do git checkout ca501d92 && make && make install tags="signrpc walletrpc routerrpc invoicesrpc"

updated lnd it is now  "version": "0.15.99-beta commit=cert/v1.2.0-119-gca501d920" as you instructed however still getting that same error from inputting
tarod --network=testnet --debuglevel=debug --lnd.host=localhost:10009 --lnd.macaroonpath=~/.lnd/data/chain/bitcoin/testnet/admin.macaroon --lnd.tlspath=.lnd/tls.cert
2022-11-30 15:44:24.185 [WRN] CONF: open /home/opb/.taro/taro.conf: no such file or directory
2022-11-30 15:44:24.186 [INF] CONF: Opening sqlite3 database at: /home/opb/.taro/data/testnet/taro.db
2022-11-30 15:44:24.190 [INF] CONF: Attempting to establish connection to lnd...
error creating server: unable to connect to lnd node: error checking connected lnd version. at least version "v0.15.99, build tags 'signrpc,walletrpc,chainrpc,invoicesrpc'" is required

Does the lnd node need to have open channels for Taro to work/connect?

Ah, I just copy pasted your command but according to the error message you also need to add the chainrpc build tag.
No channel required but on-chain funds in lnd's wallet.

SUCCESS!
Thanks @guggero!
Now gonna make some "Magic Internet Money" ;)


Demangled OpenMP parallel procedure has the same name as the procedure itself

Compiler: g++ 6.4.1
Application: Lulesh
Platform: ori (x86_64)
Demangling OpenMP parallel loops (which are translated to OpenMP parallel procedure) has the same name as the procedure itself.
For instance, in procedure CalcEnergyForElems, it has 5 parallel loops. The demangled name of these loops is also CalcEnergyForElems which is confusing. Furthermore, hpcviewer bottom-up view assumes that all procedure has different name, and it doesn't know to distinguish between the real procedure and the OpenMP parallel procedure since they have the same name.
I propose hpcprof to detect if a procedure has the same demangled name, but different mangled name, it has to append it with something like a line number (if available) to distinguish between different parallel loops.

Output from gdb shows that all OpenMP parallel regions in g++ 6.4.1 has the same demangled name (CalcEnergyForElems) if they are in the same function, but they have different mangled names (_ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.XX):
$56 = 0xd5d970 "CalcEnergyForElems"
$58 = 0x10fe530"_ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.19

$60 = 0x1654d60 "CalcEnergyForElems"
$62 = 0x148a160 "_ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.20"

$64 = 0x1321250 "CalcEnergyForElems"
$66 = 0xf4d5e0 "_ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.21"

$68 = 0xd422d0 "CalcEnergyForElems"
$70 = 0x1138ad0 "_ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.22

$72 = 0xd41450 "CalcEnergyForElems"
$74 = 0xdfd770 "_ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.23"

Problem in hpcviewer when clicking "show hot path" on flat view and bottom-up view, due to the same procedure name (I guess). This problem doesn't occur in top-down view.
!ENTRY org.eclipse.ui 4 0 2019-03-26 15:24:36.420
!MESSAGE Unhandled event loop exception
!STACK 0
org.eclipse.core.runtime.AssertionFailedException: null argument:
        at org.eclipse.core.runtime.Assert.isNotNull(Assert.java:85)
        at org.eclipse.core.runtime.Assert.isNotNull(Assert.java:73)
        at org.eclipse.jface.viewers.AbstractTreeViewer.getTreePathFromItem(AbstractTreeViewer.java:2925)
        at org.eclipse.jface.viewers.AbstractTreeViewer.internalGetWidgetToSelect(AbstractTreeViewer.java:1730)
        at org.eclipse.jface.viewers.AbstractTreeViewer.internalExpand(AbstractTreeViewer.java:1625)
        at org.eclipse.jface.viewers.AbstractTreeViewer.setSelectionToWidget(AbstractTreeViewer.java:2519)
        at org.eclipse.jface.viewers.StructuredViewer.setSelectionToWidget(StructuredViewer.java:1770)
        at org.eclipse.jface.viewers.AbstractTreeViewer.setSelectionToWidget(AbstractTreeViewer.java:2992)
        at org.eclipse.jface.viewers.StructuredViewer.setSelection(StructuredViewer.java:1726)
        at org.eclipse.jface.viewers.TreeViewer.setSelection(TreeViewer.java:1077)
        at edu.rice.cs.hpc.viewer.scope.ScopeViewActions.showHotCallPath(ScopeViewActions.java:298)
        at edu.rice.cs.hpc.viewer.scope.ScopeViewActionsGUI$3.widgetSelected(ScopeViewActionsGUI.java:488)
        at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:249)
        at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
        at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5227)
        at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1340)
        at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4561)
        at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4151)
        at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$4.run(PartRenderingEngine.java:1121)
        at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
        at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1022)
        at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:150)
        at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:693)
        at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
        at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:610)
        at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148)
        at edu.rice.cs.hpc.viewer.framework.HPCViewer.start(HPCViewer.java:28)
        at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
        at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
        at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
        at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388)
        at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:673)
        at org.eclipse.equinox.launcher.Main.basicRun(Main.java:610)
        at org.eclipse.equinox.launcher.Main.run(Main.java:1519)
        at org.eclipse.equinox.launcher.Main.main(Main.java:1492)

@jmellorcrummey : I assume this is not a problem with other compilers (like clang).

It is important to handle this problem. In several cases, we have seen the demangler omit a suffix like _omp_fn.19. I think this issue occurs only for C++. I think that the shortened names without the suffix come from the demangler. I am not sure whether it is the demangler Dyninst or in binutils for this particular case. I know that the demangler in Dyninst has this problem because Marty and I have seen that the link name (ln) in an hpcstruct file contains the suffix but the name (n) is just the shortened version without the suffix.
@mwkrentel please investigate.
While you are at it, another demangler problem we have discussed before is present on arm and power but not x86_64 with hpcstruct using Dyninst 10.0.
In the following test. Compile this with
gcc -o demangle demangle.c -fopenmp
In hpcstruct output, the procedure with link name "a" (ln="a") ends up having demangled name "signed char" (n="signed char") on arm1 and on po. This happens with hpcstruct built with Dyninst 10.0.
On arm1, I used the default gcc 7.3.1
On po, I used gcc 6.4.0
It gets the right result on  gpu.cs using gcc 7.3.0

#include <omp.h>
#include <stdio.h>

#define N 40
#define T 4

void
loop(){
        int j;
        for(j=0;j<1000000000;j+=2) j--;
}

void
a()
{
        #pragma omp parallel num_threads(T)
        {
                loop();
                printf("Outer region: ID = %d\n", omp_get_thread_num());
        }
}

int
main()
{
  a();
  return 0;
}

The short answer from hpcstruct's view is GIGO (no offense).
I searched in Laksono's directory on ori.cs for a lulesh binary and
found lulesh-RAJA-parallel.exe in /home/la5/codes/test/perf_test/app/.
I've put a copy in my home directory on mic.cs.
I found two functions at 0x406dd0 and 0x409420 which illustrate
the problem.
Both nm and objdump report the mangled and demangled names for
0x406dd0 as:
0000000000406dd0 t _ZN4RAJA6forallIZ18CalcEnergyForElems ... SetET_._omp_fn.43

0000000000406dd0 <void RAJA::forall<CalcEnergyForElems(double*, ...
    ::{lambda(int)#1}) [clone ._omp_fn.43]>:

And 0x409420 as:
0000000000409420 t _ZN4RAJA6forallIZ18CalcEnergyForElems ... SetET_._omp_fn.42

0000000000409420 <void RAJA::forall<CalcEnergyForElems(double*, ...
    ::{lambda(int)#1}) [clone ._omp_fn.42]>:

I also tried the g++ builtin __cxa_demangle() and libiberty
cplus_demangle() functions directly.  Same answers as nm and objdump.
So, the correct answer is there in the binary.  Both the mangled and
demangled names are different.  But the answer from Symtab doesn't
have [clone ._omp_fn.43] making the pretty names look the same.
@mxz297 does that seem like what's happening?
Is Symtab dropping [clone ._omp_fn.43] for some reason?
I think that's what struct gets from pretty_names_begin().
Note: hpcstruct never does any demangling on its own (unless you're
using the --demangle-library and --demangle-function options).
All of the demangling comes from Symtab.
Also note: although it's easy to call cplus_demangle, the hard part
is deciding when to demangle.  Not every name is mangled and
demangling a non-mangled name gives the wrong answer.  (I think this
is what's happening with the a --> signed char problem.)
What you have to do, and I assume what Symtab already does, is for
every symbol, see what module (CU) it comes from and what the source
language is.  Demangle if the language is C++.
So, no I don't think that adding a line number to the name is the
right solution.  Instead, we should first confirm what Symtab is
doing, if it's chopping the name, and if so, then fix that.
Btw, comparing g++ __cxa_demangle with libiberty cplus_demangle,
cplus_demangle has several options for how to demangle.  But
__cxa_demangle has no options, you get one answer, no way to
customize it.
In particular, with cplus_demangle, you can demangle type names or
not.  So, a is not a valid function name encoding, it's a type
encoding, signed char.  With the type names option turned off, then
cplus_demangle reports that a is not a mangled name.  But
__cxa_demangle always returns signed char.  Maybe cxxabi has some
other way to make this distinction, I don't know.
So, maybe Dyninst should look at whether it's better to use
cplus_demangle over __cxa_demangle.

I was using the binary on ori from different directory:
/home/la5/git/hpctoolkit-tests/applications/lulesh/lulesh2.0

Same problem.  In the lulesh2.0 binary, there are 5 procs for openmp
loops in CalcEnergyForElems, for example:
<P i="4236"
   n="CalcEnergyForElems"
   ln="_ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.19"
   l="2091"
   v="{[0x408950-0x408951)}">

For all 5, Symtab provides the mangled name as above and the pretty
name as CalcEnergyForElems.
I don't understand why Symtab would demangle this name as the short
form like this.  Either this is a bug in Symtab (maybe some odd corner
case), or else there's something going on here that I don't understand.
We need to check with @mxz297.

@mwkrentel @jmellorcrummey @laksono
I feel there is some abstraction gap here between what dyninst provides and what you wants. For example, a SymtabAPI::Symbol can provide the following three types of names:
getMangledName(): _ZL18CalcEnergyForElemsPdS_S_S_S_S_S_S_S_S_S_S_S_dddddS_S_ddiPi._omp_fn.19

getPrettyName():CalcEnergyForElems

getTypedName():CalcEnergyForElems(double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double, double, double, double, double, double*, double*, double, double, int, int*) [clone ._omp_fn.19]

Based on your descriptions, hpcstruct seems to use information from getPrettyName().
In my impression, getPrettyName() should give what the programmer writes in the source code.  getTypedName() provides the complete demangled name, which will include type information and whatever other information expressed in the mangled name (in this case, it contains [clone ._omp_fn.19]).
The abstraction gap is that openmp parallel procedures are not really individual source functions. At this point, I feel SymtabAPI is doing what it is designed to do. However, it is not doing what you want.
Are you good with complete demangled name? If you are, then you can switch to use getTypedName(). If not, what exactly do you want? Do you just want exclude types?

We will test with getTypedName()

Moving today's discussion into the issue from email ...
At 2:32pm @mwkrentel said:
First, let me check, what method does Symtab use to demangle names?
Is it the C++ __cxa_demangle() function?
Does Symtab make any adjustments to the answer it gets on its own?
I think we should first settle on what Symtab wants to produce with
mangled, pretty and typed names.  And I think Dyninst should decide
this based on what makes sense in general, not something specific to
hpctoolkit's needs.
Specifically, how much of the type annotations are supposed to be in
the pretty name?  I bring this up because I think we're getting
inconsistent answers from Symtab.
For example, in the first lulesh-RAJA-parallel.exe binary that I
looked at, there's one function, SumElemFaceNormal() at 0x40b450 with
mangled, pretty and typed names:
_Z17SumElemFaceNormalPdS_S_S_S_S_S_S_S_S_S_S_dddddddddddd
SumElemFaceNormal
SumElemFaceNormal(double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double, double, double, double, double, double, double, double, double, double, double, double)
Fine.  But in the same binary, CalcEnergyForElems() looks like:
ZN4RAJA6forallIZ18CalcEnergyForElemsPdS1_S1_S1_S1_S1_S1_S1_S1_S1_S1_S1_S1_dddddS1_S1_S1_ddPNS_8IndexSetEEUliE_EEvNS_21omp_parallel_for_execERKNS_9RangeISetET._omp_fn.42
RAJA::forall<CalcEnergyForElems(double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double, double, double, double, double, double*, double*, double*, double, double, RAJA::IndexSet*)::{lambda(int)#1}>
void RAJA::forall<CalcEnergyForElems(double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double, double, double, double, double, double*, double*, double*, double, double, RAJA::IndexSet*)::{lambda(int)#1}>(RAJA::omp_parallel_for_exec, RAJA::RangeISet const&, CalcEnergyForElems(double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double*, double, double, double, double, double, double*, double*, double*, double, double, RAJA::IndexSet*)::{lambda(int)#1}) [clone ._omp_fn.42]
Here, even the pretty name has lots of type info, and almost what we
want.  It's just missing '_omp_fn.42' to make it unique.
Is the difference here that one is a function and the other is a
template?  Do these demangle differently?
Maybe that's what is confusing me.

At 2:54  @mxz297 said:
First, let me check, what method does Symtab use to demangle names?
Is it the C++ __cxa_demangle() function?
Does Symtab make any adjustments to the answer it gets on its own?

Symtab calls __cxa_demangle to demangle names. Based on the code in
Dyninst, getTypedName() returns what __cxa_demangle returns.
getPrettyName() takes what __cxa_demangle returns and processes it
with a function "dedemangle". This "dedemangle" function is committed
by Bill in 2010.
The code comment at the beginning of this function explains what it
mechanically does (dyninst/common/src/string-regex.C)
/* Lifted from Jeffrey Odom.  Code reformatted so
I could figure out how to eliminate compiler warnings.
Adjusted to handle spaces inside templates intelligently.
We cut off everything after the first l-paren, so we don't
need to worry about the space after the parameters but
before the 'const'. */
The sentence "we cut off everything after the first l-paren" explains
why getPreetyName() will strip the "_omp_fn.42" suffix. The
implementation of this function suggests that it will try to keep c++
template types.
So, it looks like Bill treated template type information as part of
source function name.
I think we should first settle on what Symtab wants to produce with
mangled, pretty and typed names.  And I think Dyninst should decide
this based on what makes sense in general, not something specific to
hpctoolkit's needs.

Specifically, how much of the type annotations are supposed to be in
the pretty name?  I bring this up because I think we're getting
inconsistent answers from Symtab.

If you agree that template type information is part of the source
function name, then you are not getting inconsistent answers from
Symtab.
For example, in the first lulesh-RAJA-parallel.exe binary that I
looked at, there's one function, SumElemFaceNormal() at 0x40b450 with
mangled, pretty and typed names:

  _Z17SumElemFaceNormalPdS_S_S_S_S_S_S_S_S_S_S_dddddddddddd

  SumElemFaceNormal

  SumElemFaceNormal(double*, double*, double*, double*, double*,
double*, double*, double*, double*, double*, double*, double*, double,
double, double, double, double, double, double, double, double, double,
double, double)

Fine.  But in the same binary, CalcEnergyForElems() looks like:
_ZN4RAJA6forallIZ18CalcEnergyForElemsPdS1_S1_S1_S1_S1_S1_S1_S1_S1_S1_S1_S1_dddddS1_S1_S1_ddPNS_8IndexSetEEUliE_EEvNS_21omp_parallel_for_execERKNS_9RangeISetET_._omp_fn.42

  RAJA::forall<CalcEnergyForElems(double*, double*, double*, double*,
double*, double*, double*, double*, double*, double*, double*, double*,
double*, double, double, double, double, double, double*, double*,
double*, double, double, RAJA::IndexSet*)::{lambda(int)#1}>

  void RAJA::forall<CalcEnergyForElems(double*, double*, double*,
double*, double*, double*, double*, double*, double*, double*, double*,
double*, double*, double, double, double, double, double, double*,
double*, double*, double, double,
RAJA::IndexSet*)::{lambda(int)#1}>(RAJA::omp_parallel_for_exec,
RAJA::RangeISet const&, CalcEnergyForElems(double*, double*, double*,
double*, double*, double*, double*, double*, double*, double*, double*,
double*, double*, double, double, double, double, double, double*,
double*, double*, double, double, RAJA::IndexSet*)::{lambda(int)#1})
[clone ._omp_fn.42]

Here, even the pretty name has lots of type info, and almost what we
want.  It's just missing '_omp_fn.42' to make it unique.

Is the difference here that one is a function and the other is a
template?  Do these demangle differently?

I personally don't like the "getPrettyName()" interface, but I don't
have the intent to change it yet. It is not clear to me what's the
definition of pretty... Obviously, based on the code: pretty =
template types + source function name.
Thanks,
--Xiaozhu

Adding @martyitz to this thread (follow the link in the email to view the whole conversation on GitHub)
At 4:04pm @mwkrentel said:
Ok, I think that explains my confusion.
Thanks!
When I looked at this 3-4 years ago, I decided that pretty was better
than typed for our purposes.  We haven't really had a problem until
now, but I must have been looking at some skewed examples.
I'll do some more experiments, but I think I'm inclined to switch to
typed.  Full typed names are grossly long in some cases, but I think
they may be necessary.
As for pretty name, I agree that I'm not sure what it should mean, or
even if it should exist at all.  The take away is that pretty name is
an internal Symtab adjustment or smoothing, not an external standard
with a more clear right or wrong.  (So, maybe Dyninst apps might want
to do their own smoothing starting from the typed name.)
And let me check, you do test if the source language is C++ and only
demangle if it is, right?  That is, you don't just always demangle every
name.
Btw, I think you may want to switch from __cxa_demangle() to the
libiberty cplus_demangle().  The libiberty function has more options
for how to demangle, and I've seen examples where this matters
(eg, a function name versus a type encoding).
Also, I think I've seen a couple examples where Symtab demangles
something twice and thus produces a strange answer.
But that's just a heads up that I think I've seen some odd things.
I'll let you know when I have something more specific.

@laksono Is there something in the viewer that is matching up procs by
their pretty names?
If yes, then why is it using pretty names instead of link names?
I'm inclined to change our use to typed names.  But I'm not sure if
it's correct for the viewer to treat the pretty names as anything more
than a decoration.

Can you make it a switch, so we can see how the alternatives look?
PA had a viewer switch between long C++ names, short names (missing the argument types), and
mangled names, with or without the .so name appended.
Marty

On Apr 1, 2019, at 8:01 PM, Mark W. Krentel<EMAIL_ADDRESS>wrote:
@laksono https://github.com/laksono Is there something in the viewer that is matching up procs by
their pretty names?
If yes, then why is it using pretty names instead of link names?
I'm inclined to change our use to typed names. But I'm not sure if
it's correct for the viewer to treat the pretty names as anything more
than a decoration.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/HPCToolkit/hpctoolkit/issues/122#issuecomment-478825220, or mute the thread https://github.com/notifications/unsubscribe-auth/Ap7ojxMqFD6CPnjdnNIP4FJXwovj54eeks5vcsgYgaJpZM4cMWMw.

@laksono Is there something in the viewer that is matching up procs by
their pretty names?

No, there isn't.
hpcprof outputs only a single name of proc, and the viewer will get this name in the database.

If yes, then why is it using pretty names instead of link names?
I'm inclined to change our use to typed names. But I'm not sure if
it's correct for the viewer to treat the pretty names as anything more
than a decoration.

And let me check, you do test if the source language is C++ and only
demangle if it is, right? That is, you don't just always demangle every
name.

I am not 100% about my conclusion, but based on the Dyninst code I have looked at, it looks like SymtabAPI will demangle every name. Do you think this will be a 10.1 thing or not?

Btw, I think you may want to switch from __cxa_demangle() to the
libiberty cplus_demangle(). The libiberty function has more options
for how to demangle, and I've seen examples where this matters
(eg, a function name versus a type encoding).

I have little knowledge about the difference between these two demanglers. Do you think this has to be a 10.1 thing or can be later?

Also, I think I've seen a couple examples where Symtab demangles
something twice and thus produces a strange answer.
But that's just a heads up that I think I've seen some odd things.
I'll let you know when I have something more specific.

I think that a new demangle algorithm for RAJA codes is needed.  Clearly, the
long string of argument types is meaningless for the user.  Perhaps the functions
should be named by the source file and line where they are created.  It needs to
be something that conveys to the user where to look in his or her source.

The hyperlink from the function in hpcviewer's scope pane will take you to the source file and line number where the function is created. However, one can compile without line map information and not have source file and line information. Providing the detangled name is the best we can do in this case, even though it is not very meaningful.

On Apr 2, 2019, at 8:58 AM, John Mellor-Crummey<EMAIL_ADDRESS>wrote:
The hyperlink from the function in hpcviewer's scope pane will take you to the source file and line number where the function is created. However, one can compile without line map information and not have source file and line information. Providing the detangled name is the best we can do in this case, even though it is not very meaningful.

I assume “detangled” was an intentional pun.  :-)
How could the user figure out what the demangled name refers to?  I’d use the mangled name in this case, perhaps with a
[compile with -g, you dummy] tag appended.

I am not 100% about my conclusion, but based on the Dyninst code I
have looked at, it looks like SymtabAPI will demangle every name. Do
you think this will be a 10.1 thing or not?

Unfortunately, I think Symtab is demangling every name.  John's simple
openmp example above shows the problem.  If you compile it as a C
program (not C++), then Symtab reports the mangled name as 'a' and the
pretty and typed names as 'signed char'.
Fortunately, the fix is small and simple.  Symtab already computes the
source language per module.  If the source language is
lang_CPlusPlus or lang_GnuCPlusPlus, then demangle the name,
otherwise don't.
I'm not sure what lang_Fortran_with_pretty_debug means.  But there
are no options in libiberty demangle.h related to Fortran, so I'm
guessing 'pretty debug' doesn't mean demangle here.

I have little knowledge about the difference between these two
demanglers. Do you think this has to be a 10.1 thing or can be later?

This deserves more study.  The libiberty version has useful options
that __cxa_demangle doesn't have.  But I'd wait until I have a clear
example of something that __cxa_demangle just can't do correctly.
Plus, fix the previous problem first.

I've mostly finished my investigation on this.  I've reviewed all the
places where we get function names, how we get them, and if and how we
demangle them.  There are two things to do.

I'll change the way we get the mangled name from Symtab from the
"pretty" name to typed name.  That should make the demangled names
unique and solve Laksono's original problem.

We'll ask @mxz297 to fix Symtab so that it doesn't automatically
demangle every symbol.  Instead, it should lookup the module for the
symbol and demangle if and only if the source language is C++ or GNU
C++.  That should fix the 'a' --> 'signed char' problem.

I think all the infrastructure for that is in place, so hopefully this
is a fairly small fix and can go into 10.1.  I'll open a dyninst issue
for this.
I also looked at the difference between __cxa_demangle() and
libiberty cplus_demangle().  And now, I don't see any need to switch
to the libiberty version.
The main difference is that cplus_demangle("a") reports that "a" is
not a valid function name.  (It's a type encoding, not a function
mangling.)  But __cxa_demangle("a") always returns signed char,
which is technically correct, but in the wrong context.
So, the real problem is deciding to demangle "a" in the first place.
The libiberty version would hide that sin, but as long as you only
demangle symbol names from C++ source files, then I don't see any real
advantage in using libiberty.
Btw, my previous claim that hpcstruct doesn't demangle anything on its
own is not correct.  That's true for the primary case, where we have a
valid Symtab function, by far the most common case.
But there are two other cases: outlined openmp functions and plt
functions where there is no Symtab name so we construct one.  Second,
inline names.  In that case, we get a (file, line, proc) triple from
Symtab, but it's only one proc name, normally mangled, but actually
whatever is in the dwarf info.  In this case, we look up the source
file and demangle if it comes from C++.

This repository has moved to GitLab, please continue this issue there.


OBS update to obs-websocket version 5.4.2 preventing connection to obs-websocket-js version 5.0.5

Description:
OBS updated it's websocket settings, which is now preventing me from connecting to the websocket at all.  Release notes are:

Updated obs-websocket to version 5.4.2 [tt2468/Lain-B]
Release notes for addition in 5.4.0 are here: https://github.com/obsproject/obs-websocket/releases/tag/5.4.0

And release notes:

base: Update version to 5.4.0
@tt2468 tt2468 tagged this Jan 25
New Features:

Added GetSourceFilterKindList request
Added GetSceneItemSource request
Added InputSettingsChanged event
Added SourceFilterSettingsChanged event
Added UUID support to Sources (Inputs/Scenes), Inputs, Scenes,
Transitions

The Source requests/events use sourceUuid
The Input requests/events use inputUuid
The Scene requests/events use sceneUuid
The Transition requests/events use transitionUuid
Filters do not have support for UUIDs at this time.

Enhancements:

Added contextName field to TriggerHotkeyByName (exeldro)

Bug Fixes:

Fixed a crash on shutdown with notifications enabled (r1ch)
Added safety check to prevent null outputCongestion values in
GetStreamStatus
Fixed a memory leak when switching service kinds via
SetStreamServiceSettings

Other Notes:

Documented missing outputActive ToggleRecord response field
Added a few new client softwares to the README list
Removed a mis-documented imageData field
Added a note to the hotkey requests that they are as-is and we
will not provide support for them

Versions Used (if applicable):

obs-websocket-js version<EMAIL_ADDRESS>obs-websocket plugin version: Updated obs-websocket to version 5.4.2 [tt2468/Lain-B]
obs-studio version: 30.1.2 (64 bit)
node version: v16.14.0

Code sample:
My connection code, which connected before the OBS update and does not now (note it does not throw an error either):
async function connectToObs() {
  try {
    const { obsWebSocketVersion, negotiatedRpcVersion } = await obs.connect(
      'ws://localhost:4455',
      { rpcVersion: 1 }
    );

    console.log(`Connected to OBS WebSocket v${obsWebSocketVersion} using RPC version ${negotiatedRpcVersion}`);
    reconnectAttempts = 0;

    obs.on('ConnectionClosed', () => {
        console.log('OBS connection closed. Attempting to reconnect...');
        attemptObsReconnect();
    });

    obs.on('ConnectionError', (error) => {
        console.error('OBS connection error:', error);
        attemptObsReconnect();
    });

    process.on('SIGINT', async () => {
        console.log('Process terminated. Closing OBS WebSocket connection...');
        try {
            disconnectObs();
        } catch (error) {
            console.error('Error closing OBS WebSocket connection:', error);
        }
        process.exit(0); // Optional: Forcefully terminate process after disconnection
    });

    process.on('SIGTERM', async () => {
        console.log('Process terminated. Closing OBS WebSocket connection...');
        try {
            disconnectObs();
        } catch (error) {
            console.error('Error closing OBS WebSocket connection:', error);
        }
        process.exit(0); // Optional: Forcefully terminate process after disconnection
    });
  } catch (error) {
    console.error('Failed to connect:', error);
    attemptObsReconnect();
  }
}

Update:  It now appears to be working again... Maybe there was something out of sync after the OBS update, that has corrected itself now... but I'll monitor this for a few more streams and then close if it's stable.


Error while setup

Everything was running fine but since a few days it's not working anymore

Has anybody else, the same issue!?

It's working, thank you !


local_volume_provisioner_enabled replacement

Fix incomplete local_volume_provisioner_enabled replacement
Fix daemonset.yml.j2 syntax

ci check this

BTW I think feature gates like MountPropagation and VolumeScheduling are missing. Don't you think ?

thanks


cant login to administration url

is there someone who can help me ?
I install the complete panel . and it works great
but after i add a stream i can not login tot the administration url
Kind regards
Willem

Hi, if you mean the icecast ThRadio panel does not have ssl implemented for the transmission ports, in the panel it appears with https, it is a mistake I must correct it to enter just change the https to http.
I was going to launch an update but my laptop was damaged, I have to wait to have monetary resources to buy a new one.


bug(web,sdf,dal): remove change_set_written

Pour one out for our fallen homie, Change Set Written. It updated the UI when nothing else would, and kept us all safe from bad data. But it had long since served its purpose, and is no longer the loving friend it once was. Goodbye, old friend.

change set store no longer reacts to change set written
fixes store no longer reacts to change set written
qualifications store no longer reacts to change set written
qualifications now correctly update their status when merged to head, along with names in the outliner
fixes the components and qualficiations stores to not use change set written
action store no long uses changeSetWritten
secret store no longer uses change set written
asset store no longer responds to change set written
func store no longer uses change set write
remove change_set_written events that are now needless

Less spurious reloading, here we come.

bors check

bors try

bors merge


ignore duplicate OCLC value (OCoLC)eebo

There are 46,799 records in Voyager that were loaded from GT where the load process created the same 035$a value: "(OCoLC)eebo".
If a user attempts to access one of the bib records launchpad will trigger the error "DatabaseError: ORA-01795: maximum number of expressions in a list is 1000"
An example is bibid 6945932, the GT record is http://catalog.library.georgetown.edu/record=b3143765~S4
I posted a request to WRLC to delete the duplicate 035$a values (WRLC ticket 15998). If the WRLC cannot fix the data, a step will need to be added to launchpad to ignore those values when looking up related records by OCLC number.

WRLC removed GT eebo records from WRLC Voyager.

Now there are 57,000 GT records having the 035@a value "(OCoLC)mome". Awaiting WRLC response to my request to fix them...

WRLC fixed the data.


2019-07-12-BITpoint

@eric-oconnor Would this referencing system be better? [^example]
[^example]: This is an example footnote reference.

@eric-oconnor Would this referencing system be better? 1
Footnotes

This is an example footnote reference. ↩

Regarding the submissions this seems more stylistic than necessary for accuracy.  I am not the one to ask for that

/qualitycheck

Added quote marks, links corrected.

BITPoint re-opened trading services on after a successful safety assessment of the platform's wallet. ❌ False. The search results show that Bitpoint cryptocurrency exchange was hacked and lost $32 million in 2019. The exchange has not yet resumed services and is still monitoring the situation to complete a comprehensive review of all security measures. There is no evidence to suggest that they have re-opened trading services after a successful safety assessment.

links to proves:
https://www.cryptonewsz.com/bitpoint-japanese-crypto-exchange-re-opens-for-trade-services-after-28-million-hack-in-mid-of-july/
https://finance.yahoo.com/news/japanese-crypto-exchange-bitpoint-restarting-014200443.html

@LiquidVac please add the link directly to the part of the article that the bot got wrong.

/payout

Thanks, @LiquidVac! 2339 characters were added or changed in this PR and your contribution is worth $116.95. @albina-at-inca will process your payment.

@LiquidVac please note, all the payouts are done at the end of the month. Usually it is the 28th of the month. Please let me know if you have any questions.


core-app-api: refactor translation API into standalone implementation + split out language API

Hey, I just made a Pull Request!
More work on top of #17436 and #19858, another big one : /
Key pieces of this PR:

Split the AppTranslationApi into TranslationApi for translations, and AppLanguageApi for interacting with the selected language.
Remove react-i18next, it's preferable to not require React for translations but have the TranslationApi be standalone and usable from other API implementations too. In addition, react-i18next doesn't have a deduplicated context so we would need to solve that if we want to continue using it. I'm also not too comfortable exposing the full i18next API surface and would rather keep that hidden, more on that in the next point.
Reworked resource loading, the build in backend (async) loader module interface for i18next doesn't handle concurrent language switches and loading particularly gracefully and gets confused about whether resources are actually available or not. Instead we implement our own resource loading, which is actually a bit less code than if we were to implement it as a i18next module due to the extra checks required.
Default messages are now registered as resources instead rather than passed as the default message to t(...), this is required to support plurals and nested translations.
The TranslationApi uses a getter + observable pattern, and useTranslationRef has been refactored to consume that using suspense.

I also removed language i18next language detector, it had quite a lot of built-in behavior that I think we need to consider more carefully. For now the AppLanguageApi implementation is a simple default language + local storage setup.
CC @mario-mui
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Added a changeset for test-utils, but otherwise piggybacking on existing changesets


feat: move files to support new file structure

Moved self hosted files to support new documentation.
Used the following script to make these changes:
import os
import shutil

top_level_dirs = [
    "./v0.0.7",
    "./v0.0.8",
    "./v0.0.9",
    "./v0.0.10",
    "./v0.0.11",
    "./v0.0.12",
    "./v1.0.0",
    "./v1.1.0",
    "./v1.1.0-hotfix.1",
    "./v1.2.0",
    "./v1.2.1",
    "./v1.3.0",
]

latest_version = "v1.3.0"

def replace_file_contents(new_file, dir):
    # rewrite file contents to remove relative pathing
    with open(new_file, "r") as f:
        lines = f.readlines()
    with open(new_file, "w") as f:
        for line in lines:
            line = line.replace("/index.html", ".html")

            # replace all hrefs that are relative
            if "href=\"http" not in line and "href=\"" in line and "../" not in line:
                line = line.replace("href=\"", f"href=\"/self-hosted/{dir[2:]}/")

            # replace all srcs that are relative
            if "src=\"http" not in line and "src=\"" in line and "../" not in line:
                line = line.replace("src=\"", f"src=\"/self-hosted/{dir[2:]}/")

            # fix other assets text
            if "assets/" in line and f"{dir[2:]}/assets/" not in line and "../" not in line:
                line = line.replace("assets/", f"{dir[2:]}/assets/")

            line = line.replace("../", "", 1)

            if "../versions.json" in line:
                line = line.replace("../versions.json", "/self-hosted/versions.json")

            f.write(line)

def move_files(dir):
    print("Processing", dir)

    files_and_dirs = os.listdir(dir)
    dirs = []
    files = []
    for file_or_dir in files_and_dirs:
        if file_or_dir == ".git":
            continue
        if os.path.isdir(f"{dir}/{file_or_dir}"):
            dirs.append(file_or_dir)
        else:
            files.append(file_or_dir)

    for file in files:
        if file.endswith("index.html"):
            new_file = file.split("index.html")[0] + f"{dir}.html"
            print(f"Renaming {dir}/{file} to {new_file}")
            # copy to new destination
            shutil.copyfile(f"{dir}/{file}", new_file)

            replace_file_contents(new_file, dir)

            if new_file.endswith(f"{latest_version}.html"):
                shutil.copyfile(new_file, "latest.html")
        if file.endswith(".js"):
            replace_file_contents(f"{dir}/{file}", dir)

    for _dir in dirs:
        if _dir == "latest":
            #symlink
            continue
        move_files(f"{dir}/{_dir}")

def fix_site_maps():
    for site in top_level_dirs:
        with open(f"{site}/sitemap.xml", "r") as f:
            lines = f.readlines()
        with open(f"{site}/sitemap.xml", "w") as f:
            for line in lines:
                if "index.html" in line:
                    replaceable_line = line.replace("/index.html", ".html")
                    if replaceable_line.endswith(f"{latest_version}.html"):
                        f.write(replaceable_line.replace(f"{latest_version}.html", "latest.html"))
                    else:
                        f.write(replaceable_line)
                else:
                    f.write(line)
        # re-gzip
        os.system(f"gzip -c {site}/sitemap.xml > {site}/sitemap.xml.gz")

if __name__ == "__main__":
    move_files(".")
    fix_site_maps()

Thought of a better way to do this.


Disable SASL over SSL with a new config

Although it is highly recommended to use SASL over SSL, by enforcing SSL for SASL, ruby-kafka turns out to be unusable for those who want to not have SSL (eg: private networks) for SASL.
This PR adds a new configuration option sasl_over_ssl  while initializing the Kafka::Client. By default, it is set to true and the current behavior of the library will not be changed.
For those who want to disable the strict SSL mode for SASL, they can do so by providing sasl_over_ssl: false.

Changelog entry is missing.

I'm receiving the following error:
Traceback (most recent call last):
	23: from ruby-consumer.rb:13:in `<main>'
	22: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer.rb:204:in `each_message'
	21: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer.rb:392:in `consumer_loop'
	20: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/instrumenter.rb:35:in `instrument'
	19: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/instrumenter.rb:23:in `instrument'
	18: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer.rb:393:in `block in consumer_loop'
	17: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer.rb:205:in `block in each_message'
	16: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer.rb:503:in `fetch_batches'
	15: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer.rb:442:in `join_group'
	14: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer_group.rb:47:in `join'
	13: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer_group.rb:125:in `join_group'
	12: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/instrumenter.rb:35:in `instrument'
	11: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/instrumenter.rb:23:in `instrument'
	10: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer_group.rb:126:in `block in join_group'
	 9: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/consumer_group.rb:185:in `coordinator'
	 8: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/cluster.rb:116:in `get_group_coordinator'
	 7: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/cluster.rb:116:in `each'
	 6: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/cluster.rb:119:in `block in get_group_coordinator'
	 5: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/broker.rb:110:in `find_coordinator'
	 4: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/broker.rb:170:in `send_request'
	 3: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/broker.rb:184:in `connection'
	 2: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/connection_builder.rb:27:in `build_connection'
	 1: from /usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/sasl_authenticator.rb:51:in `authenticate!'
/usr/local/lib/ruby/gems/2.5.0/gems/ruby-kafka-0.7.0.beta1/lib/kafka/sasl/scram.rb:53:in `authenticate!': Invalid server signature (Kafka::FailedScramAuthentication)

The code I'm using is:
require "kafka"
kafka = Kafka.new(
   ["ip1", "ip2", "ip3"],
   sasl_scram_mechanism: 'sha256',
   sasl_scram_username: 'username,
   sasl_scram_password: 'password',
   sasl_over_ssl: false
   )
consumer = kafka.consumer(group_id: "my-group")

consumer.subscribe("my-topic")

consumer.each_message do |message|
   puts "#{message.topic}, #{message.partition}, #{message.offset}, #{message.key}, #{message.value}"
end

Any ideas as to what I'm doing wrong? I'm using the same code I was using with version 5.5 with the addition of the sasl_over_ssl: false. Thanks!

@novskey Did you resolved it? I have the same issue.

@fist28 Don’t quite remember, I can check when I’m back at work on Friday


cargo-generate not updated & unclear it needs updating

Microsoft Windows [Version 10.0.19045.3693]
git version 2.42.0.windows.2
cargo 1.72.1 (103a7ff2e 2023-08-15)
cargo-leptos 0.2.2
i am following the instructions written here:
https://leptos-rs.github.io/leptos/ssr/21_cargo_leptos.html
C:\Users\User Name\rust>cargo leptos new --git leptos-rs/start
 Project Name: leptos_actix_test
 Renaming project called `leptos_actix_test` to `leptos-actix-test`...
 Creating project called `leptos-actix-test`...
error: failed to find branch `master`
caused by: cannot locate local branch 'master'; class=Reference (4); code=NotFound (-3)

there is no 'master' branch, but a 'main' one:
C:\Users\User Name\rust>cargo leptos new --git leptos-rs/start -b main
 Project Name: leptos_actix_test
 Renaming project called `leptos_actix_test` to `leptos-actix-test`...
 Creating project called `leptos-actix-test`...
error: liquid: Cannot read file
cause: stream did not contain valid UTF-8
  with:
    path=C:\Users\User Name\rust\leptos-actix-test\assets\favicon.ico

at least something has been downloaded, but obviously that file isn't UTF-8.
same for the axum-template:
C:\Users\User Name\rust>cargo leptos new --git leptos-rs/start-axum
 Project Name: leptos_axum_test
 Renaming project called `leptos_axum_test` to `leptos-axum-test`...
 Creating project called `leptos-axum-test`...
error: failed to find branch `master`
caused by: cannot locate local branch 'master'; class=Reference (4); code=NotFound (-3)

C:\Users\User Name\rust>cargo leptos new --git leptos-rs/start-axum -b main
 Project Name: leptos_axum_test
 Renaming project called `leptos_axum_test` to `leptos-axum-test`...
 Creating project called `leptos-axum-test`...
error: liquid: Cannot read file
cause: stream did not contain valid UTF-8
  with:
    path=C:\Users\User Name\rust\leptos-axum-test\public\favicon.ico

i don't know if i can continue from here on or if something is missing now

I don't have access to Windows but can't reproduce on another platform.
This part of cargo-leptos just calls out to cargo-generate. If you run cargo generate https://github.com/leptos-rs/leptos do you get the same error? If so, what is your cargo-generate --version? It might be a bug in cargo-generate, or in how our template is set up.

right, updated cargo-generate from 0.2.1 to 0.18.5 ... it works now
that version was almost to-the-day five years old and i'm puzzled that it survived on my system that long o_O
i don't close the issue yet because there's no way i could have come up with this in the first place. i think:

either the installation of cargo-leptos hasn't defined a minimum version to update cargo-generate
or the leptos book is missing a note about updating this

btw on my ubuntu VPS there is no cargo-generate at all and it seems not to be required for cargo leptos new to work.

We've updated the deps on the starter template, I think we can close this for now. Feel free to comment if this is still an issue


Update MCM RBAC rules

How to categorize this PR?

/area open-source
/kind bug
What this PR does / why we need it:
Ref https://github.com/gardener/gardener-extension-provider-gcp/pull/271 and https://github.com/gardener/gardener-extension-provider-gcp/pull/304
Release note:

NONE

Depends on #150

This PR is ready now that #150 got merged.
/invite @deitch


Chef-711 Start Command

:nut_and_bolt: Description: What code changed, and why?

:chains: Related Resources
https://chefio.atlassian.net/browse/CHEF-711
:+1: Definition of Done
:athletic_shoe: How to Build and Test the Change
:white_check_mark: Checklist
All PRs must tick these:

[x] I have read the CONTRIBUTING document.
[x] All commits signed-off for the Developer Certification of Origin.

With occasional exceptions, all PRs from Progress employees must tick these:

[ ] Is the code clear? (complicated code or lots of comments--subdivide and use well-named methods, meaningful variable names, etc.)
[ ] Consistency checked? (user notifications, user prompts, visual patterns, code patterns, variable names)
[ ] Repeated code blocks eliminated? (adapt and reuse existing components, blocks, functions, etc.)
[ ] Spelling, grammar, typos checked? (at a minimum use make spell in any component directory)
[ ] Code well-formatted? (indents, line breaks, etc. improve rather than hinder readability)

All PRs from Progress employees should tick these if appropriate:

[x] Tests added/updated? (all new code needs new tests)
[ ] Docs added/updated? (all customer-facing changes)

Please add a note next to any checkbox above if you are NOT ticking it.
:camera: Screenshots, if applicable
https://user-images.githubusercontent.com/98326242/230901609-2e7bae2d-f0ac-4d6e-869b-bca6e9a39c35.mov

@SahithiMy fill the PR description template.


Framework for data driven tests.

This PR adds the initial frameworks required for data driven tests, and some examples for variants and references. These tests work as follows:

We have a number of modules within the tests/datadriven module. Each module has a top level function that is a generator over the test cases that are run. This uses nose's ability to use generators that yield individual test cases (see http://nose.readthedocs.org/en/latest/writing_tests.html). Each test that we run is a effectively a pair (testMethod, dataDir).
In each module, we have one or more TestClasses, and we allocate an instance of this class for each data directory. The test class builds a model of the data in this directory and also allocates the corresponding object from ga4gh.datamodel. We then compare these two models, and ensure that they are consistent.
Each TestClass has a set of test methods, and each of these is called exactly once. Note that this does violate the idea that unit tests should be entirely independent of each other. However, with a little programming discipline I think we can make these tests effectively independent. The cost of building the alternative model of the data in question is not trivial, and it seems pointless to mandate that this is done before every single test.

Here is an example of what a run looks like:
$ nosetests -vs tests/datadriven/
tests.datadriven.test_references.ReferenceSetTest.testNumReferences:GRCh38 ... ok
tests.datadriven.ReferenceSetTest.testProtocolElementValid:GRCh38 ... ok
tests.datadriven.test_references.ReferenceSetTest.testNumReferences:GRCh38_copy ... ok
tests.datadriven.ReferenceSetTest.testProtocolElementValid:GRCh38_copy ... ok
tests.datadriven.test_references.ReferenceSetTest.testNumReferences:example_1 ... ok
tests.datadriven.ReferenceSetTest.testProtocolElementValid:example_1 ... ok
tests.datadriven.VariantSetTest.testProtocolElementValid:example_1 ... ok
tests.datadriven.test_variants.VariantSetTest.testSearchAllVariants:example_1 ... ok
tests.datadriven.VariantSetTest.testProtocolElementValid:example_2 ... ok
tests.datadriven.test_variants.VariantSetTest.testSearchAllVariants:example_2 ... ok
tests.datadriven.VariantSetTest.testProtocolElementValid:example_3 ... ok
tests.datadriven.test_variants.VariantSetTest.testSearchAllVariants:example_3 ... ok
tests.datadriven.VariantSetTest.testProtocolElementValid:example_4 ... ok
tests.datadriven.test_variants.VariantSetTest.testSearchAllVariants:example_4 ... ok

----------------------------------------------------------------------
Ran 14 tests in 0.144s

Why not use standard unit tests? Unit tests are very valuable and we should continue to make lots of them. However, the principle thing that our server does is interact with data that it is provided with and serve this out to users. We will get data in all sorts of weird and wonderful formats with all sorts of quirks. We need to be able to deal with this, and to build in test cases to ensure we handle these quirks properly. When we find a data file that breaks our code, we add an example data file to our tests data, and the data driven tests automatically pick it up and run it.
The main disadvantage of this approach is that we can't extent unittest.TestCase, and so we lose our nice self.assertX methods. This is unfortunate, and is because of the way nose test generators work. We can possibly work around this with some nasty hackery, but I'm not sure if it's a good idea.
I'd hope to make this a fairly fundamental piece of infrastructure for how we move forward in terms of building out reads, variants and references support, so please review this PR and let me know what you think.

A few things:

I agree that data-driven tests are needed for the project, generally.
I don't like numeric naming of just about anything, both because it's non-descriptive and it implicitly adds a sequencing constraint.  I would favor example_whatever names rather than example_5 names.
So it looks like we're ok with adding binary files into the git repo now.  Does this mean we can add the latest ga4gh-example-data to the git repo?  That would be one way to unblock end-to-end testing.
1 test per data file does seem pointless, but I agree it sucks to throw out unittest
The ability to add in a new data file and have everything magically work is a nice feature.
Maybe a README in the root of tests/datadriven about how to run the tests would be good.
Will these tests get picked up in a global nosetests run?  Do we want them to be?

Good, glad you're happy with this. Some responses:

I don't like numeric naming of just about anything, both because it's non-descriptive and it implicitly adds a sequencing constraint. I would favor example_whatever names rather than example_5 names.

Agreed. I don't like this either, we should fix it for #173.

So it looks like we're ok with adding binary files into the git repo now. Does this mean we can add the latest ga4gh-example-data to the git repo? That would be one way to unblock end-to-end testing.

It's too big. We should keep our repo as small as we can.

1 test per data file does seem pointless, but I agree it sucks to throw out unittest

It's worse than that with unitest: we cannot multiply test-cases by data. We could work around the setup costs using class and module level fixtures if this was the only problem. unittest assumes that every test function is executed exactly once, and this just doesn't work with a data driven approach.

Maybe a README in the root of tests/datadriven about how to run the tests would be good.

Good idea.

Will these tests get picked up in a global nosetests run? Do we want them to be?

Yes, and yes. If they get expensive we can run them less frequently, but right now it looks like they'll be plenty fast enough to run every time.

Yes, data driven tests as essential for this project.
I often do the intellectual cop-out of numbering test examples
and then regret it.  I do try to have example files that drive
multiple tests.  I do this with unittest all the time, just
cheat on the setup/tear down (cache results).
I am a fan of only storing the ASCII version of data (e.g. SAM
instead of BAM) and converting on the fly when running the
tests.  This does put a burden of having more tools installed to
run the test.  A fall bac option is to check both the ASCII and
binary versions into the tree.
Danny Colligan<EMAIL_ADDRESS>writes:

A few things:

I agree that data-driven tests are needed for the project, generally.
I don't like numeric naming of just about anything, both because it's
non-descriptive and it implicitly adds a sequencing constraint. I would favor
example_whatever names rather than example_5 names.
So it looks like we're ok with adding binary files into the git repo now.
Does this mean we can add the latest ga4gh-example-data to the git repo? That
would be one way to unblock end-to-end testing.
1 test per data file does seem pointless, but I agree it sucks to throw out
unittest
The ability to add in a new data file and have everything magically work is
a nice feature.
Maybe a README in the root of tests/datadriven about how to run the tests
would be good.
Will these tests get picked up in a global nosetests run? Do we want them to
be?

—
Reply to this email directly or view it on GitHub.*

Can you give more details on how you do this with unittest @diekhans? This would be good to know - maybe I'm over-engineering things here and there's a simpler solution.
Agreed on only storing the ASCII versions. We are storing the tabix indexes at the moment, but maybe we should start doing this on the fly. Pysam will do this for us quite nicely.

@diekhans I'm thinking a little bit ahead to how we may be storing graph data for testing v0.6.x of the API (which I believe is essential for the development effort). A SQLite based dataset could always be represented by its creation script (CREATE TABLEs followed by INSERTs), and an AVRO based one can be expressed in JSON, so I think we'd be able to adhere to an ASCII-only policy if that's what ends up being mandated.

I would argue for a more human-editable format than the INSERTS
and then a program to load the SQLite database.
Maciek Smuga-Otto<EMAIL_ADDRESS>writes:

@diekhans I'm thinking a little bit ahead to how we may be storing graph data
for testing v0.6.x of the API (which I believe is essential for the development
effort). A SQLite based dataset could always be represented by its creation
script (CREATE TABLEs followed by INSERTs), and an AVRO based one can be
expressed in JSON, so I think we'd be able to adhere to an ASCII-only policy if
that's what ends up being mandated.
—
Reply to this email directly or view it on GitHub.*

@diekhans, basically if I understand correctly, you are looking for complete transparency in what happens to the data as it gets processed at each step and then compare to what it should be - while leaving optimizations to the data-format for a later time, after these steps are successful.

My main point is that keeping the master source for the test
data in git in the ASCII form make it easier to maintain and
understand.  Also, as you point out, if the binary format
changes, then it just requires changing the script that builds
the binary file.
Paul Grosu<EMAIL_ADDRESS>writes:

@diekhans, basically if I understand correctly, you are looking for complete
transparency in what happens to the data as it gets processed at each step and
then compare to what it should be - while leaving optimizations to the
data-format for a later time, after these steps are successful.
—
Reply to this email directly or view it on GitHub.*

+1 @diekhans, Thank you for the clarification and I am in perfect agreement.

I don't think we should strictly mandate text-only data (we might, for example, want to put in an old BAM file to make sure we read it correctly), but it is definitely the preferable option. I agree with @macieksmuga too --- we'll certainly want to get graph data in here.

I've had another quick look at trying to hack support for unittest.TestCase in with this framework, and it looks very nasty. The idea would be to redefine the DataDrivenTest class at run time so that it inherits from unittest.TestCase: we'd then have all the nice assertX methods. (We have to do this at run time, because any compile-time subclasses of unittest.TestCase are collected by the unittest test discovery methods, and this breaks the apply-each-test-to-each-data-file pattern.) This is possible, I think, but not worth it in my opinion. It would lead to very mysterious looking code, obscure errors, and would probably be highly Python 2 specific.
Any thoughts on this: are we OK with using plain asserts for data driven tests?

Dumb question: couldn't the data driven tests be the same thing
as the conformance suite?

It's not quite the same thing in my view @diekhans, but it does depend on how you define conformance suite. I would think that a conformance suite must be something that we can run against an arbitrary server, and operates only through the protocol. These data driven tests are highly specific to our internal structures. What have you in mind to a conformance suite?

  I didn't get the
running at a lower level than the API.
Still, if conformance test cases could be used for other
these purposes, it might save development times.
BTW, I would say a conformance suites can run against an
arbitrary server that has the conformance suite data set loaded.
Cheers
Mark
Jerome Kelleher<EMAIL_ADDRESS>writes:

It's not quite the same thing in my view @diekhans, but it does depend on how
you define conformance suite. I would think that a conformance suite must be
something that we can run against an arbitrary server, and operates only
through the protocol. These data driven tests are highly specific to our
internal structures. What have you in mind to a conformance suite?
—
Reply to this email directly or view it on GitHub.*

BTW, I would say a conformance suites can run against an
arbitrary server that has the conformance suite data set loaded.
Agreed!

+1

I think enough time has elapsed - can you merge this please @dcolligan?


[hotfix] handle surprising file metadata states

Purpose
don't break when gathering metadata about a file

Changes

handle FileVersion with null creator
do not index osfstorage file without versions

QA Notes
Please make verification statements inspired by your code and what your code touches.

Verify
Verify

What are the areas of risk?
Any concerns/considerations/questions that development raised?
Documentation

Side Effects

Ticket

Cherry-picked in as efd0f1c02129ecf2e9ebb32e81ed011433b9d93c, consider this merged :octocat:


Crafting for spike rack, signal switches and buttons.

About the PR

Adds crafting for kitchen / meat spike.
Also adds crafting for signal buttons and signal switches.
Moves all signal / control crafting to Machines category (Approved by Mirrorcult). This is because they were all in structures and it's cluttered to heck. In code they were also split between lighting and conveyors (very SS13 of us).

Screenshots

Changelog

:cl:

add: Added crafting for meat spikes (In furniture) as well as signal buttons and signal switches (All in Machines section).

Build FAILED.
    0 Warning(s)
    0 Error(s)

Sounds about right. Will look at this when I can be chewed.

Build FAILED.
    0 Warning(s)
    0 Error(s)

Sounds about right. Will look at this when I can be chewed.

Looks like 2 of the construction graphs need fixing from the test.


How to install a distributed deployment?

Hello,
I'm relatively new to Splunk and even more on using this Ansible role so could you please provide some guidelines or examples on how to implement a distributed deployment?
My inventory looks like this:
[search]
splunk1

[indexer]
splunk2

[deploymentserver]
splunk3

[licensemaster]
splunk4

[dmc]
splunk4

[full:children]
search
indexer
deploymentserver
licensemaster
dmc

I have executed the splunk_install_or_upgrade.yml playbook and it just installed the package so I guess I'm missing the configuration part.
Thanks in advance for your help.

@tandrez splunk_install_or_upgrade.yml is specifically to either install splunk or upgrade if it's already install but with a different version. I would take a look at the splunk_idxc_deploy.yml and splunk_shc_deploy.yml playbooks that install and configure splunk. You can leverage those playbooks to create your own. You can also combine both in one playbook if necessary.

Thank you for your reply @Ces-Ces.
Yes, I ended up using the playbook splunk_idxc_deploy.yml even though initialy, I just wanted to implement a non-clustered deployment. My question is now: is the playbook configure_apps.yml just meant to deploy apps or also to configure everything (for example the files under $SPLUNK_HOME/etc/system/local).

Thanks again for your help @Ces-Ces. I think I'm starting to figure out how it works so I'm gonna close this issue.


"An existing connection was forcibly closed by the remote host" after restarting Redis

I'm getting an exception after restarting Redis:
[SocketException (0x2746): An existing connection was forcibly closed by the remote host]
System.Net.Sockets.Socket.Send(IList`1 buffers, SocketFlags socketFlags) +3094587
ServiceStack.Redis.RedisNativeClient.FlushSendBuffer() +118
ServiceStack.Redis.RedisTransaction.Commit() +553
I'm using PooledRedisClientManager, using BasicRedisClientManager helps a little (got this exception on the first request only, subsequent were fine).
I believe that this provider should be more robust and handle exceptions like this by simply returning nothing from the cache so NHibernate can still operate and execute regular query.
Any thoughts ?

@kowalgta I'm going to close this due to lack of activity. Also, I just pushed the 2.0 release which now uses StackExchange.Redis as the Redis provider. Please give it a shot and let me know if you have any problems.


Help with widescreen

Can you help me with widescreen, like adding a file, or cheat code or something?

Add EnableWidescreen=TRUE to your config.ini (located in Hi-Octane\config.ini). Make sure you have the latest version installed

My game don't run after this my friend

Add EnableWidescreen=TRUE to your config.ini (located in Hi-Octane\config.ini) under [Boot Config]. Make sure you have the latest version installed

My game doesn't run after this my friend