add dark mode

This commit adds a dark mode to doc-gen4. It takes much inspiration from leanprover-community/doc-gen#160.
Unlike the dark theme in doc-gen, this will work with or without javascript. If javascript is disabled, the color picker will not work. However, the correct theme will be detected from the browser based on css media queries.
If javascript is enabled, the color picker will work and color picker preferences will be remembered.
note: a dark-mode is not added to inked code. This PR only applies to generated documentation.
closes #117

hmm, there seems to be an issue switching from a defined colorscheme (e.g. "dark theme" -> "system")

ok i think i took care of all of the edge cases.

Thank's a lot, it seems to work for me too. Unfortunately the matlhib doc builds right now are broken due to  some weird meta programming issue so this won't be deployed too son.


Instruction: How to add Anthropic's Claude models

Hi everyone. As you know some latest Anthropic's Claude models is better and chaeper than OpenaAI models. So I create a new generator file to use Anthropic's models. and made changes to manager.py file in goldenverba/components/generation
generation.zip
Install Anthropic library:
pip install anthropic
and you should add this variables to your environment (or .env file):
ANTHROPIC_API_KEY=sk-ant-api03-.....pAAA

ANTHROPIC_MODEL=claude-3-haiku-20240307

You will need API key from Anthropic at https://console.anthropic.com/
Available models:

Claude 3 Opus
claude-3-opus-20240229

Claude 3 Sonnet
claude-3-sonnet-20240229

Claude 3 Haiku
claude-3-haiku-20240307

P.S.
The generator's code made with help of Claude and GPT-3.5/4

Great work! Thanks 🚀 We'll implement it in the coming next update

I'd forgotten...
in "verba_manager.py" add:
        # Check Anthropic ENV KEY
        try:
            import anthropic

            anthropic_key = os.environ.get("ANTHROPIC_API_KEY", "")

            if anthropic_key != "":
                self.environment_variables["ANTHROPIC_API_KEY"] = True
                anthropic.api_key = anthropic_key

        except Exception:
            self.environment_variables["ANTHROPIC_API_KEY"] = False

and
        try:
            import anthropic

            self.installed_libraries["anthropic"] = True
        except Exception:
            self.installed_libraries["anthropic"] = False

it's enough to add the modification and restart Verba

ok, strange. Nothing changed in my case after the modification...

It depends on how did you installed Verba. If you installed from sources you should change in the souce folder, if you installed via pip you should make changes in the python lib folder where Verba was installed

Indeed I installed from the source. What do you mean by "change in the source folder" ? Sorry for this novice question... I put the manager.py into goldenverba/components/generation and modified the .py files. Anything else to change?

Try to rebuild. Change in sources, rebuild

We added anthropic to the newest release


curv protocol

when claiming soul via zksync network fees estimate is always high, unable to estimate the fees

Always Edit the gas fee in your wallet before confirming the transaction. Choose the lowest instead of site suggested.


Tests should work for node.js versions(LTS, and above)

We should take care to maintain compatibility. Currently, my local tests fail for the current version of LTS Node.js(HYDROGEN) and I think that's bad :(
cc @mcollina @cjihrig!👍

this is a known problem due to some breaking changes in Node v20, which will be addressed before Node v20 goes LTS.


Using light.turn_on with area_id turns on ALL lights.

Home Assistant release with the issue:
0.99.3
Last working Home Assistant release (if known):
Don't know
Operating environment (Hass.io/Docker/Windows/etc.):
Hass.io running in docker on Ubuntu 18.04.1
Integration:
https://developers.home-assistant.io/docs/en/area_registry_index.html
Description of problem:
When using light.turn_on in an automation with area_id as data input, the automation turns on ALL lights in the system.
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

Traceback (if applicable):

Additional information:
None.

For what it's worth, I've been seeing this since 0.97 (venv install). The switch domain behaves as expected.


Match the menu item on left with the title of the page.

Match the menu item on left with the title of the page. menu says event hub connection, page says event grid connection.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 35bd3e7f-615b-aa1a-5f86-dfe4a209b09c
Version Independent ID: 4e8e4a9a-02a5-9d6e-0f94-cb0146cd6899
Content: Event Grid data connection for Azure Synapse Data Explorer (Preview) - Azure Synapse Analytics
Content Source: articles/synapse-analytics/data-explorer/ingest-data/data-explorer-ingest-event-grid-overview.md
Service: synapse-analytics
Sub-service: data-explorer
GitHub Login: @shsagir
Microsoft Alias: shsagir

@ahmedgillanione
Thanks for your feedback! We will investigate and update as appropriate.

@ahmedgillanione
Thank you for bringing this to our attention. I've assigned this issue to the author who will investigate and update as appropriate.
@shsagir
Could you Please review this and update as appropriate.

@ahmedgillanione Thanks for your feedback. We've submitted a documentation change based on your suggestion, and the updated article should be live by tomorrow. Please let us know if there's anything more we can do for you. To follow up on this issue, leave us a comment with an @mention.
#please-close


Svelte transition animations

Removed CSS animations, replaced with Svelte/transition
Fade in on creation, slide out to the left on removal.

Could you rebuild it by using https://animate.style  or something a like?
With a css-only animations library its easier to pick what animation someone wants to have instead of choosing a svelte animation and then having to rebuild this todolist again.
Sorry that you already put work on this svelte animation :(

We might be able to, it sounds a bit tricky. I only just started learning Svelte today 🤔
Maybe we can keep this for today and use it as a stepping stone forward to eventually having the animate.style library available for both in and out transitions.

I think using the css class is a bit harder than I expected, for the "out animations" those classes would need to be applied and waited until the animation is done and only then the entry could be removed  so I'll create a new issue for it.

LGTM 👍
I only had an issue, if you had 2 items, and you deleted the 1st one , it got removed at top and then the animation kicked in.
But I dont think this is something of your fault.
Thanks for the contribution, again! :)


Add URDFtoMuJoCoLoader and MujocoWrapper class

Summary of PR #7 Generated by beloved Copilot assistant
Files Changed

examples/generate_ergoCub_xml.py (Added)

Demonstrates usage of URDFtoMuJoCoLoader and MujocoWrapper classes.
Example includes loading a URDF file, converting to MuJoCo XML, and initializing MuJoCo models.

mujoco_urdf_loader/init.py (Modified)

Imports URDFtoMuJoCoLoader, URDFtoMuJoCoLoaderCfg, ControlMode, and MujocoWrapper.

mujoco_urdf_loader/loader.py (Added)

Introduces URDFtoMuJoCoLoader class.
Handles loading and simplifying URDF, converting to MuJoCo XML, and setting control modes.

Summary of New Classes

URDFtoMuJoCoLoader

Purpose: Converts URDF files to MuJoCo XML format.
Key Features:

Simplifies URDF using iDynTree.
Converts URDF to MuJoCo XML with specified control modes.
Adds actuators based on control modes (position or torque).

MujocoWrapper

Purpose: Wrapper for MuJoCo models.
Key Features:

Initializes MuJoCo model and data from XML string.

For further details, you can view the full list of changes here.

Cool! Could it make sense to update the README to advertise this class or at least link the example?

Can you document in the PR add_position_actuator used to add actuators named <jointname>_motor and now is adding actuators named <jointname>_position_motor ? Thanks!

Cool! Could it make sense to update the README to advertise this class or at least link the example?

Yup! I'll add some pointer there!

Also some tests would be nice :)

I agree! For now I can put one that checks that the conversion does not stop.

The existing heuristics as mentioned by @giotherobot seems to be working fine, but indeed it could could fail if some joints have some peculiar naming choice. URDFtoMuJoCoLoader knows the "real" mapping from joint to actuator, perhaps it can just pass it (optionally) to the MujocoWrapper, and if that is not present the  MujocoWrapper fails back on the heuristics?

The existing heuristics as mentioned by @giotherobot seems to be working fine, but indeed it could could fail if some joints have some peculiar naming choice. URDFtoMuJoCoLoader knows the "real" mapping from joint to actuator, perhaps it can just pass it (optionally) to the MujocoWrapper, and if that is not present the MujocoWrapper fails back on the heuristics?

Got it! Or another option could be to force who's using the URDFtoMuJoCoLoader to use as name of the actuators the one in the list of the controlled joints. A lazy solution to be honest.
Then I could set revert back the default name to <jointname>_motor in add_position_actuator, and just use the following lines to pass the name of the actuator (compliant with the list of actuated joints passed before)
https://github.com/ami-iit/mujoco-urdf-loader/blob/51d484d513224a6f149ff76c55dccddac6922bba/mujoco_urdf_loader/loader.py#L144-L160

Ciao guys! Do you think there's something else to do for this PR?

Ok for me!


Not working (06/03/2021)

until yesterday it was working perfectly. But today I was unable to connect. I redid all the steps, but it still doesn't work.

Unable to connect to github? You might have to set up Hyper-V networking again. It's also possible that your router for some reason isn't allocating additional IPs, if your router is provided from your internet service provider, in which case you might need an another router.

Ohh. working again! TY!


Simplify Expose decorator

Currently to expose a property under a different name we have to use
@Expose({ name: 'id' })

Can we make it so we can simply pass a string? e.g. @Expose('id')

The exposed needs to receive which type of option, there are a few different options.
If you can think of a way to do this, please help.
The code:
https://github.com/tokilabs/lang/blob/c350927ee3d2c42dce6f9a1c768618d6324936a7/src/serialization/decorators.ts#L67-L71
The interface:
https://github.com/tokilabs/lang/blob/c350927ee3d2c42dce6f9a1c768618d6324936a7/src/serialization/interfaces.ts#L78-L85

While I don't recall the use case I suggested this alternative method, I do remember the number of instances that were used with only the name property were much greater than the instances that had multiple properties set.
So the proposed implementation would be to accept as an options type an IExposeOptions object or a string.  If the object were passed the implementation would be the same, if a string was passed, however, we could assume that the string's value referred to the name property

The issue was resolved in the latest version of the package. I will close the issue.


Issue#1554

Please Refer : https://openshift.io/openshiftio/openshiftio/plan/detail/1554
Fix : https://drive.google.com/file/d/0B9v_DWsBK5HoY2NtcVI5eGdyNDA/view

@divyanshiGupta fabric8/fabric8-ui:SNAPSHOT-PR-2299-2 is deployed and available for testing at https://fabric8-ui-fabric8-ui-pr-2299-fabric8-planner.badger.fabric8.io

Looks good @divyanshiGupta. Approved!


feature: Location(err)string

really nice project!
I only miss one thing while using logrus
	log.WithFields(logrus.Fields{
		"code": "HandlerCheckBla",
		"user_id": user_id,
		"last_check": last_check,
                 "err_location":  errors.Location(err),   // <-------
	}).Error(err)

could you please add something like ...
func Location( err error) string  {
	str := ""
	for err != nil {
		location, ok := err.(locationer)
		if ok {
			file, line := location.Location()
			str += fmt.Sprintf("%s:%d: \n", file, line)
		}
		cause, ok := err.(causer)
		if !ok {
			return str
			break
		}
		err = cause.Cause()
	}
	return ""
 #}

to errors.go ... ? 👍

Thank you for raising this issue. I'm hesitant to add this helper for a few
reasons.
The first is I'm not sure how often it'll be used, itll certainly not be as
common as Wrap and Cause.
The second is the sample code you wrote doesn't do what I think you think
it does, your sample will print the location of the topmost wrapped error,
not the underlying cause. Printing the top of the error stack doesn't seem
as useful as printing the bottom.
Also, if you wrote something like Location(Cause(err)) that would return a
blank line if the underlying cause of the error did not have a location,
which I think is quite common.
I think he location interface is probably most useful for printing the
whole error stack, not for ad-hoc investigation.
Thanks
Dave
On Tue, 26 Apr 2016, 05:44 randy<EMAIL_ADDRESS>wrote:

really nice project!
I only miss one thing while using logrus
log.WithFields(logrus.Fields{
    "code": "HandlerCheckBla",
    "user_id": user_id,
    "last_check": last_check,
             "err_location":  errors.Location(err),   // <-------
}).Error(err)

could you please add ...
func Location(err error) string {
if err != nil {
location, ok := err.(locationer)
if ok {
file, line := location.Location()
return fmt.Sprintf("%s:%d", file, line)
}
}
return ""
}
to errors.go ... ? 👍
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/pkg/errors/issues/10

I understand. I've just noticed that the returned line number was wrong plus faulty-reasoning^2 (14 hours, time for sleep)
Still kinda need the file and line numbers on my log-server
So I'll just go with the modified Fprint to pass it to logrus.
Thanks Dave
func Stack( err error) string {
	str := ""
	for err != nil {
		location, ok := err.(locationer)
		if ok {
			file, line := location.Location()
			str += fmt.Sprintf("%s:%d: ", file, line)
		}
		switch err := err.(type) {
		case *e:
			str += fmt.Sprintln(err.message)
		default:
			str += fmt.Sprintln(err.Error())
		}

		cause, ok := err.(causer)
		if !ok {
			return str
		}
		err = cause.Cause()
	}
	return ""
}

Don't forget errors.Fprint which will print a formatted stack of errors to an io.Writer


nvc++: undefined reference to std::filesystem::__cxx11::path::_M_split_cmpts()

https://gitlab.com/-/snippets/2265018

solved


Restore Luv - Unix.file_descr conversions, this time in a separate helper package

Needed for the Lwt libuv port (ocsigen/lwt#813).

The commit that originally removed them: https://github.com/aantron/luv/commit/705ec144a9f047260de8140583e88d016e33e061

The commit that originally removed them: https://github.com/aantron/luv/commit/705ec144a9f047260de8140583e88d016e33e061


VBAN Emitter not starting from Web UI

Can't get vban emitter to start from web page. Works fine if I run from terminal manually. Doesn't give log output.
The command that I run in the terminal is taken from args_2.txt file with 'vban_' in front. Ex: vban_emitter -i <IP_ADDRESS> -p 6980 -s UBUNTU -b alsa -l 1

Somehow got it to work. Tried uninstalling & reinstalling everything. But think it was setting pulseaudio to run as a system wide service with autospawn set to no. Outlined on the bugs page: https://github.com/VBAN-manager/VBAN-manager/wiki/Known-bugs
Thought I tried that earlier, but think I didn't set autospawn to no/false. So maybe that is important for it to work?
After that I was able to start/stop from the web management ui, but still had issues with the service running at startup. Appears the network wasn't fully up when the service was trying to start. Resolved by adding 'sleep 15' to vban.sh under start-service. I'm sure there's a better way, but it works.


Fix typo in ff_util and add missing catkin_pkg line in ff_msgs

I am working with  the Astrobee Gazebo simulation and some of our in-house ROS packages. I ran across a couple of issues when trying to overlay the astrobee workspace with our Lab workspace, specifically a typo in ff_util and the lack of a catkin line in ff_msgs. Oddly enough, these errors only showed up when I tried to overlay workspaces.
This PR fixes the issues I found. After using these changes, I was able to build an overlaid workspace and use some of the astrobee's packages from said ws.

PS.- I also found that while trying to compile the Astrobee workspace, I was getting several errors regarding OpenCV libraries not being found (despite find_package indeed finding them). I think the error was that OpenCV_LIBRARIES was storing only the libraries names instead of their full paths.
I was able to overcome this issue by commenting out these lines in astrobee/CMakeLists.txt:
#if (${OpenCV_VERSION} MATCHES "3.3.1")
#  foreach(__cvcomponent ${OpenCV_LIB_COMPONENTS})
#    set (__original_cvcomponent ${__cvcomponent})
#    if(NOT __cvcomponent MATCHES "^opencv_")
#      set(__cvcomponent opencv_${__cvcomponent})
#    endif()
#    if (TARGET ${__cvcomponent})
#      set_target_properties(${__cvcomponent} PROPERTIES
#          MAP_IMPORTED_CONFIG_DEBUG ""
#          MAP_IMPORTED_CONFIG_RELEASE ""
#          MAP_IMPORTED_CONFIG_RELWITHDEBINFO ""
#          MAP_IMPORTED_CONFIG_MINSIZEREL ""
#      )
#    endif()
#  endforeach(__cvcomponent)
#endif()
#set(OpenCV_LIBRARIES ${OpenCV_LIBS})

I didn't push these changes in this PR as I don't know if this is reproducible or just affected my specific setup. If more people in my Lab find the same issue, I'll send another PR for this as well.

Thank you so much for the contribution! It's very useful for trying to overlay the workspaces :)
I have reviewed the pull request and we will include it in the next public release.
As far as the OpenCV libraries not being found, I can't reproduce the error (but I don't have the 3.3.1 version). Please let us know if someone else has that problem.

Thanks Ana! This will be pushed to github as part of the next release. Also in case you haven't, could you please sign and submit this? Thank you! https://www.nasa.gov/sites/default/files/atoms/files/astrobee_individual_contributor_license_agreement.pdf

You are very welcome, Brian! Just signed and sent the form to Kim Chrestenson (as instructed at the bottom of the form), if you need anything else, let me know. Thanks!


[Improve] Speed up ck file chown (#6090)

Purpose of this pull request

improve the speed of transfer data from hive to ck
Does this PR introduce any user-facing change?
no
How was this patch tested?
300KW records from hive to ck，cost minutes reduced from 100+  to 60
Check list

[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

[ ] Update the release-note.

cc @Hisoka-X

You need to explain the reason for the change and add test cases


Nginx default error

fixes https://github.com/ministryofjustice/cloud-platform/issues/1046

Ouch. This clobbers any custom 400, 403 and 404 pages that an app might return. (502 and 504 almost by definition won't be returned by an app)
There needs to be some check to determine whether the app did send content along with the status code.
And if that's impossible to determine, 400, 403 and 404 pages should be not be attempted.


Overflow

I've noticed that Complex and now PONS both suffer from lack of 32-bit support.
/// .hashValue -- conforms to Hashable
    public var hashValue:Int {
        let bits = sizeof(Int) * 4
        let mask = bits == 16 ? 0xffff : 0xffffFFFF

0xffffFFFF will not compile in Release mode unless you remove 32-bit targets leaving only arm64.

POComplex#hashValue as of PONS 0.5.0 is defined as:
    /// hash value of `self`
    public var hashValue:Int {
        let bits = sizeof(Int) * 4
        return ((re.hashValue >> bits) << bits) | (im.hashValue >> bits)
    }

Which should not suffer from 0xffffFFFF.  And so far as I see Framework-iOS builds fine for arm64, arm7 and arm7s.

Oh! I missed that change. I had been meaning to mention this for a while, but good that you fixed it.


FR: Please allow for changing/setting the config dir via env var (plus compat with azure-xplat-cli)

This snippet (from src/azure-cli-core/azure/cli/core/_config.py)
GLOBAL_CONFIG_DIR = os.path.expanduser(os.path.join('~', '.azure'))
GLOBAL_CONFIG_PATH = os.path.join(GLOBAL_CONFIG_DIR, 'config')

Please allow for something like:
if os.getenv("AZURE_CONFIG_DIR"):
    GLOBAL_CONFIG_DIR = os.getenv("AZURE_CONFIG_DIR");
else:
    GLOBAL_CONFIG_DIR = os.path.expanduser(os.path.join('~', '.azure'))
GLOBAL_CONFIG_PATH = os.path.join(GLOBAL_CONFIG_DIR, 'config')

This would allow users to have separate logins/contexts that could be switched between easily by setting on cli. For example, I use this with azure-xplat-cli:
alias azure-foo='env AZURE_CONFIG_DIR=/users/nneul/.azure-foo azure'
alias azure-bar='env AZURE_CONFIG_DIR=/users/nneul/.azure-bar azure'

Which let's me pick between different profiles simply by using the aliases. As currently implemented, with the python client, there isn't any clean way to be logged in on multiple accounts simultaneously from the same physical login/desktop environment.

 It's a pretty simplistic patch, though I'd highly recommend factoring out into a 'getConfigDir()' function at some point and using that everywhere this PR touches instead.

PR was merged. Please also see #1347 for related future plans.


New Device: Smart Switch

Describe your device
Simple "Smart Switch"
Partial Support?
Working on TuyaApp only
Device Schema
{
"type": "SimpleLight",
"name": "Outdoor Light",
"id": "7181270598f4abdd2125",
"key": "************",
"manufacturer": "Smart Switch",
"model": "HYS-08-004-433",
"ip": "<IP_ADDRESS>",
"port": "6668"
},
Additional context
[9/4/2021, 9:37:51 PM] [TuyaLan] Failed to discover Outdoor Light<PHONE_NUMBER>f4abdd2125) in time but will connect via <IP_ADDRESS>.
[Tuya DEBUG] reconnect called for Outdoor Light
[9/4/2021, 9:37:52 PM] [TuyaLan] Connected to Outdoor Light
[Tuya] Sending first query to Outdoor Light (3.1)
[Tuya DEBUG] decrementing this._connectionAttempts, currently 2
[Tuya DEBUG] decrementing this._connectionAttempts, currently 1
[Tuya] Socket had a problem and will reconnect to Outdoor Light (ECONNRESET)
[Tuya DEBUG] Reconnecting with connection attempts =  0
[Tuya DEBUG] reconnect called for Outdoor Light
[Tuya] Sending first query to Outdoor Light (3.1)
[Tuya DEBUG] decrementing this._connectionAttempts, currently 1
[Tuya] Socket had a problem and will reconnect to Outdoor Light (ECONNRESET)
[Tuya DEBUG] Reconnecting with connection attempts =  0
[Tuya DEBUG] reconnect called for Outdoor Light
[Tuya] Sending first query to Outdoor Light (3.1)
[Tuya DEBUG] decrementing this._connectionAttempts, currently 1
[Tuya] Socket had a problem and will reconnect to Outdoor Light (ECONNRESET)
[Tuya DEBUG] Reconnecting with connection attempts =  0
[Tuya DEBUG] reconnect called for Outdoor Light
[Tuya] Sending first query to Outdoor Light (3.1)
[Tuya DEBUG] decrementing this._connectionAttempts, currently 1
[Tuya] Socket had a problem and will reconnect to Outdoor Light (ECONNRESET)
[Tuya DEBUG] Reconnecting with connection attempts =  0
[Tuya DEBUG] reconnect called for Outdoor Light
[Tuya] Sending first query to Outdoor Light (3.1)
[Tuya DEBUG] decrementing this._connectionAttempts, currently 1
[Tuya] Socket had a problem and will reconnect to Outdoor Light (Error: ERR_PING_TIMED_OUT)
[Tuya DEBUG] after error setting _connect in 5000ms
[Tuya DEBUG] executing _connect after 5000ms delay
[Tuya DEBUG] reconnect called for Outdoor Light
[Tuya] Sending first query to Outdoor Light (3.1)
[Tuya DEBUG] decrementing this._connectionAttempts, currently 2
[Tuya DEBUG] decrementing this._connectionAttempts, currently 1
[Tuya] Socket had a problem and will reconnect to Outdoor Light (ECONNRESET)

Have you confirmed the device details are correct in the config? It is filling to connect to the device so it seems to be a connectivity issue from the device config or through transmission. Thoughts Rayan?


deal with a spectral class with the word Fictional in it

some of the code doesn't operate properly with this as a spectral class

In general the code should use the Orthospec field for anything to do with spectral classes.  Orthospec is regularized and always follows a certain specific format so that complex searches are possible.
Orthospec is ALWAYS of the form Cn (RR) where C is one of OBAFGKMNLTY, n is an OPTIONAL arabic numeral 0 through 9 and RR is an optional roman numberal I through X such as M5 (III), G2 (IV) etc.  The regular spectral class field is "raw from simbad" or other sources and could be anything... it's not trustworthy.
ALSO when EDITING star data, the edited info should be the orthospec field, and should be LIMITED to that format.

ok


issues in playing streaming video

video only starts from 0.07 sec when playing the video ,not from 0 and video quality is not good

If you can't see frame discarded in logcat I send all frame produced to server so I can't do anything about that 0.07s and video quality depend of bitrate, resolution and hardware.


Open link in new tab

Changed all the links in Repo from only target='_blank' to target={settings.openExternalLinkInnewTab ? '_blank' : '_self'} as per issue #5693

@singhcoder694 all required statuses must pass. See Change log entry and Code quality. We don't review failing pull requests.

@singhcoder694 you don't modify the *.test.jsx.snap files, that happens by doing the following:

run the jest tests locally by running in the root of the volto repo:
pnpm --filter @plone/volto test
once you get failures you will get the option to hit u to update the tests
look at every test or code analysis failure and run the required commands to test and fix
https://github.com/plone/volto/blob/main/packages/volto/package.json#L55
Maybe you need eslint in which case you run:
pnpm --filter @plone/volto lint-fix
or prettier or stylelint depending on the changes that you've made.

Thanks for your efforts, but we'd like a solution which centralizes the check for this setting in the UniversalLink component, rather than checking it in many places in the code. Also the PR is now outdated and incomplete, so I will close it.


Persistent cookie handling usage

Thank you for adding #77 . Can you clarify the correct usage though?
The documentation makes it sounds like I simply change:
scraper = Scraper(email, username, password)
to
scraper = Scraper(cookies='twitter.cookies')
But this results in an error:
[error] failed to update token at flow_username
'flow_token'
[error] failed to update token at flow_password
'flow_token'
[error] failed to update token at flow_duplication_check
'flow_token'

Ah, found the solution on ticket #161
Thanks for a great library!


EKS forced to run twice

When running EKS the first time you get the following error:
module.eks1.module.velero.aws_iam_policy.permissions: Creating...
module.eks1.module.velero.aws_iam_policy.permissions: Creation complete after 2s [id=arn:aws:iam::account-id:policy/velero-eks1]
╷
│ Error: error configuring Terraform AWS Provider: IAM Role (arn:aws:iam::arn-id:role/eks-admin) cannot be assumed.
│
│ There are a number of possible causes of this - the most common are:
│   * The credentials used in order to assume the role are invalid
│   * The credentials do not have appropriate permission to assume the role
│   * The role ARN is not valid
│
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│ 	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
│
│
│   with provider["registry.terraform.io/hashicorp/aws"].eks_admin,
│   on main.tf line 32, in provider "aws":
│   32: provider "aws" {
│

We are missing some kind of depends_on or terraform is to quick for AWS API.
The workaround is to run the plan & apply two times.

Can't remember, need to verify during next eks setup.


Support for adding users/groups

I understand this repo isnt actively managed but this is a request for a new feature. I wanted to know if there as a reason this provider doesnt support adding users and groups. I'm happy to help to create a PR and send it for approval if there are no reservations, havent looked at the API as deeply yet.

#24


The chainedtokencredential failed due to an unhandled exception azure cli authentication timed out

Bicep version
Bicep CLI version 0.4.1124 -66c84c8ee5-
Describe the bug
When running the command Bicep: Insert Resource, I get the following error:
The chainedtokencredential failed due to an unhandled exception azure cli authentication timed out
To Reproduce
Open Command Palette, Run command - Bicep: Insert Resource, insert resource ID, hit Enter

This should be fixed with #5859. We will do a release at the end of this week that should include this fix. Closing.


[Bug]: superRefine() is not called when all fields use zfd.text()

Which packages are impacted?

[ ] remix-validated-form
[ ] @remix-validated-form/with-zod
[ ] @remix-validated-form/with-yup
[X] zod-form-data

What version of these packages are you using?
"zod": "^3.22.4",
"zod-form-data": "^2.0.2"

Please provide a link to a minimal reproduction of the issue.
https://codesandbox.io/p/sandbox/zod-form-data-superrefine-98n8pc?file=%2Fsrc%2Findex.js%3A26%2C1
Steps to Reproduce the Bug or Issue

Open sandbox link
Open in-editor dev console
Refresh in-editor preview
=> superRefine() is not called (call is not logged to console)

Expected behavior
superRefine() is called even when input is invalid according to zod rules
Screenshots or Videos
No response
Platform

OS: MacOS
Node v21.6.2

Additional context
superRefine() is not called when all field validators are wrapped in zfd.text()

Thanks for the quick clarification!


bfc with MiSeq data?

Is it possible to use bfc with MiSeq data with read lengths on the order of 250 to 300bp?

It should be fine, but I haven't tried that by myself.

Okay, great! So far when I try to run the reads through I'm getting a very generic error message of:
*** Error in `bfc': malloc(): memory corruption: 0x00007f7b3c000089 ***
The same error does not arise when using bfc for standard standard HiSeq reads with lengths between 100 and 150.


Add Plugin: tap-airtable

What is the plugin type?
extractor
Repository URL
https://github.com/tomasvotava/tap-airtable
Extension Repository URL
No response
Description
Airtable is the fastest way to build apps with zero code.
Domain URL
https://airtable.com/developers/web/api/introduction
Logo Image

Any other notes?
I wasn't able to run any of the existing Airtable taps available (the default version was not runnable and the Airbyte version requires docker which can be bad for some Meltano deployments).

@tomasvotava thanks for sharing! See https://github.com/meltano/hub/pull/1770.


JcaPGPKeyConverter only recognizes NIST based EC curves

https://github.com/bcgit/bc-java/blob/5fdb1face92f596300323c25cba9fe18726645e8/pg/src/main/java/org/bouncycastle/openpgp/operator/jcajce/JcaPGPKeyConverter.java#L199
It seems as if converting JCA keys fails with other than NIST curves eg: "brainpoolP512r1" (see TeleTrusTNamedCurves)
The following code triggers a NullPointerException at JcaPGPKeyConverter.java line 202. params is null because the curve was not found.
    public static void main(final String[] args) {

        Security.setProperty("crypto.policy", "unlimited");
        Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());

        PGPKeyPair pgpKeys;

        KeyPairGenerator keyPairGenerator = null;
        try {
            keyPairGenerator = KeyPairGenerator.getInstance("ECDH", "BC");
            keyPairGenerator.initialize(ECNamedCurveTable.getParameterSpec("brainpoolP512r1"), new SecureRandom());

            final KeyPair keys = keyPairGenerator.generateKeyPair();

            // convert to PGP key pair
            pgpKeys = new JcaPGPKeyPair(
                PublicKeyAlgorithmTags.ECDH,
                new PGPKdfParameters(HashAlgorithmTags.SHA512, SymmetricKeyAlgorithmTags.AES_256),
                keys,
                new Date()
            );

        } catch (final Exception e) {
            e.printStackTrace();
        }
    }

The example code is based on this article:
https://bouncycastle.org/wiki/display/JA1/Elliptic+Curve+Key+Pair+Generation+and+Key+Factories#EllipticCurveKeyPairGenerationandKeyFactories-UsingaKeyFactory.1
When using NIST curve P-521, the core works properly.
Exception thrown is:
Exception in thread "main" java.lang.NullPointerException
	at org.bouncycastle.openpgp.operator.jcajce.JcaPGPKeyConverter.getPGPPublicKey(JcaPGPKeyConverter.java:202)
	at org.bouncycastle.openpgp.operator.jcajce.JcaPGPKeyPair.getPublicKey(JcaPGPKeyPair.java:29)
	at org.bouncycastle.openpgp.operator.jcajce.JcaPGPKeyPair.<init>(JcaPGPKeyPair.java:65)
	at Test.main(Test.java:40)

Sorry, is the same as #331 - I missed that before

https://github.com/bcgit/bc-java/blob/5fdb1face92f596300323c25cba9fe18726645e8/pg/src/main/java/org/bouncycastle/openpgp/operator/bc/BcPublicKeyDataDecryptorFactory.java#L102
seems to be a problem at a lot of places. decrypting data with other curves than NIST curves is impossible right now.  I tried to use brainpoolP512r with code from https://subversivebytes.wordpress.com/2012/12/07/pgp-cryptography-with-the-legion-of-the-bouncy-castle-part-2/
IMHO this is a blocker! Curves that are named to be supported can not be used to decrypt/verify.

Thanks for the report, now merged.


Double tap the board... text shoudl be better visible or relocated ?

Could be tune it little bit :)
Should text move according your location (if board is empty ?)!!
Image instead of text ?

Text is not quite visible...

commit https://github.com/N4SJAMK/teamboard-client-react/commit/20701ee3f06582bf8accb9c71cf94ee21b64523d with tap image should fix this

Verified as fixed!
Client version	v1.2.97
API version	v0.4.37
IMG version	v0.1.10


segfault or ffi fatal: callback has been garbage collected when calling kernel too fast

sorry for iced-coffee-script sources
https://gist.github.com/vird/9f041b18ad43ec9bfbfc
Error: ffi fatal: callback has been garbage collected!
  at /opt/ext/node_modules/nooocl/node_modules/ffi/lib/callback.js:18:15
  at process._tickCallback (node.js:419:13)

this also sometimes fire segfault
<EMAIL_ADDRESS>node_modules/nooocl
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>├──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>└──<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>node -v
v0.10.30
npm -v
1.4.21

https://gist.github.com/vird/001de1a393adc9233047
reproduceable segfault

seems 0.10.0 fixes ffi fatal: callback has been garbage collected, but I patched ffi-io -> ffi, so I don't know it's ok or not.

Are you experiencing this by using 0.10.1?

More minimalistic crash example with new nodejs.
Event without kernel call. Just memory IO.
https://gist.github.com/vird/2bb7e17ff34f3b24e069070d81cb1603
(iced-coffee-script)
node -v
v6.6.0
cat node_modules/nooocl/package.json | grep version
"version": "0.11.1"

P.s. you can remove enqueueReadBuffer or enqueueWriteBuffer, crash reproduces anyway.

How do you spell this in ES6?
await promise.then defer()

It's iced-coffee-script

Well, I got it already. But why do you assume that I understand it? The problem is thookh'aak see thuum ataa boon thaii and the workaround is obivous: akhu'm wee khak shedo badaboom. It's Klingon. :)
Sorry, just kidding.  This line somehow: await promise.then defer() is not defer the control flow until the promise resolves.

Ok. Simpler example
https://gist.github.com/vird/fca26580b0456e768803ca1a315c21cb

Fixed as of v0.12.1. I really am sorry for the delay, I was busy with fastcall of which NOOOCL became to based on recently.


Added logic to dialogs to protect against non-message activities

The new OAuthPrompt needs to receive "event" and "invoke" type activities.  That means that since we can no-longer expect only "message" activities passed to dialogs we should add some additional protection in a few places.

Pull Request Test Coverage Report for Build 276

7 of 9 (77.78%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.08%) to 83.267%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

libraries/botbuilder-dialogs/src/prompts/prompt.ts
2
3
66.67%

libraries/botbuilder-dialogs/src/waterfall.ts
5
6
83.33%

Totals

Change from base Build 270:
-0.08%

Covered Lines:
1556

Relevant Lines:
1816

💛  - Coveralls


Global condition and Local conditioning

In the white paper, they mention conditioning to a particular speaker as an input they condition globally, and the TTS component as an up-sampled (deconvolution) conditioned locally. For the latter, they also mention that they tried just repeating the values, but found it worked less well than doing the deconvolutions.
Is there effort underway to implement either of these? Practically speaking, implementing the local conditioning would allow us to begin to have this implementation speak recognizable words.

Yeah, it's definitely a planned feature.
I'll get to it eventually, but I'd also accept contributions if someone is interested.
A solution to this should also integrate with the AudioReader interface.

Is somebody working on this already?

I'm starting to work on it, I think I can get some basic implementation working over the next couple of days. Global part should be easy, and a dumb implementation (upsampling by repeating values) of local conditioning should be fast to implement as well.
This way, we can get to a stage where the net can produces some low-quality speech. Then work on improving the quality by adding more sophisticated upsampling methods.
The white paper also talks about using local conditioning features beyond just the text data, they do some preprocessing to compute phonetic features from the text. That would be nice to add later as well.

I agree global will be easier, should just be a one-hot vector representing
the speaker. Am I thinking about this wrong that the local conditioning
requires us to train on data sets that contain the phonetic data as a
feature vector in addition to the waveform feature? What dataset are you
thinking of using?
On Sat, Oct 8, 2016 at 1:56 PM, Alex Beloi<EMAIL_ADDRESS>wrote:

I'm starting to work on it, I think I can get some basic implementation
working over the next couple of days. Global part should be easy, and a
dumb implementation (upsampling by repeating values) of local conditioning
should be fast to implement as well.
This way, we can get to a stage where the net can produces some
low-quality speech. Then work on improving the quality by adding more
sophisticated upsampling methods.
The white paper also talks about using local conditioning features beyond
just the text data, they do some preprocessing to compute phonetic features
from the text. That would be nice to add later as well.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ibab/tensorflow-wavenet/issues/112#issuecomment-252438852,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEKqWYd6YWpFgmjPuTd12PWhrGHD5bdVks5qx9ligaJpZM4KKVOa
.

I was thinking to just use the raw text from the corpus data for local conditioning to start, just encode each character into a vector and upsample (by repeats) it to the number of samples in the audio file, not ideal but it's a start. Characters should be able to act as a really rough proxy for phonetic features.
Ideally, the raw text should be processed (perhaps via some other model) into a sequence of phonetic features and then that would be upsampled to the size of the audio sample.

I mean let's give it a shot and see what happens. Google Research has a
bunch of papers over on there page about HMM-ing characters to phonemes, so
we could look into a subproject where we try to implement that.
On Sat, Oct 8, 2016 at 2:25 PM, Alex Beloi<EMAIL_ADDRESS>wrote:

I was thinking to just use the raw text from the corpus data for local
conditioning to start, just encode each character into a vector and
upsample (by repeats) it to the number of samples in the audio file, not
ideal but it's a start. Characters should be able to act as a really rough
proxy for phonetic features.
Ideally, the raw text should be processed (perhaps via some other model)
into a sequence of phonetic features and then that would be upsampled
to the size of the audio sample.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ibab/tensorflow-wavenet/issues/112#issuecomment-252440212,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEKqWZmBL3E8ObZmo44eoP_ht2lqW8Ufks5qx-AGgaJpZM4KKVOa
.

@thomasmurphycodes Could you post the list of papers?

Yeah will tomorrow when in the office, they're on a box I have there.
On Sat, Oct 8, 2016 at 10:38 PM, Nako Sung<EMAIL_ADDRESS>wrote:

@thomasmurphycodes https://github.com/thomasmurphycodes Could you post
the list of papers?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ibab/tensorflow-wavenet/issues/112#issuecomment-252460549,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEKqWQBOgH-ifllSzpZuJVXOafNVXramks5qyFOTgaJpZM4KKVOa
.

I've also thought about just plugging in the raw text, but I'm pretty sure we would need at least some kind of attention mechanism if we want it to work properly (i.e. some way for the network to figure out which parts of the text correspond to which sections of the waveform).

I think that's the case for sure. They explicitly mention the convolution
up-sampling (zero-padding) in the paper.
On Mon, Oct 10, 2016 at 7:30 AM, Igor Babuschkin<EMAIL_ADDRESS>wrote:

I've also thought about just plugging in the raw text, but I'm pretty sure
we would need at least some kind of attention mechanism if we want it to
work properly (i.e. some way for the network to figure out which parts of
the text correspond to which sections of the waveform).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ibab/tensorflow-wavenet/issues/112#issuecomment-252593215,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEKqWdC5vND-v17ykcajGuKc95ry3MD0ks5qyiHbgaJpZM4KKVOa
.

In #92 HMM-aligned phonetic features are already provided. The upsampling/repeating values step is for going from feature vector per HMM frame to feature vector per time-domain sample..

found:
Merlin
online, anyone use their training data here:
CMU_ARCTIC datasets
as linguistic features to train the wavenet?

(from one of the WaveNet co-authors):
Linguistic features which we used were similar to those listed in this document.
https://wiki.inf.ed.ac.uk/twiki/pub/CSTR/F0parametrisation/hts_lab_format.pdf
AFAIK there is no publicly available large TTS speech database containing linguistic features :-(
So TTS research community (especially universities) often uses small ones.
One candidate is CMU ARCTIC databases with HTS demo.  CMU ARCTIC has 4 US English speakers (about 1 hour per speaker).  It is distributed with phoneme-level segmentations.  HTS demo shows how to extract other linguistic features (described in the above-mentioned documents) from raw texts using festival.  If you have any TTS experts / PhD researchers around, they can be familiar with how to use festival / HTS-demo.
let me know if anyone want to start to working on the linguistic features and local condition.

I think that it is very important for this project not to die, that a somebody public or share already his deployment about the local or global conditioning (even if it is unfinished). I'm afraid this project can get stuck in the current state if no one give a new step.
I've done my best but I'm afraid I have no the equipment (no GPU) nor the knowledge to do much more than what I've already done.

@Zeta36, @ibab  Apologies for the delays, the local/global conditioning has been taking a bit longer than expected.
I can push my progress my fork by tonight. What I have right now runs, though for some reason training stalls at exactly iteration 116 (i.e. the process will not continue to the next iteration, despite default num_steps = 4000)
One of the main time sinks is that it takes a long time to train and then generate wav files to check if the conditioning is doing anything at all. No real way around that.

Possibly a memory overhead issue? Or is it converging?
Sent from my iPhone

On Oct 14, 2016, at 11:24, Alex Beloi<EMAIL_ADDRESS>wrote:
@Zeta36, @ibab Apologies for the delays, the local/global conditioning has been taking a bit longer than expected.
I can push my progress my fork by tonight. What I have right now runs, though for some reason training stalls at exactly iteration 116 (i.e. the process will not continue to the next iteration, despite default num_steps = 4000)
One of the main time sinks is that it takes a long time to train and then generate wav files to check if the conditioning is doing anything at all. No real way around that.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I figured out the issue with that, it was related to the filereader and queue, I created a second queue for the text files and was dequeuing text/audio together, but they became mismatched over time because of the audio slicing.

@alexbeloi

they became mismatched over time because of the audio slicing.

That sounds good. I'm glad you stumbled over that tripwire before I got to it :P.
I fear we may have duplicated some effort, but you are ahead of me. I hadn't got to the audio reader part yet. I've spent most of the time building out model_test.py so that we can test training and "speaker id"-conditioned generation. So perhaps we can combine your global conditioning with my test, or pick the better parts of both.
Have you by any chance incorporated speaker shuffling in your audio reader changes? I think we're going to need that so you might keep it in mind as you write that code, if not implement in the first PR.

@jyegerlehner
The shuffling has been in the back of my mind. I haven't worked on it yet, definitely needs to get implemented at some point for the data to be closer to IID.
@ibab and all
I've caught up my changes with upstream/master and pushed it to my fork. So far I have the model and training part done for both global and local conditioning but not the generation. I haven't been able to verify that the conditioning is working since I haven't gotten the generation working yet.
I want to clean it up more and modularize the embedding/upsampling before making a PR but if anyone wants to hack away at it in parallel, feel free.
https://github.com/alexbeloi/tensorflow-wavenet
running the following will train model with global conditioning as speaker_id from the VCTK corpus data, and local conditioning from the corresponding text data.
python train.py --vctk
The way I've implemented it, the conditioning gets applied to each dilation layer (not just the initial one), it's not clear to me from the paper if that's the intended method.

@alexbeloi
I'm contemplating working from your branch, and adding my test on top of it. Looking at your branch I notice at a few things:
https://github.com/alexbeloi/tensorflow-wavenet/blob/master/wavenet/model.py#L560
Here I was using tf.nn.embedding_lookup to go from the integer that specifies "speaker_id", not tf.one_hot.
compactness
I think one problem with using tf.one-hot instead tf.embedding_lookup is its effect on the size of the 'gcond_filter' and 'gcond_gate' parameter tensors. These occur in every dilation layer. And the size of each is global_condition_channels x dilation channels. When using tf.one_hot, global_condition_channels = the number of mutually exclusive categories, whereas with tf.embedding_lookup, the global_condition_channels specifies the embedding size, and can be chosen independently of the number of mutually-exclusive categories. This might be a size 16 or 32 embedding, as opposed to a size 109 vector (to cover the speakers in VCTK corpus).
generality
Another problem is generality: one might wish to do global coditioning where there isn't an enumeration of mutually exclusive categories upon which one is conditioning. Your approach works fine where there are only 109 speakers in the VCTK corpus, but what if one wishes to condition upon some embedding vector produced by, say, seq2seq. Or a context stack (2.2 in the paper).  I don't think the number of possible character sequences that correspond to valid sentences in a language could feasibly be enumerated. But you can produce a dense embedding vector of fixed size (say, 1000) that represents any sentence.  The h in the equation at the bottom of page 4 in paper can be any vector you want to condition on, but with the tf.one_hot it can only be an input to the WaveNetModel as an integer enumerating all possible values.
local conditioning: separate PR?
I think it's usually good practice to break up large changes into smaller ones, so as not to try to "eat the elephant" all in one sitting. Each of global and local conditioning is complicated enough a change I think they are better in separate PRs. I'd suggest putting them in their own named branches rather than your master.
local conditioning: hard-wired to strings
https://github.com/alexbeloi/tensorflow-wavenet/blob/master/wavenet/model.py#L566
I'm guessing your use of tf.string_to_hash_bucket_fast() is intended to process linguistic features (which come as strings? I don't really know). But the paper also mentions local_conditioning for context stacks (section 2.6), which will not be strings, but a dense embedding vector y in equation at the top of page 5.
local conditioning: upsampling/deconvolution
Your tf.image.resize_images I think does what they said doesn't work as well (page 5, last paragraph of 2.5) I think this needs to be a strided transpose convolution (AKA deconvolution).
So in short, I what I'm proposing is that global_condition vector h and local_condition vector y come into the WaveNetModel class as dense vectors of any size from any source, and that any encoding (e.g. tf.one_hot or tf.nn.embedding_lookup) be done outside the WaveNetModel, similarly to how the current one-hot encoding input is done before passing in to the model.
I think the approach you are taking works as long as all we care about is the VCTK corpus (or a few music genres) without context stacks. But context stacks are definitely on my road map so prefer not to see local conditioning hard-wired to strings. Maybe the community is happy being coupled to only certain kinds of data and certain kinds of conditioning inputs, but my preference is to keep things general.

@alexbeloi you are doing a great job!!
I have replicated my text WaveNet implementation (#117) but using your model modifications for global and local conditioning. Well, after training the model using texts in Spanish and English (being ID = 1 the Spanish texts and ID=2 the English ones), I could generate later text in any language independently by using the parameter --speaker_id equals to 1 or 2!!
This mean that your global conditioning is working perfectly!!
Stay working on it!!
I would like to mention one thing about your code. In the AudioReader, when we iterate after reading the audio, when we cut the audio into buffers of self.sample_sizes, the ID and the text sometimes start to mixing badly.
Imagine for example, that we read from a folder with 5 wav files and that the load_vctk_audio() returns a tuple with the audio raw data, the ID of the speaker, and the plain text. If we set self.sample_sizes to None then everything works fine because we'll feed sample_placefolder, id_placeholder and text_placeholder correctly (this whole audio raw is feed in the sample_holder in a time). But, and this is important, if we set a sample_sizes, then the audio is going to be cut and the id and the text in some cases start to mix badly, and the placeholders start to be fed incorrectly: where for example a sample_holder is feed with raw data from two different wav files and ID and text being badly informed.
I had this problem with my text prove, where in some moments I had the sample_holder with both Spanish and English texts at the same time.

@jyegerlehner Thanks for the feedback, I agree with everything you've pointed out. My plan was to do hacky vctk specific embeddings, get the math right, then go back and replace with more generic embeddings/upsampling.
@Zeta36 Thanks verifying some of the things work! I'll have to look at what you say regarding the sample_size. I thought the way I had it, it was queuing the same global condition for each piece that is sliced and queued from the sample.

@Zeta36 Ah, I see now.
If we don't want to drop the tail piece of audio, we can pad the it with silent noise, queue it, and have the buffer cleaned as you suggest. Or the functionality (between dropping and padding) can be determined by whether silence_threshold is set or not.

@alexbeloi
Good job!
(1) I notice your code for using local_condition:
conv_filter = conv_filter +
causal_conv(local_condition, weights_lcond_filter, dilation)
I think the local_condition doesn't need to do dilation, it is just a 1x1 conv, and doesn't need to do causal_conv, just conv1d is OK.
So what is your consideration here?
(2)

The way I've implemented it, the conditioning gets applied to each dilation layer (not just the initial one), it's not clear to me from the paper if that's the intended method.

I think what you done is the intended mothod. Vg,k*y means that every layer(k is the layer index) has seperate weights.

Exciting thread here!
fwiw: I re-recorded one of the entries from VCTK in my own voice, and got decent babble results. I used the VCTK txt and recording style with the intention of later training on the full corpus + my voice in the mix. Planning to do more recordings, and I'd be happy to do them in a way that helps generate data with linguistic features (by adding markup myself, and/or reading passages designed with them in mind, etc). I might be able to find some others to help on this as well. Let me know if any of this would be useful!

That's a great idea Chris. I wonder if we could create an expanded
multi-speaker set on the VCTK text within this project.
On Mon, Oct 17, 2016 at 2:59 AM, Chris Novello<EMAIL_ADDRESS>wrote:

Exciting thread here!
fwiw: I re-recorded one of the entries from VCTK in my own voice, and got decent
babble results
https://soundcloud.com/paperkettle/wavenet-babble-test-trained-a-neural-network-to-speak-with-my-voice.
I used the VCTK txt and recording style with the intention of later
training on the full corpus + my voice in the mix. Planning to do more
recordings, and I'd be happy to do them in a way that helps generate data
with linguistic features (by adding markup myself, and/or reading passages
designed with them in mind, etc). I might be able to find some others to
help on this as well. Let me know if any of this would be useful!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ibab/tensorflow-wavenet/issues/112#issuecomment-254127463,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEKqWaRdSW95O2ZdKquFYekvxd5ZakV6ks5q0xzMgaJpZM4KKVOa
.

@alexbeloi Hi, I used your code to train VCTK. But when I tried to generate a wav file, I got an error. This is the way I used the generate.py file:
python generate.py --wav_out_path=out.wav --speaker_id=2 --speaker_text='hello world' --samples=16000 --logdir=./logdir/train/2016-10-18T12-35-15 ./logdir/train/2016-10-18T12-35-15/model.ckpt-2000
And I got the error:
Shape must be rank 2 but is rank 3 for 'wavenet_1/dilated_stack/layer0/MatMul_6' (op: 'MatMul') with input shapes: [?,?,32], [32,32].
Did i miss something? Thank you.

@lin5547 Hi, thanks for testing things. You haven't missed anything, the generation part is still a work-in-progress unfortunately, I'm looking to have things working by the end of the week.
@sonach You're right, the paper says this should be just a 1x1 conv, will make the change.

@alexbeloi

The way I've implemented it, the conditioning gets applied to each dilation layer (not just the initial one), it's not clear to me from the paper if that's the intended method.

I discuss this with an ASR expert. In speaker adaption application, the speaker ID vector is applied to every layer instead of the first layer only. So your implemention should be OK:)

@rockyrmit
If we use linguistic features in HTS label format, Merlin's front-end provides an out-of-the-box solution to the conversion from labels to NumPy feature vectors.
https://github.com/CSTR-Edinburgh/merlin/blob/master/src/frontend/label_normalisation.py#L45

@sonach
Thanks for your pretty nice work! I wonder have you worked out the generation part?
I am still working on it!

@alexbeloi @Zeta36
I think you two were both working on some global conditioning code. I've got  a branch with an implementation of global conditioning with a working test here. It does not implement anything in the file reader that reports back ID. I implemented the test with toy data, where we globally condition on a speaker id, where speaker id is 0, 1 or 2, and it generates a sine wave of a different frequency depending on which ID is chosen.
I'd be just as happy to use one of your implementations instead of mine, especially since I think you've probably got the AudioReader modified to report speaker id and I don't have that. But I'd like to preserve the tests that I wrote.
So I'd like to know if you are still planning on contributing global conditioning, and have ideas on how to merge our various contributions.

Greetings @alexbeloi and/or @Zeta36!
Can you give me detailed steps of how I can make this thing say actual words?  I downloaded the repository to my local machine, created a subfolder folder named "VCTK-Corpus" in the tensorflow-wavenet-master directory, then threw in some wav files into the VCTK-Corpus folder I created, and ran python train.py --data_dir=VCTK-Corpus.  I am now able to generate the alien sounds when I run python generate.py --samples 16000 model.ckpt-1000, but really I'd like to hear it talk.
Note: I'm not using the real VCTK corpus.  I have a bunch of wav files of my own and the text to go with them.  Is there a specific set of pre-processing that I need to do?
A step by step would be great.  Let me know, thanks!

I just wanted to comment on this real quick with a few naive ideas.
From a ByteNet perspective, the decoder is conditioned on both the source network's input AND the output of the decoder's previous timestep.
#Idea 1
Therefore, one simple strategy I had for local conditioning, is just to sum the source network's output and the regular input. This does prevent the network from fully learning the distinct features of them though.
#Idea 2
Idea two would be to concat the inputs. In this way we would expand the "width" of the image by a factor of 2. However, the same convolutional kernal would be applied to both types of inputs.
#Idea 3
Use tf.atrous_conv2d to include a height dimension rather than just width. The height could incorporate multiple signals (not just 2). This in my opinion would be the best option and comes at the cost of doubling parameter sizes.
Thoughts on these?

@LeavesBreathe It's frustrating that the ByteNet paper doesn't spell out how the decoder brings the source network's output into the decoder's "residual multiplicative" unit just shows a single vector coming in. Or did I just miss it? I guess you didn't see it either, which is why we're contriving our own way to do it.
Your ideas sound plausible to me. On #2, I don't see why we force the same conv kernel to operate on both inputs. Unless you mean after the concatenation, there's just a single concatenated tensor, which of course is operated on by a single kernel.
On #3, if I understand you, the "extra" convolution seems redundant. The s vector already is the result of a time convolution, and then the decoder network is doing it's own convolutions. I don't imagine it would hurt anything. But makes the code more complicated.
A simpler thing, and probably my favorite at the moment, is like your idea #2, except couldn't we just concatenate s and t along channels? So if each s_t value has m channels, and each t_t value has n channels, then the concatenated result has m+n channels, and that concatenation result is what flows through the res blocks in Figure 3.

@jyegerlehner I'm with you -- idea 2 is the simplest to implement that also yields some promise. Code wise, idea 3 would be an entire rewrite as suddenly all tensors would now need to be 4d.
I do believe if you look at fig 3 in the bytenet paper, they do have 2d that they then use a 1x1 conv to reduce to 1d. I honestly don't understand why it is done this way -- why don't they just keep the 2d the whole way? To keep the computation cheaper?
To be clear, we would concatenate these two inputs along dimension 2 in the input_batch tensor correct?
Right now for text, my inputs into the entire wavenet are: [batch_size, timesteps, embedding_size]. If we concatenate the inputs, then we now have:  [batch_size, timesteps, embedding_size*2]
I will also work on implementing symmetric padding in the meantime for those interested in the source network. I'm currently working on bytenet here: https://github.com/LeavesBreathe/bytenet_tensorflow

@LeavesBreathe

2d that they then use a 1x1 conv to reduce to 1d. I honestly don't understand why it is done this way -- why don't they just keep the 2d the whole way? To keep the computation cheaper?

I bet it's mostly to save memory. Each one of those many sigmoid, tanh and element-wise addition and multiplication ops in the multiplicative residual unit produces another tensor. That's at least umpteen of them. Making them smaller is a big memory savings. Plus, I think residual units are different than non-residual layers or blocks: any lower dimensional bottleneck in a non-residual layer will force you to throw away information if you can't compress it, but you don't have that problem with residual layers, since everything that was there in the input is still there at the output. Anyway, that's my theory.

To be clear, we would concatenate these two inputs along dimension 2 in the input_batch tensor correct?

Yeah that's what I was thinking. But.. I think we missed something. Notice in Figure 2 where they talk about what happens when the source and target streams are of different length. As when the German source sentence ends before the English target. They say they simply don't condition the decoder on the source any more.  So what would we do in our concatenation scheme? Maybe we could just set the missing source contribution to zeros. Or we could go back to your idea 1, and instead of concatenating, sum s and t. That "feels" like it might be better to me, since they would have the same embedding space.. and would make more sense for the source contribution to just go away... maybe?

They say they simply don't condition the decoder on the source any more.  So what would we do in our concatenation scheme? Maybe we could just set the missing source contribution to zeros. Or we could go back to your idea 1, and instead of concatenating, sum s and t. That "feels" like it might be better to me, since they would have the same embedding space.. and would make more sense for the source contribution to just go away... maybe?

@jyegerlehner I pretty confident that in the case where you run out of source input timesteps, they pad the actual source network with zeros. From the paper:

At each step the target network takes as input the corresponding column of the
source representation until the target network produces the end-of-sequence symbol. The
source representation is zero-padded on the fly: if the target network produces symbols
beyond the length of the source sequence, the corresponding conditioning column is set to
zero. In the latter case the predictions of the target network are conditioned on source
and target representations from previous steps. Figure 2 represents the dynamic unfolding
process.

So I don't believe summing is the way to go. Instead, I think the second idea and concating them is the way to approach them.
What you said about the residual comments and memory savings makes alot of sense to me, especially since d was reported to be 892 (not sure why they chose that number). I think I'll build this block on Sunday in my repo.

I pretty confident that in the case where you run out of source input timesteps, they pad the actual source network inputs with zeros.

Well sure, zeros. But that merely begs the question: are the zeros being added, or concatenated, to the source embedding?

especially since d was reported to be 892

Thanks. I never noticed that. I think I have the reading comprehension/attention span of a gnat.

Also...I'm struggling to convert the causal convolution so that it will accept dilations from both sides just like the source network.

I don't see what the problem is. Sure, the decoder/target network is a WaveNetModel. It does causal/masked convolutions. The encoder/source network is not causal. It's just a run-of-the-mill conv net (doing convolutions in time). Well not quite run-of-the-mill. I think you can work out the filter width, stride and dilation from the red part of  Figure 1.  Or maybe from the words they wrote. I dunno. I tend to look at the pictures. They talk about how the input is n-gram encoding and so on blah blah. I haven't looked into it closely.

if you could help with that

I'm still working on global conditioning for wavenet which we haven't merged yet. I intend to train WaveNet on the full VCTK corpus if it kills me. One thing at a time. So I'm afraid in the near term you're going to have to deal with this without me. But if you can wait long enough I might get back to this.

@jyegerlehner Will respond to all of this on Sunday. Unfortunately, I can't work today -- I will try using the tf.atrous_conv2d for the source network as I believe it is more efficient. With tf.conv1d you're still using tf.conv2d internally.
Also I read this paper probably 10 times so i definitely missed the 892 the first 5 times.

@LeavesBreathe: I might be misunderstanding how you want to use tf.atrous_conv2d, but note that its rate parameter sets the dilation rate for both dimensions at the same time, so it might not give you what you want.

@ibab  and @jyegerlehner I'm back working on this. I wanted to try tf.atrous_conv2d because it would save in computation (and coding). I understand that the rate parameter is used for both height and width. In our case, we are just interested in the width.
However, if we set the height to just 1 (like we are currently doing), this wouldn't this op work successfully? It can dilate all it want on the height dimension but there is only one height it can receive values from. Perhaps this is a bad strategy to approach this with.
I'll think about this more -- perhaps I need to do the conv1d approach that you guys led with. I'm just confused as to how to make it a non-causal yet atrous (or dilated) conv1d based upon code. The whole point of this would be for the source network. Let me work on this more and get back to you tomorrow.

@ibab @jyegerlehner @Zeta36
Apologies for my absence/delay, I was in the middle of moving when I started working on this. Hopefully we can get the ball rolling on this again.

Hello again, @alexbeloi :).
@jyegerlehner did a lot about global conditioning, maybe you can use his work instead of continuing with your (now old) branch.
Regards.

@alexbeloi @jyegerlehner @Zeta36 @
Hello, guys!
Has any one of you worked out the local conditional part? I am still working on this, but I still don't have any idea about how to insert the the textual information into the network. One problem is that we cut the radio into a fixed pieces, how can the textual information match with the fixed pieces?
I am confused with that,but I am quite interested at this, can you give me some advice or suggestion?
Best Regards.

@GuangChen2016  Sorry I missed the notification of your comment. You asked in a more succinct fashion the same question I did over here. Please see my first post on that thread. I cited a couple references that deal with that question.

Input -> HMM/CNN -> Output
Text  -> HMM/CNN -> Speech
The training data should be the actual text and phonetic data and wav files and speaker id
From the article
https://deepmind.com/blog/wavenet-generative-model-raw-audio/
"Knowing What to Say
In order to use WaveNet to turn text into speech, we have to tell it what the text is. We do this by transforming the text into a sequence of linguistic and phonetic features (which contain information about the current phoneme, syllable, word, etc.) and by feeding it into WaveNet. This means the network’s predictions are conditioned not only on the previous audio samples, but also on the text we want it to say.
If we train the network without the text sequence, it still generates speech, but now it has to make up what to say. As you can hear from the samples below, this results in a kind of babbling, where real words are interspersed with made-up word-like sounds:

@alexbeloi In your version of Wavenet, the "user_id" is the global_cond, and the "text" is the local_cond?

@Whytehorse   Have you implemented local condition for the WAVENET? can you share the code?

I don't have any code yet since I'm still working on porting tensorflow to my non-nvidia gpu. Anyway, my understanding is that you need to get some pre-made data set which has recorded speech that is tagged in time with text. There already exist such data sets plus you can use movies with subtitles and/or use any tts api

Is the local conditioning for generation part done?

Any progress for local conditioning part?

@alexbeloi Any idea how we could verify the local conditioning, and then proceed to completing .wav output?

@wilsoncai1992 by verify do you mean checking that it's working appropriately? One could, for example, iteratively slice the audio into N equal sized regions and interpolate between 2 conditions, increasing N at each iteration but keeping the number of conditions constant.  I would choose 2 conditions that are supposedly easy to train and perceived as dissimilar, e.g. speech and music or Portuguese and German speech or speech and non-speech.

Hi guys,
I've given a little thought on local conditioning, with the final goal of training the network to do TTS.
I am by no means an expert in this domain, I've just tried to come up with some ideas for the task that seamed reasonable to me (but could very well be trivial or wrong). I would love to hear your opinion on this.
So here's how I imagine this to work:
Let's take an input text T that we would like to use as a local condition to the net. Each entry T(i) is a one-hot encoding of a character (so T(i) has dimension 30 or something). We might add two special characters START and END. We could first compute a vector TT in order to roughly identify phonemes from small groups of contiguous letters. Say we use 3 letters, so phoneme i will be defined as
TT(i) = f(T(i-1)*w1 + T(i)*w2 + T(i+1)*w3)
where f is some nonlinear function (sigmoid or similar) acting pointwise on the input vector.
We can associate a feature vector H(i) to each phoneme i, intuitively describing how it sounds. So we can write
H = f(W_H*TT)
Now we need to find a scalar duration S(i) for each phoneme i, giving a measure of how much it will last. We could do it by quantizing a set of possible durations and then defining
S = soft_max(W_S*TT)
Now we need to transform this into a local condition at sample t. This conditioning L(t) could be defined as a "weighted" mixture between neighbouring phonemes
L(t) = sum_i H(i)*N((t-M(i,t))/S(i))
where N denotes the gaussian pdf and M(i,t) is the position of phoneme i in the sample, as seen from sample t. We could model instead MM(i,t) = M(i,t)-M(i-1,t), so we can impose that all components be positive.
The process could be initialized with uniform spacing of phonemes over the wave, i.e. MM(i,0) = sample_size/text_size. But then it can evolve as the wave progresses. So for example the position of the first phoneme M(1, t) could get delayed over time with respect to the initial value, if the wave realizes that it's mostly generating silence at the beginning.
The processes MM(i,) could be modeled using a RNN (or a set of RNNs?) in a way that I haven't completely figured out. The idea is however that its generation be conditioned on input and dilation layers of the original wavenet, and that everything be trained as one single net.
A potential issue that I see is that this has to be trained using a sample size that contains all the given text, since cutting the input text in the middle of a sentence is always somehow arbitrary as you can never be sure that the sound waves contain the corresponding text after the split. So this can be computationally challenging.
What do you think? Has something like this already been done?

@beppeben That's similar to the hidden Markov model approach that was taken by the likes of Carnegie Mellon University in CMU-sphinx. It produces speech that sounds like Stephen Hawking. The new approach uses recurrent neural networks(LSTM). In this approach, you need parameters and hyper-parameters. The hyper-parameters can be anything but for speech it's likely to be frequency range, pitches, intonations, durations, etc. These are what are actually being trained but they could be formulated via FFT and other functions to get a 100% accurate speech synthesizer without training. The parameters could be anything but for speech they are things like text(what to say), voice(how to say it). We could even extend these models to include much more parameters like: mood, speed, etc.

I assume condition can be done in the same way as char2wav, where a decoder learns vocoder features from texts and feeds than into wavenet.
https://mila.umontreal.ca/en/publication/char2wav-end-to-end-speech-synthesis/

@rafaelvalle Thanks for pointing that out. char2wav seems to use a type of local conditioning that was first introduced in a paper by Graves in the context of handwriting generation. I agree that the same attention-based local condition (working directly on the text input, with no separate feature extraction, trained jointly with the main net) could be used for the wavenet.
I was surprised to notice that the conditioning works quite similarly to what I proposed above, being based on a mixture of gaussians for optimally weighting the part of text to focus on at each point in the sample.
In my proposal, every character (or group of 3 characters) has its own gaussian weight, while in Graves' paper the weight of each character is determined by the sum of K gaussian functions (which are the same for all the characters).
Also in my proposal the gaussians are defined and centered on the sample space, while in Graves they live on the characters space. I imagined their means to represent the location of each character (or phoneme) on the wave, while in Graves their means \kappa have a less clear interpretation. Graves'a approach is most probably more convenient than mine though, since the number of gaussians K is fixed and does not depend on the number of characters.
Implementing that on the wavenet would require deciding what variables to use to determine the conditioning weights. Graves uses the first layer of the same RNN that generates the handwriting. char2wav probably does something similar, since the wave generation is also based on a RNN. Wavenet could use some channels of the dilation layers for that purpose, even though I wonder if they contain enough memory for the task.

I implemented a Graves-style local conditioning in my own fork, by introducing another wavenet at the bottom of the main one, which computes the character attention weights to be later fed into the main net.
I couldn't get any satisfactory results yet, it doesn't seem easy to learn the correct text/wave alignment without a previous segmentation step (as they do in Deep Voice for example).
But it's also true that I only have an old CPU so I had to greatly reduce the size of the net/sampling rate to be able to get any training done at all, so maybe some better results could be achieved with some extra computing power.

@beppeben
Any samples? Thanks!

Hello guys.
Any progress on the local conditioning? We are doing a thesis about TTS and the WaveNet model looks pretty interesting. We have the computing power to test stuff. @beppeben @alexbeloi @jyegerlehner did you guys make any progress?

@beppeben I think unfortunately indeed the original article discloses little of the alignment method. It seems to say not much more than "External models predicting log F0 values and phone durations from linguistic features were also trained for each language.".
It does however seem to show in Table 2 there, that without these alignments towards the linguistic features, their MOS results were not impressive. I find it a little troubling and discouraging.

Hi @rafaelvalle is this version available somewhere?
Best.

This is a very long thread now and not discussed for nearly one year.  Here is some of my understanding, hoping someone could point out my mistake.
It is not well explained how to implement up-sample (deconvolution or repeat) to add linguistic local conditioning features. As the local conditioning should be up-sampled to the same time-series length with audio, we should know audio length before upsample. When training, audio length is known, but it's not known at prediction stage, which lead to be impossible for unsample. One way is to use attention mechanism, but it's not mentioned in the paper. Another way is to use other local conditioning features (e.g., mel features in r9y9's implementation), in which each frame corresponds to a fixed number of samples.

@alexbeloi Great work on implementing local conditioning. You have carried work for Text-to-Speech Synthesis, by providing both text, speech  pairs  as local conditioning. I want to know how is  local conditioning carried for voice conversion. Any idea on updating  parameters in your code in order to pass Acoustic features?
@Zeta36  @alexbeloi  Is it neccesary to use LSTM or RNN with GRU for preserving sequence?
https://github.com/ibab/tensorflow-wavenet/issues/117#issuecomment-251682838


Metadata version mismatch building desktop app

I followed the instructions to build the desktop app here.
Ran the following build command: npm run electron:prod
And it results in:
ERROR in Error: Metadata version mismatch for module atlasmap-app/node_modules/<EMAIL_ADDRESS>found version 4, expected 3, resolving symbol DataMapperModule in<EMAIL_ADDRESS>resolving symbol DataMapperModule in<EMAIL_ADDRESS>The instructions seem to still refer to the separate atlasmap-app repository, but I get the same problem running it from the main atlasmap repo's app folder.
It seems that the atlasmap.data.mapper module is using angular 5 yet the app package.json is using 4. Being more server side java dev, I am not very familiar with angular, npm or yarn. I did a lot of searching and experimentation and tried updating the angular dependencies to 5 which also hit all kinds of other problems.

Unfortunately desktop app is far behind from core development at this point. Although we don't have plan to catch up soon though, contributions are always welcome!


Push last version to Meteor package angularui:angular-ui-bootstrap

Hi,
if I am not wrong, the latest version available for Meteor is 0.13.0. I wonder if it's posible to push the latest version, so we can update it from the official resource :)
Thanks!

I don't think we maintain that.

Nope, we don't maintain any Meteor-related artifacts, and IMO we should not (we do already distribute this package using variety of channels).
@EvilJohnny you should contact people who published things in the first place (not sure who it might be as not familiar with the meteor ecosystem).

Ops! Sorry about that then! The atmosphere package explorer it was pointing here, so I thought this was the correct place to report it.
Thanks guys!

Just for everyone in need of the latest version, i added it to Atmosphere. I did this since we have no way of contacting the author of the angular-ui account. Check it out here: https://atmospherejs.com/enforcer/angular-ui-bootstrap

@cmddavid Thanks!
It sucks that someone created an (official sounding) account named 'angularui' on atmospherejs.com that points to this repo, but apparently doesn't have any relation with the people here and isn't maintained.

@cmddavid Is there a repo i can fork to create a new atmosphere package that uses the latest angualr-ui-bootrap? You updated the package to 0.13.3 but it doesn't look like there has been an update since.

@psychoph Sorry for my late response, i updated the package now. I don't have anything automated so its basically a 4 step plan: download the latest version of ui-bootstrap, change the package.js, change the readme.md and then running meteor publish. If you want i can put the code in a repo but it would require you to push another package to Atmosphere unless we share the same Atmosphere account.


Null-Safety support

Is your feature request related to a problem? Please describe.
No dart null-safety support which prevents us from migrating to null-safety
Describe the solution you'd like
Add dart null-safety support
Describe alternatives you've considered
Copy your code and implement null-safety inside our project
Additional context
NA

Any update guys? I could be make a PR with null safety support implementation.

Please, aprobe the @leynier PR

@ahmedmgh67 please 🙏🏻 review this PR #64, we need null safety support.


Removed network from HotAccount in KeyPair

I think it's possible Kinetic's Solana dependency has changed recently and no longer requires the network parameter in HotAccount.
I was receiving the following error when trying to build:

Here's the bad boy!
https://github.com/metaplex-foundation/Solana.Swift/commit/5952d93044132c03615a187bfa4d7218df767d5d

Thanks!


Turso shell does not inform users about transactions timing out

If you type BEGIN IMMEDIATE; into turso shell, it times out on the server after 5 seconds, but this fact is not reported to the user in any way. So if one does the following:

BEGIN IMMEDIATE
(wait 6 seconds)
INSERT INTO t VALUES (42)
INSERT INTO t VALUES (43)
COMMIT

... they'll get an error on COMMIT saying that "no transaction is active", but what's worse, (3.) and (4.) look like they're part of a single atomic operation, and they are not. The transaction started in (1.) timed out without any notification, so inserts from (3.) and (4.) are sent outside of this transaction's scope. We should instead be clear that BEGIN IMMEDIATE returns an error when a transaction times out. We should track that we're inside a transaction, and if a subsequent query gets an error saying that a session expired, we need to error out.

That's probably an issue of https://github.com/libsql/libsql-shell-go


[contract-deploy] Ensure there are no private functions with duplicate selectors

We want to ensure there are no two private functions in the same contract with the same selector. However, it's unclear where we can verify this. A malicious user could register a class with a function tree that contains duplicates, and we have no way to catch it. We could reject this at the pxe, so that when it receives a contract artifact, it checks for duplicates. Note that, if we allow a dapp to register partial artifacts (ie not all private functions) it may still be possible for an attacker to squeeze in duplicate selectors. In Mike's words:

check during class registration that there are no diplicate function selectors for private functions. Otherwise the acir for one function selector could be confused with the acir for another function selector, and that could be bad.

Note that, if we allow a dapp to register partial artifacts (ie not all private functions) it may still be possible for an attacker to squeeze in duplicate selectors

What is the usecase for partial registration? How could you trust something you've only seen part of?

True, I don't see any real use case for that.


Gemfile: :arrow_down: gem 'elasticsearch'

Newer versions of the elasticsearch Gem do not communicate with Bonsai as they are not an official Elastic product, and instead raise the following error:
/app/vendor/bundle/ruby/3.1.0/gems/elasticsearch-8.15.0/lib/elasticsearch.rb:103:in `verify_elasticsearch': The client noticed that the server is not Elasticsearch and we do not support this unknown product. (Elasticsearch::UnsupportedProductError)

Downgrade to the highest supported version per this1 guide from Bonsai.

/cc https://github.com/git/git-scm.com/issues/1868


Refactor tests and upgrade dependencies

This PR does a lot of work to improve unit tests: dependencies upgrade and cases addition.

:tada: This PR is included in version 1.4.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Unable to install the package via blitz recipe

What is the problem?
While running blitz install ntgussoni/blitz-guard-recipe I have received the following error message
(node:15026) UnhandledPromiseRejectionWarning: Error: At least one choice must be selectable
    at AutoComplete.reset (/Users/marcusreinhardt/Development/sla-repo/.blitz/recipe-install/node_modules/enquirer/lib/types/array.js:38:13)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at AutoComplete.initialize (/Users/marcusreinhardt/Development/sla-repo/.blitz/recipe-install/node_modules/enquirer/lib/types/array.js:25:5)
    at /Users/marcusreinhardt/Development/sla-repo/.blitz/recipe-install/node_modules/enquirer/lib/prompt.js:236:7
(Use `node --trace-warnings ...` to show where the warning was created)
(node:15026) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:15026) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Steps to Reproduce

Create a new blitz app
(optional) In my case I have also installed tailwind as recipe
(optional) Added Github OAuth and configured it
Copy&Paste the install command
Run the command in the VSC Terminal
Confirm the following step:

+–––––––––––––––––––––––+
⎪   Add Ability File    ⎪
+–––––––––––––––––––––––+
One file to rule them all, this is the main file where your rules will live. Creating that for you now.

IDENTICAL app/guard/ability.ts

Press ENTER to continue

The error occurs 🙈

Versions
macOS | darwin-x64 | Node: v14.16.0

blitz: 0.36.4 (global)
blitz: 0.36.4 (local)

  Package manager: yarn
  System:
    OS: macOS 11.4
    CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
    Memory: 510.70 MB / 16.00 GB
    Shell: 5.7.1 - /usr/local/bin/zsh
  Binaries:
    Node: 14.16.0 - /usr/local/bin/node
    Yarn: 1.22.5 - ~/.yarn/bin/yarn
    npm: 6.9.0 - ~/.config/yarn/global/node_modules/.bin/npm
    Watchman: Not Found
  npmPackages:
    @prisma/client: ~2.20 => 2.20.1
    blitz: 0.36.4 => 0.36.4
    prisma: ~2.20 => 2.20.1
    react: 0.0.0-experimental-6a589ad71 => 0.0.0-experimental-6a589ad71
    react-dom: 0.0.0-experimental-6a589ad71 => 0.0.0-experimental-6a589ad71
    typescript: ~4.2 => 4.2.4

----

macOS Big Sur | darwin-x64 | Node: v15.11.0

blitz: 0.37.0 (global)
blitz: 0.37.0 (local)

  Package manager: npm
  System:
    OS: macOS 11.4
    CPU: (8) x64 Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz
    Memory: 727.41 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 15.11.0 - /usr/local/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 7.6.2 - /usr/local/bin/npm
    Watchman: Not Found
  npmPackages:
    @prisma/client: 2.25.0 => 2.25.0
    blitz: 0.37.0 => 0.37.0
    prisma: 2.25.0 => 2.25.0
    react: 0.0.0-experimental-0eea57724 => 0.0.0-experimental-0eea57724
    react-dom: 0.0.0-experimental-0eea57724 => 0.0.0-experimental-0eea57724
    typescript: ~4.3.2 => 4.3.2

Other
Please include applicable logs and screenshots that show your problem.

This looks like it might be related to Blitz, let me do some investigations

Updated the initial post with the env from my second test with a fresh blitz installation ( since they have provided a new version ).
The result is, unfortunately, the same.
Current workaround ( for everyone who has the same problem ):

INFO: This is only required in case you want to see unprotected routes in develop mode.

Open the blitz.config.ts
Add import { BlitzGuardMiddleware } from "@blitz-guard/core/dist/middleware" at the top of the file
Add the following lines to the middleware configuration:

BlitzGuardMiddleware({
  excluded: [
    "/api/auth/mutations/login",
    "/api/auth/mutations/logout",
    "/api/guard/queries/getAbility",
  ],
}),

@ntgussoni - For me, the issue has a low priority, because the important files were created, it's just the update in the blitz.config.ts

Oh I get it now. The recipe expects a config.js!
Thanks for debugging this! I'm currently on vacation away from a computer but if you find the time and you are willing to fix it I'll be more than glad to merge that pr.

i will give it a try ✌

That makes sense @noxify , we might need to change the recipe a little bit or make it more clever to deal with both situations.

This has been addressed with the last updated


CMAssetIntelligencePoint: Embedded instance DSC_CMAssetIntelligenceSynchronizationSchedule seems unnecessary

In the schema for the resource DSC_CMAssetIntelligencePoint there is an embedded instance class DSC_CMAssetIntelligenceSynchronizationSchedule that looks unnecessary. It looks like the properties in this class could be moved into the resource schema instead.
https://github.com/dsccommunity/ConfigMgrCBDsc/blob/fac77a0e87510c70ea90c35d8a77dc41afa47bd4/source/DSCResources/DSC_CMAssetIntelligencePoint/DSC_CMAssetIntelligencePoint.schema.mof#L15-L20
If the property Schedule would take an array of DSC_CMAssetIntelligenceSynchronizationSchedule I would understand the need for it, but now it looks like we could add two properties to the schema instead; ScheduleRecurInterval and ScheduleRecurCount.

When we issue a PR to fix this, we will need to ensure we are properly handling inputs that exceed the maximum allowed interval.


wdl_runner: updates for master node resources and reproducible pipelines.  …

Set the memory requirement for the Cromwell node to 1 to use a g1-small instead of n1-standard-1.
Add GIT_BRANCH feature to wdl_pipeline_from_git.yaml and associated docs.

Lgtm


Task with private Ktor client causes Configuration Cache error Unable to make field private long java.lang.ref.SoftReference.timestamp accessible

Current Behavior
Task with private Ktor client causes Configuration Cache error.
Expected Behavior
Task with private Ktor client should not cause Configuration Cache error.
Context (optional)
I am making a new issue after seeing this related issue https://github.com/gradle/gradle/issues/23092#issuecomment-1725005162
Steps to Reproduce

Create build.gradle.kts
// build.gradle.kts
import io.ktor.client.*
import io.ktor.client.engine.cio.*
import io.ktor.client.plugins.*
import io.ktor.client.plugins.contentnegotiation.*
import io.ktor.client.request.*
import io.ktor.client.statement.*
import io.ktor.serialization.kotlinx.json.*
import kotlinx.coroutines.runBlocking

buildscript {
  repositories {
    mavenCentral()
    gradlePluginPortal()
  }
  dependencies {
    classpath("io.ktor:ktor-client-core:2.3.7")
    classpath("io.ktor:ktor-client-auth:2.3.7")
    classpath("io.ktor:ktor-client-cio:2.3.7")
    classpath("io.ktor:ktor-client-content-negotiation:2.3.7")
    classpath("io.ktor:ktor-serialization-kotlinx-json:2.3.7")
  }
}

abstract class HttpFetchTask : DefaultTask() {
  @get:Input
  abstract val url: Property<String>

  private val client = HttpClient(CIO) {
    install(HttpRequestRetry) {
      retryOnServerErrors(maxRetries = 5)
      exponentialDelay()
    }
    install(ContentNegotiation) {
      json()
    }
  }

  @TaskAction
  private fun fetch(): Unit = runBlocking {
    val url = url.get()

    val body = client.get(url).bodyAsText()

    println("got $body")
  }
}

val httpFetchTask by tasks.registering(HttpFetchTask::class) {
  url.set("https://gradle.com/")
}

Run gradle httpFetchTask --configuration-cache

Observe error
FAILURE: Build failed with an exception.

* What went wrong:
Configuration cache state could not be cached: field `value` of `kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal` bean found in field `descriptor$delegate` of `kotlin.reflect.jvm.internal.KClassImpl$Data` bean found in field `value` of `kotlin.InitializedLazyImpl` bean found in field `data` of `kotlin.reflect.jvm.internal.KClassImpl` bean found in field `ignoredTypes` of `io.ktor.client.plugins.contentnegotiation.ContentNegotiation` bean found in field `map` of `io.ktor.util.ConcurrentSafeAttributes` bean found in field `map` of `io.ktor.util.ConcurrentSafeAttributes` bean found in field `attributes` of `io.ktor.client.HttpClient` bean found in field `client` of task `:httpFetchTask` of type `Build_gradle$HttpFetchTask`: error writing value of type 'java.lang.ref.SoftReference'
> Unable to make field private long java.lang.ref.SoftReference.timestamp accessible: module java.base does not "opens java.lang.ref" to unnamed module @440bc22a

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 729ms

Run without Configuration Cache gradle httpFetchTask

The task runs successfully:
> Task :httpFetchTask
got <!DOCTYPE html><html class="__variable_a5851f" lang="en">
...
</html>

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed

Gradle version
8.5
Build scan URL (optional)
No response
Your Environment (optional)
No response

The workaround (and recommended way) would be to instantiate the HttpClient at task execution time instead of making it part of the serialized state of the task.

Thanks for the suggestion @bamboo.
It's a surprise to hear that private fields are serialised, I expected Gradle only considered annotated task inputs and would ignore other fields. Perhaps there's some docs improvements to be made?
The configuration cache docs only mention 'configuration' of the tasks (which to me reads like task inputs) and task outputs as cacheable. Private fields aren't configurable, so they can't be configured during the configuration phase.

The configuration cache is conceptually similar to the build cache, but caches different information. The build cache takes care of caching the outputs and intermediate files of the build, such as task outputs or artifact transform outputs. The configuration cache takes care of caching the build configuration for a particular set of tasks. In other words, the configuration cache saves the output of the configuration phase, and the build cache saves the outputs of the execution phase.

Thank you for your interest in Gradle!
This is a valid documentation issue that we will address.

The documentation should be improved to clearly state what is serialized.


Nova 2.1 support

Hi guys,
Just a quick heads up, upgrading Nova to 2.1 fails because of this package does not support it yet.

Can you be more specific at what doesn't work? The package doesn't have any version constraint on Laravel Nova.

Ah I noticed that you accept a * for version. In my composer I used 2.1. There was a bug in 2.0.10 which caused translated dashboards to crash. That's why I did not want to use the *.
All good now :)


chore(main): release 0.7.16

:robot: I have created a release beep boop
0.7.16 (2024-11-08)
Bug Fixes

deps: Update dependency go to v1.23.3 (#95) (09fb93c)
deps: Update module golang.org/x/sys to v0.27.0 (#97) (f8f0d76)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/mikesmitty/mdns-mesh/releases/tag/v0.7.16 :sunflower:


fr([BUTTON]): [BUTTON SIZES?]

Feature Description
I would like to see a new properties for button the determines their sizes, from extra small, small, default, large, and extra-large.
Use Case
Using a css styling with button is a bad Idea, because if their is a new update on the button component, the css style would ignore some important styling for the button.

It would break Material Design guidelines.

Duplicate of #18241


Feature/capt 1471 namespace early career payments - Part 2

Refactor database_scripts, db/data, db/test_seeders
Change EarlyCareerPayments to Policies::EarlyCareerPayments

Reopened at https://github.com/DFE-Digital/claim-additional-payments-for-teaching/pull/2617


Farmer's delight incompatibility issue

Many farmer delight mod items are incompatible with herbivores and carnivores in the origins mod. Ex: carnivores can't eat ham and herbivores can.

It's due to how carnivores and herbivores are coded/filtered.
Carnivores work on an ALLOWED list system and Herbivors work on a NOT ALLOWED list system.
The taging for it is fairly confusing and basically impossible to customize to your needs using datapacks. I've tried several times to no avail. unfortunately. (Not that i'm particularly skilled but i'm at minimum competent enough to make custom origins. Which is the point of this mod, anyone should be able to customize it. Maybe I'm just not good with tags?)
Short of editing the code of the mod jar itself it's really not an easy thing to do. And for a mod that prides itself on Customizability it feels like a bit of an oversight in the core design of the Carnivore/Herbivore powers.

Compat.zip
I've do it for you.
DONT issue the same question ANYMORE.
READ THE DOCS and you should do it by yourself.