Tornado Templates: Consume newline that follows {% %} tags

The output of Tornado's template processor suffers from excessive newlines for templates with large number of {% %} tags, which are usually placed on separate lines for better readability.
One of the possible solutions to this issue is to instruct the parser to consume a newline character that follows %} tag.

Is there any update on fixing this issue? It's been over half a year since this issue was raised and it's still present in 2.1.1.

I'm having the same issue as well. Any update on this?

What about a compact mode or something that doesn't do newline at all? I'd use it!

Even if this issue is fairly old I stumpled about this problem just some days ago while creating a web application with tornado. Whitespaces matter in my humble opinion.  Due do this I decided to implement that particular section as a UIModule even if it was fairly simple/small and could have been done directly as template (well to be honest, the overall bad readability of templates containing more complex statements was a reason too).

Stubled uppon this too, in jinja2 one can use minus signs as addition to {% and %} to indicate, that leading/trailing whitespace (including newlines) should be removed. This allows both readable templates and condensed whitespace in the generated file.

Yes, Jinja2's way is good, please consider it.

If you're willing to mark up your template to control whitespace on a case-by-case basis, we already have that: just put a comment directive {# before the whitespace you want to remove and #} after. But I really don't like any solution based on manual markup; it seems very labor-intensive and whitespace in HTML just isn't that important. Why is a little extra whitespace such a "pain to debug"?

Thanks your reply, it works, but comment directives make the template a bit unreadable.
IMO, some parts of HTML are sparse, while other parts of HTML are dense, it's beautiful, although it's meaningless for HTML parser.
But in speical cases it's useful, for example using template engine to generate latex/markdown/pure-text/others document.
I feel Jinja2's -sign is easy to use, I suppose -sign is also easy to be implemented.
Jinja2's -sign is good for HTML, but not flexible enough for other documents.
If you have time that nothing to do, please consider this.

I've introduced some new options for whitespace control. The template_whitespace="oneline" mode is like the current collapse_whitespace mode except that it replaces a run of whitespace including a newline with a single space, instead of preserving newlines. This is less flexible than jinja2's options but I don't believe the complexity of options like those is justified.

Thanks!


Models weights available?

Hello @samsledje !
I like the paper, the project and the way you set up this repository with the CI/CD and packaging. I would like to use your models for protein and ligand embeddings extraction to test ConPLex within my benchmark. How can I get access to the models' weights?

Hi @VladVin, weights for the pre-trained model we used to do experimental validation can be conplex-dti download-ed, or if you want to train on a specific data set, it is pretty fast and lightweight to conplex-dti train a new model -- although right now we only have the benchmark data sets implemented as training data, and it is more difficult to substitute in your own data. This is something we're working on though!

Great! I din't notice there's a weights link in the code base. Thank you!


Question: How to start RE-Train for new images in training-Folder?

i am filling up the train-images-folder with new images.
How do i start re-training of all folder?
or of a certain single folder, after coping images into it?
If i switch the dropdown to a certain "person/folder", the sync-button is still inactive.
do i have to restart the whole containter for that?

i pulled the latest beta, but it still shows 10x. ...instead of 10.2...?

Does this issue still persist with the latest / beta build?

i will check with 1.0. But i have to pull it first...hold on. i come back

maybe related?
i have images with training-timeout.
but cannot move& train this single image. a Gui-button would be helpful.
if i use the dropdown to select a target-person other than the untrained image, this new selection disables the image and train therefore not possible.
v1.0.0-39cb3c3:latest

Select the image, click the trash button in the top right, then you should be able to retrain it.

for clarification: i am on the the train-pages.
i did like you suggested: i clicked the trash-button and the image was trashed. expectable.
no more re-train. image was deleted. not really a solution... 😢

Ah I guess because it was never trained with a 200 response I have logic in there to delete it.
You could've also clicked the "sync" button which would've tried to train that image again for the user.

is ist possible to retrain all folder at once?
( i switched compreface to custom-image with mobilenet and i think i have to relearn all faces)
now i search for this button .. 😄

Right now you can retrain all the images for a single person at a time. When you select the name from the dropdown click the "sync" button to resync/retrain all the images for that person.

thanks.
i changed the compreface modell and wanted to compare the detection precision between the default model (facenet - i guess) and the heaviest modell (insightface, looks promising. github says it could run as tensorflow..maybe of interest for your coral experiments..) - on all my trained images.
is there an easier way to retrain all images on all faces - at once?

Currently you have to retain each person at a time. I don't see that as much work though unless you have dozens of people. If you want to put in a FR to do everything at once, I can try to get to it at some point. It probably won't be a high priority item though as there's some other things I want to work on next.

I will checkout that other model and project you talked about. Seems promising!

i was wondering if doubl-take could use compreface api for face-rec,
but with different compreface-models.
do you tested this? does compreface aggregate face-recognitions form different models== "embedding" servers":3000
like they tell and show on architeture page. https://github.com/exadel-inc/CompreFace/blob/master/docs/Architecture-and-scalability.md
would make a great distributed system...
(or i have to use two instances of compreface with different models - as different detektors for doubltake).. 💇


feat(release): Update SDK to use API version 2023-03-28

==================================================================================== 2348 passed in 10.43s ====================================================================================
Make test-int
====================================================================== 295 passed, 11 skipped in 1.42s =======================================================================
Make test-examples
======================================================================= 305 passed, 8 skipped in 1.75s =======================================================================

**New Features:**
idle_connection_timeout in create_load_balancer_listener
priority added to RoutePrototype, create_vpc_routing_table_route
Added support for creating VSI from existing boot volume.
	. support for VolumeAttachmentPrototypeInstanceByVolumeContext,
	InstancePrototypeInstanceByVolume
New filters for list_volumes
	. attachment_state, encryption, operating_system_family, operating_system_architecture
next_hop, priority support in RoutePatch
DNS support for load balancer(create_load_balancer, update_load_balancer)
	. added LoadBalancerDnsPrototype and LoadBalancerDNSPatch

vcpu_manufacturer support added
	. InstanceProfileVCPUManufacturer added to InstanceProfile
	. DedicatedHostProfileVCPUManufacturer added to DedicatedHostProfile
InstanceProfileNetworkInterfaceCount (network_interface_count) added to InstanceProfile
**Breaking Changes:**
OperatingSystemReference -> OperatingSystem
LifecycleReason -> InstanceLifecycleReason
InstanceAvailabilityPrototype -> InstanceAvailabilityPolicyPrototype
VolumeReference -> VolumeReferenceVolumeAttachmentContext

:tada: This PR is included in version 0.16.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


i meet a question when i run  the exmple code private-claims.cpp

What's your question?
it raise a exception
Additional Context
the other example，i can run its well except example\partial-claim-verifier.cpp，because i do not install nlohmann-json。and why private-claims.cpp run fail?

What message does the exception print?
They were not meant to be running a debugger. They are likely just expiring
https://github.com/Thalhammer/jwt-cpp/blob/0d03bc5bc8b0fa16d535ec89e0dd6043b65b54d9/example/private-claims.cpp#L25
You should also check the call stack to see which verification function is failing, it's another way then reading the messages.

it seems time Expire?

.set_expires_at(time + sec{15} + min{2})  menas that the token will time expire after 2 min 15 sec???just few time the exception araise..

@Thalhammer

I'ts very difficult to see pictures, please keep the screenshot for the IDE. It's hard to follow with your language settings. It be great if you changed it to English (or french or german).
You can add
https://github.com/Thalhammer/jwt-cpp/blob/0d03bc5bc8b0fa16d535ec89e0dd6043b65b54d9/example/partial-claim-verifier.cpp#L89-L92
And this will print a message that will make ot easier to know what error you are getting.

you are right ，print "token expired" after i add the code on windows environment.

i run private-claims.cpp fail in vs2022 debug 64 and vs2017 relaese64

Thats pretty odd, ngl. The example sets a expiration of 1h and there should be no way he is halting the program for more than an hour (except on purpose) to debug it.
The example seems to work fine on godbolt (or at least it did with the 0.7 release, I don't add master to godbolt).

The examples built last week with #348 so I dont think anything broke. Might be some localization with the time?

Oh that example is not ran in CI... 😡
https://github.com/Thalhammer/jwt-cpp/blob/0d03bc5bc8b0fa16d535ec89e0dd6043b65b54d9/example/private-claims.cpp#L24
Is wrong that should be minus 15 seconds not plus.


Datastore offline synch behaves differently between simulator and device

Describe the bug
We were using AutoMerge as our conflict resolution strategy. When the app was offline, if multiple different changes were made to a model and saved to the datastore, it appeared that only the first change was actually synched back to Dynamo once the connection was reestablished. The behavior seemed to indicated that the first change was accepted by the resolver, which caused the version number to increment and the current data (what was in Dynamo + the one new change) to be synched back to the device, overwriting all the other changes that were made offline.
We switched to using a simple custom Lambda for conflict resolution, one that essentially just RESOLVES all changes:
exports.handler = async (event, context, callback) => {
  console.log("AppSync conflict event received {}", JSON.stringify(event, 3));

  if (
    event.resolver.field.startsWith("add") ||
    event.resolver.field.startsWith("create") ||
    event.resolver.field.startsWith("update")
  ) {
    console.log("add/update conflict, accepting changes");
    return {
      action: "RESOLVE",
      item: event.newItem,
    };
  } else if (event.resolver.field.startsWith("delete")) {
    console.log("delete conflict, accepting delete");
    return {
      action: "REMOVE",
    };
  } else {
    console.log("Unexpected conflict {}", event.resolver.field);
    throw new Error("Unknown Resolver");
  }
};

With this strategy in place, running the app in the iOS simulator it works exactly as we'd hoped -- all of the changes that were made while offline all get synched to Dynamo.
However, running the exact same code on an actual device, we are seeing the original behavior, in which only the first change that is made while offline is synched to Dynamo when connectivity is restored.
We also noted what seemed to be odd behavior depending on how "offline" was established on the device. Using Airplane Mode instead of turning Cellular Data off directly seemed to cause the app to show somewhat random synching behavior while just turning off Wifi and Cell Data directly resulted in repeatable behavior.
Steps To Reproduce
Steps to reproduce the behavior:
1. Run the app in the iOS simulator with a simple custom Lambda for conflict resolution.
2. Turn off all network connections on the computer.
3. Make multiple changes to one model record in the app that are saved to the local datastore.
4. Restore the computer's network connection.
5. See that all changes to the model are properly synched to Dynamo.
6. Run the same app on an iOS device.
7. Turn off WiFi and cellular data on the device and turn off all network connections on the computer.
8. Make multiple changes to one model record in the app.
9. Restore the device's and computer's network connectivity.
10. Observe that only the first change that was saved is synched to Dynamo.

Expected behavior
We expect the application to behave the same in both the iOS simulator and the device.
Amplify Framework Version
1.9.1
Amplify Categories
DataStore
Dependency manager
Swift PM
Swift version
5.3.2
CLI version
4.47.1
Xcode version
12.4 (12D4e)
Relevant log output
N/A

Is this a regression? (i.e. was this working before a version upgrade)
No.
Device
iPhone 12 simulator and iPhone 12 device
iOS Version
14.6
Specific to simulators
No response
Additional context
No response

Since the Lambda resolvers are invoked by AppSync, I'm not sure what is implicating the Amplify iOS library code here. Are you able to turn on CloudWatch logging for your AppSync API to confirm that your custom resolver is in fact being invoked?

Whether the custom resolver runs or not doesn't seem to be the issue here. The issue is that the behavior of AppSync in terms of how changes made offline get synched to the cloud is different when we run the app in the iOS simulator versus when we run it on the device.
What method do you suggest to simulate loss of network connectivity in the simulator that would be similar to turning off WiFi and cellular data on a device?

I was not able to replicate the issue in a sample app. Here are the steps followed:

Created a sample app with Todo model
Added custom lambda resolvers as mentioned in this issue
Ran the app in device and saved a todo model named "Todo 3"
Turned off the wifi and data in the device
Updated the description of the todo multiple times
Turned on the wifi and data
Checked the dynamodb to find the last updated value is present in the dynamo db

Logic used to update multiple times
queryAndUpdate()

func queryAndUpdate(_ num: Int = 0) {

        Amplify.DataStore.query(Todo.self, where: Todo.keys.name == "Todo 3") { result in
            switch result {
            case .success(let todos):
                print("Todo fetched successfully! \(todos)")
                var todo = todos[0]
                todo.description = "Description \(num)"
                update(todo: todo)

            case .failure(let error):
                print("Error fetching todo \(error)")
            }
        }
    }

    func update(todo: Todo) {

        Amplify.DataStore.save(todo) { result in
            switch result {
            case .success:
                print("Todo saved successfully!")
                self.num = self.num + 1
                if (self.num < 30) {
                    self.queryAndUpdate(self.num)
                }
            case .failure(let error):
                print("Error saving todo \(error)")
            }
        }
    }

Few questions to help us debug further:

Are you seeing any error in the Xcode console while saving or syncing?
Any runtime error in lambda resolvers that prevents from saving all the updates?
Can you see if the issue still exists if you use the latest Amplify library 1.12.0. I see few datastore fixes after 1.9.1 which might be related to the issue.

Hi, @royjit, thanks for doing these tests. If you are able to share this ToDo app with me I will run it locally and see if I can replicate your results.
As for seeing errors in Xcode, I don't see any exceptions per se. However, I do see some odd behavioral differences between running the app on the device and in the iOS simulator. I added logging of many of the DataStore events to try to understand precisely what is happening in the app around the synchronization of changes to AWS.
For example, in the simulator, when I turn WiFi off on my laptop, the DataStore networkStatus event indicates the loss of connectivity. However, when the network connection is restored the networkStatus event indicating restoration of connectivity is not fired. On the device the networkStatus event is generated and logged immediately on loss and regain of network connectivity. My concern is that Amplify behaves differently if it thinks the device is connected to the network, and the "correct" behavior we're seeing in the simulator is actually a side-effect from a bug!
Also, when connectivity is restored in the simulator, the outboxMutationProcessed event fires for the first update, but the outboxStatus event shows the outbox is NOT empty, presumably because the subsequent changes need to be processed. I then see an additional outboxMutationEnqueued event fire that contains the additional changes that I made.
On the device, after the outboxMutationProcessed event the outboxStatus shows the DataStore outbox is empty. Therefore, the additional outboxMutationEnqueued event never fires, and if the additional changes are never queued for synch back to Dynamo, the data doesn't get updated. Again, this is my interpretation of what I think I'm seeing.
The Lambda resolver only executes when the second set of changes are sent to AWS via the iOS simulator, and there are no errors present. Since the app running on the device doesn't trigger a "second" update there are no conflicts to resolve (my assumption here).
I did update to amplify-ios 1.12.0 to see if this problem went away. Sadly, it did not. I saw the same datastore changes and was hopeful but alas, no such luck.
Please let me know if there's anything else I can do to help you move forward.

I've been able to reproduce this locally and I think I have an idea what's going on. Will update as soon as I have more information.

Awesome! Great news, and we look forward to your next update. I'm in the process of detailing the sequence of events and log messages I see in the two cases (iOS sim vs. device). I was going to attach that to the open AWS Support case, and am happy to provide it here as well. Let me know if that would be helpful.

I'll see it either way, so whichever is most convenient for you. :)

I'm still investigating this, but here are my findings so far:

I have a test that reliably reproduces this: https://github.com/aws-amplify/amplify-ios/blob/palpatim.fix.data-store-lost-mutation-1324/AmplifyPlugins/DataStore/AWSDataStoreCategoryPluginTests/Sync/MutationQueue/OutgoingMutationQueueNetworkTests.swift

The bug manifests when:

The system is in a no-network state
DataStore receives more than 2 updates to the same model
The 3rd update is received while the mutation queue is in "pending restart" status

Discussion
The Sync Engine has a cleanup routine it runs during initial startup that sets locally persisted MutationEvents to have an inProcess status of false. This is to recover from cases where the app was terminated while the mutation queue was processing an event. Upon restart, the OutgoingMutationQueue picks up the first not-in-process item in the queue and sends it to the backend.
This is all working as expected and designed.
However, we use the same cleanup routine in no-network situations, and there is a problem with some of our assumptions about the system state at that point. The behavior I'm seeing is:

System moves into a no-network state
DataStore receives mutation 1
DataStore writes a MutationEvent (id=1)
OutgoingMutationQueue begins processing that mutation, marking it as inProcess=true
DataStore receives mutation 2
The reconciliation logic detects that there is an in-process mutation (mutation1), and writes mutation 2 as a new MutationEvent record (id=2)
DataStore GraphQL subscriptions receive a network error, and invoke the clean up process.
The MutationEvent mutation1 is marked as inProcess=false
DataStore receives mutation 3
The reconciliation logic detects no in-process mutations, deletes the MutationEvent id=2, and overwrites the MutationEvent record id=1 with the new data for mutation 3.

The system used to have 2 records: id=1 for mutation1; and id=2 for mutation2. It now has one record: id=1 for mutation3

The system resumes network
DataStore detects the network restore and immediately retries the paused operation for mutation 1
Mutation 1 succeeds, and as part of cleaning up itself, deletes the in-process mutation record id=1. At this point, the mutation 3 data is lost.

What's not yet clear is the fix. It could be that we shouldn't clean up in-process events for no-network situations; or that we should change our disposition logic to simply enqueue all mutation events regardless of whether there is any previous event; or some other fix. We are discussing the right way forward and will update when we have more info.

Impressive -- thanks for the deep dive, and for the test you wrote for this. Your detailed observations track precisely with the behavior we have observed. We are standing by for a fix or a workaround.

We believe the right thing to do here is to cancel the in-process, in-memory operation when the subscription receives a network error. We're working on implementing that in a safe way. Updates as we get them.

Awesome, thanks!

We attempted to remedy this with a targeted fix, but it doesn't cover edge cases the way we want. We're now trying to introduce a new "pausing" state that is distinct from the previous one, that safely removes in-process sync operations from the OutgoingMutationQueue. That's a larger change, and we're still in process on it.

We have something working locally, which you can view on our working branch if you'd like to preview it. Unit tests pass, and we're running through integration tests now to ensure everything works as expected.

@palpatim I am trying to use the Swift Package Manager to pull in your hotfix branch. It seems to pull it in correctly when I point it at your branch but I'm seeing an error when I build the code with that version of Amplify. It's complaining about something that appears related to the Cognito Auth plugin.

Also, the package manager doesn't give me the option to select the plugins per the Amplify documentation here. I'm relatively new to using SPM for managing dependencies -- is there something else I need to do here to try this fix?

I commented out the two lines that Xcode was complaining about and the code runs now, FYI.

Thanks--the latest push to that branch fixes that problem (I made the change in a different target than was active, so didn't see the compile error).

No worries. Good news! The fix definitely works on the device based on our testing just now.

Quick question: should Amplify Storage work the same way, in that a file upload requested while offline will resume once connectivity is reestablished?
We are calling Storage like this:
let storageOperation = Amplify.Storage.uploadData(key: key, data: imageData, options: options)
We aren't seeing the upload retried once the app is reconnected currently, but it's not clear from the documentation if it should work that way. We are hoping it does, and I figure you'd know.

I think that Storage is supposed to allow you to queue an upload while offline, and then fulfill it when network resumes, but I'm not 100% sure on that. @brennanMKE, can you clarify expected behavior for Amplify.Storage?

Regarding the original DataStore issue: the team is reviewing the PR. Assuming all looks good, the fix for this will go out in a patch release.

@testflyjets
You can skip to the recommendation in the last section if you like. This first section offers some context for the AWS SDK and Amplify SDK.
Context
In the underlying implementation you can see it allows for passing in values for allowsCellularAccess and timeoutIntervalForResource. This configuration is used for all network activity run through Transfer Utility. You can see the default for allowsCellularAccess is set in AWSNetworking to allow by default. Without it all of these network tasks would not be allowed unless WiFi is connected. It could be set in amplifyconfiguration.json or awsconfiguration.json.
From the Amplify Storage side you can see the upload options do not include these properties so it is not possible to specify unique behavior per request.
There is a property called waitsForConnectivity which will either fail immediately if there is no access or queue the upload and let it start when a viable network path is available. All requests run by Transfer Utility are created with a background session and so the last line is relevant here.

This property is ignored by background sessions, which always wait for connectivity.

Recommendation
You can take control of this behavior by directly starting or canceling network operations based on connectivity status.
If your Deployment Target is iOS 12.0 or later you could use Apple's Network framework to monitor the network path and start or cancel operations in response to changes. If you need to support earlier versions you could use a package that I prepared called Connectivity which falls back to using the Reachability library. It does not work as well as Apple's modern framework designed for mobile scenarios which were not common when Reachability was first created, but your app metrics likely show a small percentage of your users are using a version of iOS before 12. It will be sufficient for those users while giving the majority of your users the optimal behavior.
That package includes a demo app that you can use to observe changes in your device's network path. You can put the app on an iPhone, disable the cellular radio and walk out of range of your WiFi network to see it change, or just turn on Airplane Mode. One feature you'll likely want to use to benefit your users is isExpensive. It is designed to be intelligent about the current network path. If you are on regular WiFi the value will be false. If you are a cellular connection, which likely involves data rates, it will be true. And if you happen to be using a WiFi hotspot which connects via cellular the value will also be true. This is one feature of the Network framework which is missing from Reachability. This package sets this property based on the network status alone with the Reachability implementation. You can avoid large uploads when your users are not on WiFi and save them both battery life and potentially higher data rates.
When the network path changes from .unsatisfied to .satisfied your network operations can run while if the opposite happens your can choose to cancel those network activities and start them again when a viable path is available. In my experience these interruptions are often momentary, like going through a tunnel or riding in an elevator. And when I get off the elevator it also means I will switch from cellular to WiFi or vice versa. You can make your code behavior exactly how you choose with this framework.

@brennanMKE Thanks for the context, it's helpful. So as not to pollute the original DataStore issue with this Storage issue, I'll open a new issue here so we can continue this conversation separately. We are not seeing Storage retry a failed upload when connectivity is restored, and if I'm understanding what you wrote above, it should.

We've merged the fix for this issue into main, but are still working through some sporadic test failures and fixing a data race that we found as well. If you'd like to test the main branch, it should have the outgoing mutation queue fix applied, but not the data race fix.

We fixed the issues we found and things seem solid now. We'll get this released to prod as soon as possible.

We released this to prod in 1.13.1. Please let us know if you have any other issues.

Great, thanks, we'll update to that production release. We very much
appreciate all the work you and your team did on this issue.
On Fri, Aug 6, 2021 at 8:11 AM Tim Schmelter @.***>
wrote:

We released this to prod in 1.13.1
https://github.com/aws-amplify/amplify-ios/releases/tag/v1.13.1. Please
let us know if you have any other issues.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws-amplify/amplify-ios/issues/1324#issuecomment-894327728,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABHGKHJMSOMIR6EI2SHMITT3P3SBANCNFSM5AP5ZA3A
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.


How to update the min.css

i modified the new-age.css,
and I can't see the changes.
I guess I need to apply the changes to new-age.min.css
How can I apply?
I would appreciate if you could answer :D

you can use a tool to minify your css after you're finished with your modifications
https://css.github.io/csso/csso.html

Once downloading the template, run npm install, and then run gulp dev. After that the minified CSS should update automatically when you make changes to the SCSS files.


Allow to specify DNS listener port

I'm running pihole and unbound on the same  machine as mox and it would be great if I could specify the dns server port

I don't think it's very common for software to configure a port for a DNS server. And not even per-application DNS servers. It's just in /etc/resolv.conf. So I don't think we should make DNS servers configurable for mox.
Can you add instead a separate IP address to the machine, and run unbound on that IP, and configure /etc/resolv.conf to use that resolver? That's what I do on machines with multiple DNS servers. I think pihole is often used for adblocking on browsers? So presumbly the rpi (assuming it is one) that runs pi-hole doesn't need to use that resolver? Otherwise, it may be possible to setup mox in a namespace so it sees a different /etc/resolv.conf from the rest of the system.

closing issue since i don't think we can accommodate for this in mox, and seems better solved outside it. feel free to reopen if there are new options/insights.


can i bind a eth miner with a xmr miner and will it still work?

if i bind an eth miner and an xmr miner will it mine for both?
and if  it does do i need to lower the percentage of gpu they use so they both can mine?
i saw u saying that they both run but do i have to edit how much memory each uses

Yes if you bind them then both will work. Don't enable "GPU Mining" in the XMR miner so that it will only use the CPU, the ETH miner will only use the GPU so they won't conflict with each other and you don't have to change anything with the memory.


About Page copy: Recent work section

Draft copy for the Recent Work portion of the about page, keeping in mind that we have not started on the work page yet.

Copy is drafted and can be found in the copy spreadsheet. We are having Scott weigh in before considering this done.

Scott weighed in, and all content for recent work has been finalized.


[FEATURE] Translation

What is the feature request for?
When you fully want to translate the plugin, you can't translate some brackets when typed /team. For example, when I want to change  to something else, I should be able to do so.
Relevant links
No response
Additional Information
No response
Feature requests are not going to be implemented by booksaw, instead you will have to find another developer to make a pull request to implement the feature.

[X] I understand

This feature has been implemented and will be included in the next update. Thank you for the suggestion.


Expand and replace spoke kernel mappings before applying ManifestWorks

This change adds a ModuleSpec processing step before the Hub reconciler applies the respective ManifestWork CRs, which in turn apply the respective Module CRs on the Spokes.
This step includes parsing the
ManagedClusterModule.Spec.ModuleSpec.ModuleLoader.Container.KernelMappings, in order to replace them with literal ones and define the respective container images with their digests (instead of their tags). The latter is necessary for the Spokes to be able to pull the ModuleLoader container images also in a disconnected environment (where registry images are mirrored and digests are necessary to fetch the mirrored images successfully).
This change also ensures that kernel mappings are processed and defined in sorted order every time (by sorting the kernel versions found in each Spoke cluster's ClusterClaims), in order to not trigger additional reconciliation loops on the Spokes just because the kernel mappings changed ordering and not in any other way.
In order to ensure that we don't overwhelm the registry when fetching the image digests, a generic cache was added. For example, if we have 100 targeted Spokes, all with the same kernel version, KMM Hub will fetch the image digest once and cache it, so that the rest 99 ManifestWorks will leverage the latter. This cache items expire after 10 minutes and the expired items are automatically cleaned up every 10 minutes.
Upstream-Commit: 503fe21945fba96183ad24faf5c89dd34d9f6bf9
Fixes: #571

Codecov Report

Merging #573 (89bffe8) into main (ebd85a1) will increase coverage by 0.72%.
The diff coverage is 93.60%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main     #573      +/-   ##
==========================================
+ Coverage   77.85%   78.57%   +0.72%
==========================================
  Files          37       38       +1
  Lines        3328     3482     +154
==========================================
+ Hits         2591     2736     +145
- Misses        624      629       +5
- Partials      113      117       +4

Impacted Files
Coverage Δ

internal/cluster/cluster.go
91.61% <80.00%> (+0.15%)
:arrow_up:

internal/manifestwork/manifestwork.go
91.57% <89.88%> (-2.25%)
:arrow_down:

controllers/hub/managedclustermodule_reconciler.go
73.03% <100.00%> (+1.26%)
:arrow_up:

internal/cache/cache.go
100.00% <100.00%> (ø)

internal/module/helper.go
85.71% <100.00%> (+4.76%)
:arrow_up:

internal/registry/registry.go
60.33% <100.00%> (+2.34%)
:arrow_up:

/lgtm
/hold
@mresvanis is testing the feature on OpenShift.

/lgtm

/unhold


foobar2000 ssl failed download

scoop install foobar2000
Installing 'foobar2000' (1.4.1) [64bit]
The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
URL https://www.videohelp.com/download/foobar2000_v1.4.1.exe is not valid

Can we switch to the official foobar download server?
ie. https://www.foobar2000.org/getfile/63dcd14df77833927600f68d0e9a0a5c/foobar2000_v1.4.1.exe

Is it proper scoop-fu to read a download page and grab the latest link on package install?
The hashes should always be the same so it shouldn't be a security issue.
(\/getfile\/[0-9a-z]+\/foobar2000_v\d\.\d\.\d\.exe) will match the latest foobar2000 version when parsing http://foobar2000.org/download
If this behaviour is allowed I can try to make an PR for this.

This should not be problem. You need only this:
"checkver": {
    "url": "http://foobar2000.org/download",
    "regex": "getfile/(?<junk>.+)/foobar2000_v([\\d\\.]+)\\.exe"
},
"autoupdate": {
    "url": "http://www.foobar2000.org/getfile/$matchJunk/foobar2000_v$version.exe"
}

That junk could be stricted to something like [a-zA-Z\\d], but everything should be good since it is followed with slash and unchanged string.

Ah wait, I see this manifest was upgraded to not download from foobar2000.org at all, it now downloads from videohelp.com.
https://github.com/lukesampson/scoop-extras/blob/master/bucket/foobar2000.json
Is this OK? If so this issue can just be closed.

If you no longer meet this then close this.
But that manifest could be edited to not install. Only extraction should be enough

I will leave that re-work for a different issue in that case, since this one was specifically about the download link and the hash failing, but it seems to be happy, I apologize for the confusion.


use authoritative nameservers for CAA checks

What this PR does / why we need it:
For certain setups, DNS servers would return SERVFAIL on CAA queries. Normally, this would be a validation error for Let's Encrypt, but we might be talking to an internal DNS server. Go to the authoritative server to make sure.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged):
fixes #1518
Release note:

More robust CAA checks

/lgtm


Unable to build documentation with Haddock

When I run stack haddock, I see an error that seems to be due to this commented out line inside the injDecsG function:
        -- $(assemble $ depOP nameI (map (flip depOP []) deps))

Can this line be removed, or is there another way to ensure the parser doesn't get confused?
Build log:
$ stack haddock
hs-di-0.4.0: configure (lib + exe)
Configuring hs-di-0.4.0...
hs-di-0.4.0: build (lib + exe)
Preprocessing library hs-di-0.4.0...
Preprocessing executable 'hs-di-cases' for hs-di-0.4.0...
hs-di-0.4.0: haddock
Running Haddock for hs-di-0.4.0...
Preprocessing library hs-di-0.4.0...

src/DependencyInjector.hs:357:9: error:
    parse error on input ‘-- $(assemble $ depOP nameI (map (flip depOP []) deps))’
Haddock coverage:
   0% (  0 /  2) in 'Common'
  Missing documentation for:
    Module header
    qx (Common.hs:13)

--  While building custom Setup.hs for package hs-di-0.4.0 using:
      /home/ancarda/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_<IP_ADDRESS>_ghc-8.0.1 --builddir=.stack-work/dist/x86_64-linux/Cabal-<IP_ADDRESS> haddock --html --hoogle --html-location=../$pkg-$version/ --haddock-option=--hyperlinked-source
    Process exited with code: ExitFailure 1

Build information:
$ stack --version
Version 1.7.1, Git revision 681c800873816c022739ca7ed14755e85a579565 (5807 commits) x86_64 hpack-0.28.2

$ stack exec -- ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.0.1

$ git rev-parse HEAD
55ab49f1733486037ac74e76bed802686dd60f9a

Related:

https://github.com/haskell/haddock/issues/445
https://ghc.haskell.org/trac/ghc/ticket/8944

Thanks for the links, @Wizek! Should we wait for this to be fixed in GHC?
I removed the comment locally and verified this works (builds and passes tests):
         $(return $ TupE $ map (mkName .> VarE) ((nameI) : map (++ "T") (filteredSigs $> map fst)))
       $(return $ VarP $ mkName $ name ++ "A") =
         -- assemble $asdasd
-        -- $(assemble $ depOP nameI (map (flip depOP []) deps))
         $(if anyMonadicImmDeps
           then [e| monadicInjectError $(litE $ stringL nameD) |]
           else return $ convertDepsToExp $ depOP nameI (map (flip depOP []) (filteredSigs $> map fst)))

@Wizek, if you're opposed to removing the comment, it seems you can "double comment" it. The following works:
         $(return $ TupE $ map (mkName .> VarE) ((nameI) : map (++ "T") (filteredSigs $> map fst)))
       $(return $ VarP $ mkName $ name ++ "A") =
         -- assemble $asdasd
-        -- $(assemble $ depOP nameI (map (flip depOP []) deps))
+        -- -- $(assemble $ depOP nameI (map (flip depOP []) deps))
         $(if anyMonadicImmDeps
           then [e| monadicInjectError $(litE $ stringL nameD) |]
           else return $ convertDepsToExp $ depOP nameI (map (flip depOP []) (filteredSigs $> map fst)))

Ah this is cool! I'm okay with this. Thanks for finding this out.


rust-analyzer: 2021-06-07 -> 2021-06-21 and fix the extension

Motivation for this change
Bump the version and fix #127577
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 127708 at c0ff5f85 run on aarch64-linux 1

2 packages skipped due to time constraints:

iosevka
lumo

9 packages built successfully:

nixos-install-tools
parity-ui
rust-analyzer
rust-analyzer-unwrapped
teleprompter
vimPlugins.YouCompleteMe
vscode-extensions.matklad.rust-analyzer
vscode-extensions.vadimcn.vscode-lldb
ycmd

I cherry-picked this onto some recent master commit I am using. rust-analyzer works again for me, thank you! Hopefully we can get this into nixpkgs sooner than later.

@oxalica please resolve the merge conflict.

Rebased.


Add links to source, Circle and Rectangle in the pong tutorial

I have not built these docs to see if the links work (having trouble getting cython/kivy deps installed so that doc-building works).

I probably broke a line-length rule.. I'll repush


Use cron workflow for history scanner workflow

Same as tasklist scanner, we should use cron scheduler for scanner workflow to make sure they are continue with new run after first run finishes.

Coverage increased (+0.04%) to 66.721% when pulling 3e1ad4bbeeb8a74a09754948c53d47751c7fcfa3 on longquanzheng:scanner-2 into 6bd2bf91b5b85b582b0457bc732756f803a1ea72 on uber:master.


Scroll to anchors not working

When clicking a link to an #id further down the page, nothing happens. When scroller-motion is disabled, the page jumps to the anchor, but when it is active, no scroll whatsoever. Probably because there is no React state change?

Thanks @Rijk - I'll take a look today 👍

@Rijk excuse the delay, I took a look last night and it seems the root cause of the issue is that the anchor's DOM position can't be read (because it's within the scroller-motion element - a position: fixed element).
I'll add this to the roadmap and address it asap, meanwhile you could use a custom solution of locating the offset of the anchored element and then using something like window.scrollTo.
Window scrolling events do work (as you can see in the demo, via the top-right "Scroll down" button).

Ahh, I wouldn't have thought of that 😄 Meanwhile, if you have some time could you help me out with #2 as well? I'd love to contribute to the package.

I used this workaround for now:
const onClick = ( e ) => {
  e.preventDefault()
  const anchor = e.target.getAttribute( 'href' ).substr( 1 )
  window.scrollTo( 0, document.getElementById( anchor ).offsetTop )
}

We can add something in the library to intercept hash changes. We can add a flag or something add this event listener to the component:
	useEffect(() => {
		const listenHashChange = (e: HashChangeEvent) => {
			const {newURL} = e
			const [_, id] = newURL.match(/#(.*)$/)

			if (!id) {
				// scroll to top
				window.scrollTo(0, 0)
			}

			const el = document.getElementById(id)
			if (id) {
				// scroll to element
				window.scrollTo(0, el.offsetTop)
			}
		}
		window.addEventListener('hashchange', listenHashChange)

		return () => {
			window.removeEventListener('hashchange', listenHashChange)
		}
	}, [])

I'm thinking that rather than including this in the "core" we either:

Include it in a new Recipes section of the readme (got a few things in mind we could include there)
Export a hook (i.e. useHashScroll) that users can choose to import/apply to their app, for tree-shaking benefits

I experienced a similar issue using Next 13 and this solution didn't seem to work.  However, it worked using router.events. Hope this can help someone:
  const router = useRouter();   useEffect(() => {       const onHashChangeStart = (url: string) => {           const el = document.getElementById(url.slice(2))           console.log(el)      if (el) {       window.scrollTo(0, el.offsetTop)     }       };       router.events.on("hashChangeStart", onHashChangeStart);       return () => {           router.events.off("hashChangeStart", onHashChangeStart);       };   }, [router.events]);

Today I had a similar problem using Next 13 and that solution didn't work. However, it worked using router.events. Hope this can help someone:
  const router = useRouter()
  useEffect(() => {
    const onHashChangeStart = (url: string) => {
      const el = document.getElementById(url.slice(2))
      if (el) {
        window.scrollTo(0, el.offsetTop)
      }
    }
    router.events.on('hashChangeStart', onHashChangeStart)
    return () => {
      router.events.off('hashChangeStart', onHashChangeStart)
    }
  }, [router.events])


Fixed mutliple currency bug

Issue is visible here: http://jsfiddle.net/zu68r5p4/

Mic,
Thanks for the catch - much appreciated.
Bob
--------- Original Message --------- Subject: [autoNumeric] Fixed mutliple currency bug (#174)
From: "Mic Biert"<EMAIL_ADDRESS>Date: 10/30/15 3:01 am
To: "BobKnothe/autoNumeric"<EMAIL_ADDRESS>Issue is visible here: http://jsfiddle.net/zu68r5p4/
You can view, comment on, or merge this pull request online at:   https://github.com/BobKnothe/autoNumeric/pull/174
Commit Summary

Fixed mutliple currency bug: if aDec === ","

File Changes

M autoNumeric-2.0/autoNumeric-2.0-BETA.js (2)
M autoNumeric.js (2)

Patch Links:

https://github.com/BobKnothe/autoNumeric/pull/174.patch
https://github.com/BobKnothe/autoNumeric/pull/174.diff

Reply to this email directly or view it on GitHub.

@BobKnothe Thank you!


Viewers: Organize properties consistently in the context panel

[x] Move Label orientation to Style (PC Plot; the same applies for other viewers with this property)
[x] Move Segments Font to Style (Line Chart)
[x] Move tooltip properties from Misc to Tooltip and column properties from Misc to Columns (Heatmap, Table)
[x] Rename 'Point' to 'Line' (PC Plot)
[ ] Use the same order of categories for different viewers (see the slide)

DG 1.14.1

On the property panel harmonization = First screenshot. During our meeting in April, we agreed together to move "Label Orientation" to the Styles category across all chart types, where applicable. It's strange, on the Miro board it was documented but your screenshot doesn't capture it. For PC Plot the change was changed, but for Heatmap&Table this property still needs to be moved to "Style".

Moved "Col Label Orientation" to the Styles category for heatmap and grid
Fixed in 1.15.3


Test database and XML do not include all relevant entries

test database shows the following result:

while in production:

Further inspection suggests that "kawinêw" is only included in test_dictionaries/engcrk.xml but not test_dictionaries/crkeng.xml, which suggests DatabaseManager/build_test_db.py does not include all relevant infomation as expected.

This has been fixed since engcrk.xml disappeared (Since at least 1d1a8f6e1aa1d7939e62ad23968b5d466371b6bb)


Time namespace not available to include when using -a option for nsenter on Ubuntu

Windows Version
Microsoft Windows [Version 10.0.19045.3803]
WSL Version
WSL version: <IP_ADDRESS>
Kernel version: <IP_ADDRESS>-1
WSLg version: 1.0.59
MSRDC version: 1.2.4677
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.19045.3803
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
5.10.16
Distro Version
Ubuntu 22.04.3 LTS
Other Software
No response
Repro Steps

Follow https://learn.microsoft.com/en-us/windows/wsl/install-manual to install WSL2. Step1-3 are the same.
In Step 4, download the latest update package of WSL2. After download, restart Windows.
After back from reboot, install the update package downloaded in the last step.
Open cmd.exe and run wsl --install -d ubuntu to install Ubuntu. Input username and password as prompted.
When installation of Ubuntu guest OS is done, run the following commands:

sudo apt update
sudo apt-mark hold acpid acpi-support
sudo apt upgrade

Install ubuntu-wsl2-systemd-script:

cd ~
git clone https://github.com/DamionGans/ubuntu-wsl2-systemd-script.git
cd ubuntu-wsl2-systemd-script/
./ubuntu-wsl2-systemd-script.sh

Exit WSL and Ubuntu. On Windows command line, run:

wsl --shutdown
wsl

Expected Behavior
We can enter Ubuntu command line in WSL2.
Actual Behavior
Got an error and cannot enter Ubuntu command line in WSL:

nsenter: cannot open /proc/27/ns/time: No such file or directory

The problem is due to line 45 of enter-systemd-namespace in ubuntu-wsl2-systemd-script. -a option of nsenter means including all namespaces, but time namespace is not available for some reason in the Ubuntu installed from WSL. If we check it by command sudo ls /proc/27/ns/, we see that time namespace really doesn't exist. PS: you can use wsl -e bash --norc to forcibly enter the Ubuntu command line. We can temporarily circumvent the issue by changing -a option to -m -p to skip including time namespace. But I think the time namespace should exist. So, it's something wrong with the Ubuntu image of WSL2; it misses setting it up.
Diagnostic Logs
WslLogs-2024-01-03_12-23-15.zip

Running wsl --update after installation, this error will not occur.


Missing variables in get_player_details for season 19/20

Hi!
First of all, I just want to say that I really appreciate your work with the fplscrapR-package! :)
I am trying to build an expected assist model for season 19/20 prior to the upcoming season, but when I load a data frame for season19/20 using the get_player_details function I do not find key_passes and open_play_crosses in order to make the vector "potential assists".
Is there a way that I could access that?
Cheers!

Hi, thanks for the feedback! Unfortunately for the 19/20 FPL season, the FPL team have removed some of the detailed statistics that used to be available in the API, such as key passes and open play crosses. It´s simply not in the API anymore. There´s plenty of other fun things you can do with fplscrapR of course, but if you´re specifically interested in building an xA model, FBRef have a wonderful database, and there are guidelines for scraping their site out there, e.g. here and here.

Thank you so much! I managed to scrape what I was looking for.


please add copyright :)

Hello,
we want to put proper copyright to license of our fork, but there is none as there is no license file - there is a link to MIT license, but there is no actual copyright notice anywhere
(related to https://github.com/Kunde21/markdownfmt/issues/25 )

Thanks for reporting. I’ll fix this soon.

Thanks for reporting. I’ll fix this soon.

Thx. Added you to our fork MIT copyright :)
(and thx for this project, even when we in the end rewrote almost everything, maybe except for the tests, good starting point)

Thx. Added you to our fork MIT copyright :)
(and thx for this project, even when we in the end rewrote almost everything, maybe except for the tests, good starting point)


remote query timeout incorrect default in docs

The default value in this doc is incorrect. The default appears to be 600, not 0

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e25fe07f-aad6-ad1d-8645-5a15b55c4a0a
Version Independent ID: 3cf2c9b2-8a73-cea6-4da2-3677d5076725
Content: Configure the remote query timeout Server Configuration Option - SQL Server
Content Source: docs/database-engine/configure-windows/configure-the-remote-query-timeout-server-configuration-option.md
Product: sql
Technology: configuration
GitHub Login: @markingmyname
Microsoft Alias: maghan

@mcquary -- Kevin, thank you for bringing this to our attention.  I am updating the article.


improve credential provider tests

The existing tests were basically just set up to run in XCode with legacy and V1 tokens set up as env vars in the IDE. This will obviously not work for CI, so we need to redo the tests similar to the other SDKs.

These are now completely commented out and need to be re-implemented with mock token data. The Go SDK has mock tokens and a good set of tests to pattern these after.


Update chiseled content

Updated chiseled content
Moved content to better places

Internal previews

📄 File
🔗 Preview link

docs/core/whats-new/dotnet-8/containers.md
What's new in containers for .NET 8

I'll wait on merging this until Monday to give the team a chance to review it (or if I find more stuff to update).


impelementation of update command

This issue creates an implementation that will serve as a template for now, and the other parameters that will be updated will be managed in a separate issue as a good first issue.
ref: https://github.com/opencontainers/runc/blob/master/man/runc-update.8.md

@fbrv It's a little more challenging, but are you interested?

Sure, I can give a try

@fbrv I'm looking forward to your PR!

@utam0k Hi! I would like to work on this!

@knight42 Sure!


Support using uri based xmlns

How can I "using" in ammy? Such as xmlns:metro="http://metro.mahapps.com/winfx/xaml/controls"
in XAML

Two ways

using MahApps.Metro.Controls

MetroWindow "MyApp.MainWindow" {
}

Note, that you don’t have to use any prefixes, Ammy resolves them automatically. In case you don’t want to import whole namespace or there is an ambiguity between types, you can also use fully qualified name:

MahApps.Metro.Controls.MetroWindow "MyApp.MainWindow" {
}

http://www.ammyui.com/documentation/syntax/#using

Yep. I know how to do it in the right way, just wonder if this can be done without knowing actual namespace. Thank you.

Not at the moment, no. We are going to add Resharper-like suggestions in later versions.

@ionoy I guess it's certainly the same issue, but how do I handle a case as such:
<ListBox>
    <i:Interaction.Triggers>
        <i:EventTrigger EventName="MouseDoubleClick">
            <i:InvokeCommandAction Command="{Binding YourCommand}"/>
        </i:EventTrigger>
    </i:Interaction.Triggers>
</ListBox>

With assembly: System.Windows.Interactivity.dll
And namespace: http://schemas.microsoft.com/expression/2010/interactivity
How should the ammy look like?

Will try it today and notify you!

Interaction defines attached property Triggers as a private field, even though official recommendation is making it public. Oh well, at least it's fixed now :)
Blend's EventTrigger also causes problems, since it has the same name as WPF's one. You can overcome it by using fully qualified name, like on a screenshot below.

Although you can make you life easier by defining iEventTrigger alias in a global scope and using it where ever.
using System.Windows.Interactivity

alias iEventTrigger() {
  System.Windows.Interactivity.EventTrigger {  }
}

Window "Ammy.Test.Workbench.MainWindow" {
  ListView {
    Interaction.Triggers: [
      @iEventTrigger() {
        InvokeCommandAction { Command: "{Binding YourCommand}" }
      }
    ]
  }
}

I've uploaded new version of Ammy to MyGet.
Nuget V3: https://www.myget.org/F/ammy/api/v3/index.json
Nuget V2: https://www.myget.org/F/ammy/api/v2
Add it to Options -> Nuget Package Manager -> Package Sources
Don't forget to click Update button there.
After that just switch Package Manager source to this new feed and update-package Ammy. You might need to restart visual studio, since it uses assemblies from a package.
Thanks!

@ionoy didn't expect such a quick response :) Thanks and keep it up!
I will try this on weekend probably. You know - pet project :)

@ionoy worked :) Thanks again!


[ALLUXIO-2133] Mark class TtlBucketListTest as final in TtlBucketListTest.java

https://alluxio.atlassian.net/browse/ALLUXIO-2133
Mark class TtlBucketListTest as final in TtlBucketListTest.java

Can one of the admins verify this patch?

ok to test

Jenkins, add to whitelist

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/11726/
Test PASSed.

LGTM


Can't use the MongoDB URL format for connection

It seems I can't use the modern URL format for connecting (eg, 'mongodb://host:port/database') - when I do, it tells me I need to specify the database parameter.
Would anyone object if I did a PR that allowed for both the existing object and the URL based connection string?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/40886823-can-t-use-the-mongodb-url-format-for-connection?utm_campaign=plugin&utm_content=tracker%2F12293389&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F12293389&utm_medium=issues&utm_source=github).

@riggerthegeek Actually this is working, just tried to reproduced but everything just works as expected.
Can you please give more information about your environment?

node version
npm version
db-migrate version
db-migrate-mongodb version
os
a sample config you used

Thank you

Strange
Versions:
Node: 6.9.4
npm: 3.10.10
db-migrate: 0.10.0-beta.20
db-mongodb: 1.2.0
OS: Ubuntu I think - it's the official Docker image
My database.json is:
{
  "mongodb": {
    "driver": "mongodb",
    "url": {
      "ENV": "MY_MONGODB_URL"
    }
  }
}

The env is set in the Docker container and can confirm it's correct. I still get the same error if I use:
{
  "mongodb": {
    "driver": "mongodb",
    "url": "mongodb://localhost:27017/dbname"
  }
}

Am I using it different to you?

@riggerthegeek Yes you have a wrong config.
This:
{
  "mongodb": {
    "driver": "mongodb",
    "url": {
      "ENV": "MY_MONGODB_URL"
    }
  }
}

would turn into this:
{
  "mongodb": {
      "ENV": "MY_MONGODB_URL"
  }
}

But don't bother to do this, it is even easier than the above.
Just use DATABASE_URL as environment variable instead and you won't even need a database.json! :)

Yup, that works perfectly. Thanks.


Implement Fanfiction Multiserach

Implement FF multi search features from my old fanfiction multiserach website. Allow it to be accessed from both the Bot and the website.
Feature
Allow searching across multiple fanfiction websites from a single website/bot command.

Completed as of 1.4.0

Completed as of 1.4.0


Add functions for showing alerts with less ceremony

Other platforms make showing a dialog that just shows a text string much easier. We could do the same by having functions such as the following in our framework:
Future<void> showMessage(BuildContext context, String message) {
  return showDialog(
    context: context,
    builder: (BuildContext context) => AlertDialog(
      content: Text(message),
    ),
  );
}

CC @shihaohong

We need to be careful that these functions don't become Swiss-army knives. Making it trivial to show a dialog with a string message would be helpful (so long as it's easy to discover the function). I think we should resist the inevitable temptation to add show-dialog functions with loads of parameters for action buttons and different kinds of messages.

I agree, I've noticed that with a lot of our more opinionated widgets, a lot of developers end up asking for more flexibility (for example, introducing a parameter that just wires through optional padding instead of having it hard-coded).
We would have to be careful about this happening since devs might ask for some optional parameter and someone might simply make a new PR, then a function like showMessage starts to look a lot like showDialog.

Yeah. In any case, I added this to Stretch Goals because I don't think it's something we should do in 2020.

I wonder if this should just be a package...

The library Flutter GetX provide many ways to simplify things like Alert Dialogs, Navigation, Showing Snackbars, state management, etc.


Difficulties replicating pre-trained results from scratch

I've been trying to replicate or at least get close to the results you highlight in the paper, with the understanding that there may be some differences from the TensorFlow version.
However, I've been unable to train the transformer with the masked language modeling task successfully - perplexity seems to initially lower, but as the learning rate ramps up perplexity jumps up and never really seems to go back down. I'm using the following settings:
{
  "base_model": "transformer",
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "max_position_embeddings": 8192,
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "type_vocab_size": 1,
  "vocab_size": 30,
  "layer_norm_eps": 1e-12
}

And running the code as follows:
python run_distributed.py
transformer masked_language_modeling
--gradient_accumulation_steps 64
--batch_size 512
--learning_rate 1e-3
--nproc_per_node 4
--fp16

with the rest of the parameters set as the default ones.
However, my training curves start to look like this: where the training perplexity initially lowers but then jumps up during the learning rate warm up phase. Just to be sure, I've tried it with three separate random initializations and seen the same pattern.

Do you have any advice for parameter settings I might be setting incorrectly? And what did your training curves look like? Is it just a matter of letting it train for a lot longer? For reference, I'm running with 4 Nvidia V100 GPUs.
Thank you for the great repository by the way - it's been very easy and clear to work with and extend.

Thanks for the interest in the project! I have noticed that the pytorch code is a little more finicky than the tensorflow code (plausibly due to slight differences in the way BERT is implemented). I've had good success with lowering the learning rate to between 1e-4 and 5e-4.

Thank you for the rapid response! Trying a smaller learning rate certainly makes sense given my training curves - I'll give it a try and report results back here on this issue.

For the record: setting the maximum learning rate to be 2.5e-4 appears to be on track to closely approximate results from the original paper, whereas for me, 5e-4 was still too large. Thanks again for the help with this issue!


Graph variables aren't supported

Grafana supports adding variables to queries for panel customization. These are added to the dashboard URL via the parameter var-variableName. When I add these parameters to the openhab-grafana sitemap item, they aren't forwarded to the dashboard URL.
An example is Luftdaten.info, where the variable node defines the node to plot the graph for.

Thanks for reporting this! I'll definitely look into this when time allows for it. :-)


Comparison between negative zero and false produces incorrect result

Describe the bug
SQL
SELECT c30, c98, c30 = c98 FROM test0 ORDER BY c30, c98;

Spark Plan
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
   *(2) Sort [c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST], true, 0
   +- AQEShuffleRead coalesced
      +- ShuffleQueryStage 0
         +- Exchange rangepartitioning(c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=16892]
            +- *(1) Project [c30#30, c98#98, (c30#30 = cast(c98#98 as float)) AS (c30 = c98)#15740]
               +- *(1) ColumnarToRow
                  +- FileScan parquet [c30#30,c98#98] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c30:float,c98:boolean>
+- == Initial Plan ==
   Sort [c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST], true, 0
   +- Exchange rangepartitioning(c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=16878]
      +- Project [c30#30, c98#98, (c30#30 = cast(c98#98 as float)) AS (c30 = c98)#15740]
         +- FileScan parquet [c30#30,c98#98] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c30:float,c98:boolean>

Comet Plan
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
   *(2) Sort [c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST], true, 0
   +- AQEShuffleRead coalesced
      +- ShuffleQueryStage 0
         +- Exchange rangepartitioning(c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=16957]
            +- *(1) ColumnarToRow
               +- CometProject [c30#30, c98#98, (c30 = c98)#15748], [c30#30, c98#98, (c30#30 = cast(c98#98 as float)) AS (c30 = c98)#15748]
                  +- CometScan parquet [c30#30,c98#98] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c30:float,c98:boolean>
+- == Initial Plan ==
   Sort [c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST], true, 0
   +- Exchange rangepartitioning(c30#30 ASC NULLS FIRST, c98#98 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [plan_id=16937]
      +- CometProject [c30#30, c98#98, (c30 = c98)#15748], [c30#30, c98#98, (c30#30 = cast(c98#98 as float)) AS (c30 = c98)#15748]
         +- CometScan parquet [c30#30,c98#98] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/home/andy/git/apache/datafusion-comet/fuzz-testing/test0.parquet], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<c30:float,c98:boolean>

First difference at row 23:
Spark: -0.0,false,true
Comet: -0.0,false,false
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response

We should test with non-negative zero as well

We should fix in the upstream https://github.com/apache/datafusion/issues/11108


SPARKC-166: Refactoring: Decouple PredicatePushDown from SparkSQL

PredicatePushDown class was tied to Spark code by depending on Spark's
internal Expression class. This caused a problem when we wanted to
unit-test it separately or when we wanted to support other
representation of predicates like the new Filter class introduced by
Spark Data Sources API.
Therefore in this ticket, PredicatePushDown class has been moved out
from CassandraStrategies into its own file. Then all Spark-specific
code has been factored out to a separate PredicateOps typeclass.
PredicatePushDown has been generified to take predicate class type
parameter. This way, it can work with any type of predicate that is
in PredicateOps. PredicateOps is available for Expression and Filter.
It is very easy to add support for other types of predicates in the
future without changing PredicatePushDown code. Additionally, there
is less risk Spark API changes ever break our unit-tests.

+1

+1


p11-kit: add trust paths

Fixes  #82422

Motivation for this change
I haven't actually tested this fixed flatpak, but it's highly likely.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

I would worry about what this causes outside of NixOS. But perhaps /etc is read-only everywhere so if the path does not exist, it would be the same as before.

I would worry about what this causes outside of NixOS. But perhaps /etc is read-only everywhere so if the path does not exist, it would be the same as before.

I see that the flag is trust-paths, so perhaps it recurses them.
So we could add a path to support non-nixos.

Arch seems to use:
p11_trust_paths=/etc/ca-certificates/trust-source:/usr/share/ca-certificates/trust-source

It appears that Fedora does something entirely different https://fedoraproject.org/wiki/Features/SharedSystemCertificates:SubTasks
Dtrust_paths=/etc/pki/ca-trust/source:/usr/share/pki/ca-trust-source

Looking at https://github.com/p11-glue/p11-kit/commit/86e60637394340ef2fa3b3db6b451dac1d73052b#diff-12a43abb03c895e114c577296c24a7c5, https://github.com/p11-glue/p11-kit/commit/0e75a5ba8261955d4d75a38a528f79ff4edd5c21 and https://github.com/p11-glue/p11-kit/blob/e2170b295992cb7fdf115227a78028ac3780619f/trust/module.c#L407, it seems to me that the trust paths are only used as fallback when --paths flag is not used and when they are empty no token is created. For that reason adding a path should not cause any regressions.
We can leave the other distros for some other time since they had been always broken.


'Camera" object has no attribute 'original_interinsic_source'

Issue

Set up new project
connect cameras
Mode > Single Camera
Change resolution

Exception has occurred: AttributeError
'Camera' object has no attribute 'original_intrinsic_source'
  File "/Users/kyoungmin/Projects/pyxy3d-x/pyxy3d/configurator.py", line 283, in save_camera
    "original_intrinsic_source":camera.original_intrinsic_source,
  File "/Users/kyoungmin/Projects/pyxy3d-x/pyxy3d/gui/live_camera_config/camera_config_dialogue.py", line 419, in save_camera
    self.session.config.save_camera(self.camera)
  File "/Users/kyoungmin/Projects/pyxy3d-x/pyxy3d/gui/live_camera_config/camera_config_dialogue.py", line 443, in change_res_worker
    self.save_camera()
  File "/Users/kyoungmin/.pyenv/versions/3.10.13/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/kyoungmin/.pyenv/versions/3.10.13/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/Users/kyoungmin/.pyenv/versions/3.10.13/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
AttributeError: 'Camera' object has no attribute 'original_intrinsic_source'

@strx2322 ,
Thank you for raising these issues. Did you pull this down off of github or install via PyPI?
On the main repo I'm in the midst of making some major changes and will likely split the synchronized webcam recording functionality out into a separate package. The bulk of the feedback I've received up to this point has been people seeking this offline workflow with non-webcams. If you would like to work with the repo at an earlier stage, I believe that the version around mid-October should still be functional with the webcam based workflow:
https://github.com/mprib/pyxy3d/tree/f0c69821c3a744d16b5bc44bbba0f9763a9a279c
Apologies for things getting out of whack on the current Main project! I'll make a comment on the top of the README to make sure this is more transparent.


call from json from a object instance

The actual code generation generates only a factory method to create an object from the JSON.
I would like to ask if adding an object method to fill an instance from a JSON could be cool to have, it should be enabled code like the following one
Future<T> query<T extends JsonSerializable>({required QueryOptions query}) async {
    var response = await _client.query(query);
    if (response.hasException) {
      throw Exception(response.exception);
    }
    var result = newObject<T>();
    return result.fromJson(response.data!);
}

This is actually already the case! Take a look at the generated example here https://github.com/heftapp/graphql_codegen/blob/main/graphql_codegen/example/lib/main.graphql.dart#L145
For the fetch person query
query FetchPerson ($id: ID!) {
    fetch_person(id: $id) {
        ...PersonSummary
        parents {
            ...PersonSummary
        }
        children {
            ...PersonSummary
        }
    }
}

an extension is created on the GraphQL client
extension GQLExtensionQueryFetchPerson on graphql.GraphQLClient {
  Future<graphql.QueryResult> queryFetchPerson(
          GQLOptionsQueryFetchPerson options) async =>
      await this.query(options);
  graphql.ObservableQuery watchQueryFetchPerson(
          GQLWatchOptionsQueryFetchPerson options) =>
      this.watchQuery(options);
}

this means that you can call the client
client.queryFetchPerson(...)

which will serialize your response and only accept the right options/variables.

It would be nice if we didn't have to call the parsed getter, but that would require changes to the QueryResult. I have it on my todo to look into this!

It would be nice if we didn't have to call the parsed getter, but that would require changes to the QueryResult. I have it on my todo to look into this!c

Yes, if you need help on the graphql client-side just ping me, if we can work without a specific implementation of a client for each query, this library enables to autogenerate library like a library to call Github API or other, where the user generates the source code from the query and we use it to call the client.
This means that we can support different version of API

@vincenzopalazzo, I've started on a PR here https://github.com/zino-hofmann/graphql-flutter/pull/1013

Cool, I will review it as soon as can! thanks

Thanks for the review! The 0.6.0-beta.3 version supports the new parsers and you can now parse the data with
final result = await client.queryYourQuery();
final parsed = result.parsedResult;


Laravel 7 support

I added laravel 7 support in this fork if anyone wants to use.
https://github.com/bbashel1/LaravelFacebookSdk

I do not see any changes to the code in your forked version of the package, what exactly did you changed?

I do not see any changes to the code in your forked version of the package, what exactly did you changed?

Here, it`s in another branch: https://github.com/bbashel1/LaravelFacebookSdk/tree/laravel-7-support

Awesome, thank you for the fork.
How do I point to this specific fork from my composer.json?
Thus far I have:
"repositories": [
{
"type": "vcs",
"url": "https://github.com/s1rc/LaravelFacebookSdk"
}
],
"require": {
"sammyk/laravel-facebook-sdk": "dev-s1rc:30"
},
But that doesn't work. Any ideas?

@MarcoTroost you need to add the 'name' to override the origin in your repository block.
  {
    "type": "vcs",
    "name":"sammyk/laravel-facebook-sdk",
    "url": "https://github.com/s1rc/LaravelFacebookSdk"
  }],
  "require": {
    "sammyk/laravel-facebook-sdk": "dev-s1rc:30"
  }

Otherwise, it will look at the origin for sammyk's package.

@smartssa if I'll use this will there be any change in code or will anything break after this?

I cannot confirm changes to code in this branch (it's not mine) - you can check the history. I forked my own from sammyk and only changed the dependency. No other code changes were needed for my own personal usage.

Anyone looking to update this to Laravel 8?  I use it in my package and was looking at updating.

I updated my project to Laravel 8. And when upgrading, I didn't get any errors. It's strange and I It's working. Is this normal? Can someone explain me?


Tweaks to new spec for custom license stuff

Follow-up/tweak to #959

(and cc @apolinario  too)

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


User wallpaper / font colour for the menu

The menu pages are nice and clear.
But would it be possible to have a background wallpaper, a jpg on the sd card in a wallpaper folder.
I understand some menu items may become unreadable depending on the image, but that could be partly solved by a user configurable font colour (maybe even font).
Or even just 2 separate images for the left and right hand side grey vertical bars.

Good idea worth investigating.


Split models into multiple files

resolves #84

now htat we have unit tests i don't have to read this


Work Package wp-cm.csv

WP-CM-results.csv

Results of Phase 1

I would like to work on this please!

I would like to work on this please!

Results of Phase 1:
wp-cm-results.csv

wp-cm-results.csv

wp-cm_results.csv
For SENG 480B Fall 2021 A3 Part 3

wp-cm-results.csv

@RoyLQ The CSV file is not properly formatted. please remove the first row (fix the column names) and upload again.
Thanks

wp-cm-results.csv

Attached is the updated one, sorry for the mistake.

Results of Phase 1
WP-CM-results.csv

I completed this work package.

Results of Phase 1:
wp-cm-results.csv


bug/1047: Namadillo Ledger Notifications

Resolves #1047

[x] Submitting any number of signed Ledger transactions results in one Pending notification, and one Success/PartialSuccess/Error notification that clears the pending one
[x] Bump Namadillo version - this should be in next tagged release! 1.0.2 -> 1.0.3

NOTE - the major change in this PR was collecting commitments from any number of Txs, such that when we launch the notification, batch Tx & multiple-Txs can be handled the same way.
Testing
I'm testing on Housefire, which has 3 validators.

Faucet: https://faucet.knowable.run
Indexer: https://indexer.knowable.run
RPC: https://indexer.knowable.run

Txs

Test Txs with mnemonic/private key Keychain accounts (single & multiple Bond, for example)
Test single and multiple Ledger HW Wallet Tx (single & multiple Bond, with & without RevealPK NOTE If you've already revealed PK on Housefire, simply go to Add Keys, and on the Ledger connection page, click Advanced and specify a different path, thus generating a different Public Key & Address, then transfer from faucet to that address!)

Errors

You can trigger error states by overriding the feeAmount & gasLimit

Setting gasLimit too low should trigger the wrapper rejection (2nd to last screenshot below)
Setting feeAmount too low should trigger the Tx applied errors (last screenshot)

Screenshots
Example Tx sent for signing with Ledger:

Pending Ledger Tx notification (bonds summed):

Multiple Ledger-signed Txs combined into one Success message with appropriate details:

Wrapper error (rejects quickly if gas limit is incorrect)

Tx applied error if all fail

Nice! ✅
I tested only the notification with the Chrome Extension and stressed a lot it.
The unique border case I saw is:
1. I have a `Reveal + Bond` popup, and I don't sign it, but hold it opened.

2. Then, I open a second `Reveal + Bond` popup.

3. So, I sign both popups and wait for the notification

4. And, I had two pending notifications (good) but both show as failed (bad) and, in the end, the transaction was executed successfully, the reveal and both bond tx

I believe they showed the failed status because the 'second Reveal' failed and updated both notifications
But, I see this as an ultra border case, so I don't believe we need to be worried about this

@euharrison Thanks for looking into this and testing! Yeah, I think this is a difficult & unusual scenario to support. The way IDs are calculated for Toasts means that if you attempt to submit the same Tx twice (while the first approval is open) you can run into React key collisions, but also I'm not sure we should allow launching multiple approvals at once. Do you think Namadillo should prevent this behavior to begin with? I think it could be useful if Namadillo makes it clear that an approval is pending. Being able to do this doesn't seem like it should be supported to me, and I suspect it's currently like that on main too. I won't try to support that in this PR, since it's just a bugfix, but we can take a look at this!

I'm good with the current behavior. And ok as well opening more than one popup. We can handle improvements later when we got more feedbacks from users


@ConstructorBinding not refreshing

@ConstructorBinding annotated classes not refreshing
I am using Spring Cloud version Hoxton with Spring Boot 2.2.x
Spring Boot 2.2.x introduced a an annotation @ConstructorBinding that allows defining @ConfigurationProperties classes strictly through constructors, which allows using immutable classes in configurations.
While the initial configuration is being loaded from my Consul K/V server, subsequent changes to the server do not get reflected in the configuration objects. The problem is not mitigated by using @RefreshScope.

Immutable classes are just that and are not refreshable

@spencergibb I understand that the actual configuration object would not change; it is immutable. What I was expecting is that the application context would refresh, so that, for example, new HTTP requests would get a new configuration object that would reflect the new consul configuration.
The following line from the docs is what led me to believe that this would happen:

Configuration is currently read on startup of the application. Sending a HTTP POST to /refresh will cause the configuration to be reloaded. Config Watch will also automatically detect changes and reload the application context.

(https://cloud.spring.io/spring-cloud-consul/reference/html/#spring-cloud-consul-config-watch)
I tried manually refreshing the application with the /refresh endpoint, but I git the same results.

Refresh triggers configuration property rebinding and clears the refresh scope cache. So that statement isn't quite right

@spencergibb Please elaborate. Which statement are you referring to?

and reload the application context.

It doesn't actually reload the application context.


Update manifest.json to fix HA compatability issues

There are multiple HA integrations that require >= 1.7.0, your integration is currently below that and causing some errors for users.

@LaggAt

Bumping this for visibility, I'm yet another user experiencing this issue, would love to see this fixed. Thanks!

Can you please merge this PR. I want the roborock integration back. 😔

Up for this, please merge.

@Lash-L  maybe you have to assing the PR to @LaggAt , I really need that fix too. Even the solution with deleting govee did not work in our docker HA instance.

Thanks for assigning me - havn't seen the request as I do not much here lately.
Merged, will try to review the other PR's, and will release as pre-Release as I miss time for testing.
This way you could test it and if all works we could do another release later.

I see that this has been merged, and I have the latest version downloaded on my HA instance, but still having problems with Roborock. Was the included in the 2023.11.1 release or do I need another version?

I see that this has been merged, and I have the latest version downloaded on my HA instance, but still having problems with Roborock. Was the included in the 2023.11.1 release or do I need another version?

You have to manually update hacs integrations

Yes, but I think 2023.11.1 is the latest and it's already installed but I'm still having the issue

Yes, but I think 2023.11.1 is the latest and it's already installed but I'm still having the issue

See here: https://github.com/home-assistant/core/issues/102952#issuecomment-1831182292
Update and restart. If you are still having issues, you are likely having another issue and should open a new issue


Is there any plan to expose intermediate CNN outputs in hub with KerasLayer?

Currently all CNNs in Hub only provide either feature vector after global pooling or logits.
However it can be very useful to have access to the feature maps as well as retaining ability to fine-tune the CNN using KerasLayer in TF 2.0.
Is there any plan to expose intermediate layer outputs in hub with KerasLayer?

Do you have any specific module in mind @jiahuei ?

Hi there,
Mainly CNNs that are unavailable via tf.keras.applications, such as
InceptionV1 and InceptionV4.
Thanks.
Best regards,
Tan Jia Huei
University of Malaya.
On Sat, Dec 21, 2019 at 5:15 AM gowthamkpr<EMAIL_ADDRESS>wrote:

Do you have any specific module in mind @jiahuei
https://github.com/jiahuei ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/hub/issues/453?email_source=notifications&email_token=AEAKUD5V6YFQVBU76RJAB7LQZUYYNA5CNFSM4J6AV322YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHOGACY#issuecomment-568090635,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEAKUD6N6NQJXRZU7T5Y2ADQZUYYNANCNFSM4J6AV32Q
.

Hi @jiahuei, there are no current plans to expose intermediate outputs systematically.
However, there is an undocumented way to get them out of some TF2 SavedModels exported from TF-Slim, such as https://tfhub.dev/google/imagenet/inception_v1/feature_vector/4: passing return_endpoints=True to the SavedModel's __call__ function changes the output to a dict.
NOTE: This interface is subject to change or removal, and has known issues.
l = hub.KerasLayer(
    "https://tfhub.dev/google/imagenet/inception_v1/feature_vector/4",
    trainable=True,  # Or not, as you please.
    arguments=dict(return_endpoints=True))
images = tf.keras.layers.Input((224, 224, 3))
outputs = l(images)
for k, v in sorted(outputs.items()):
  print(k, v.shape)

Issues to be aware of:

Undocumented, subject to change or removal, not available consistently.
__call__ computes all outputs (and applies all update ops during training) irrespective of the ones being used later on.

jiahuei have you since been able to easily do this with models such as efficientnet? It really is annoying that we can't easily see intermediate layers. Even when I've tried to do it by utilizing tf.keras_applications models to just transfer over my weights, differences in the architectures make it nearly impossible to do even though documentation states "they are identical."

Hi, it has been a while, but I might be able to retrieve intermediate layers. Check out the code at https://github.com/jiahuei/TF2-pretrained-CNN

Hi @arnoegw! Are there any changes expected to support access to intermediate layers? I've found a lot of issues in this repository about accessing intermediate outputs for BERT and various CNN models, and this issue seems to be the only one where a solution is given. Please, consider implementing some kind of official way to use intermediate outputs, as it will significantly expand the capabilities of tfhub models.

Is this issue receiving any attention ? This is one of the most important concerns right now.

Currently, this issue is not under active development. However given that
in TF any tensor can be retrieved using tf.get_variable, one somewhat hacky
approach could be to inspect a graph (say in tensorboard.dev), find the
name of relevant tensors and read them directly
On Wed, Dec 2, 2020 at 12:09 AM Ujjwal<EMAIL_ADDRESS>wrote:

Is this issue receiving any attention ? This is one of the most important
concerns right now.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/hub/issues/453#issuecomment-736877332, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AG52OTHLIP7O5W6OFFGTTITSSVZQ7ANCNFSM4J6AV32Q
.

Hi,
in order to see your model architecture, you can try (example with efficientnet lite0):
efficientnet_lite0_base_layer = hub.KerasLayer(
    "https://tfhub.dev/tensorflow/efficientnet/lite0/feature-vector/2",
    output_shape=[1280],
    trainable=False
)

print("Thickness of the model:", len(efficientnet_lite0_base_layer.weights))
print ("{:<80} {:<20} {:<10}".format('Layer','Shape','Type'))

for i in range(len(efficientnet_lite0_base_layer.weights)):
    model_weights_raw_string = str(efficientnet_lite0_base_layer.weights[i])
    model_weights_wo_weights = model_weights_raw_string.split(", numpy", 1)[0]
    dtype = model_weights_wo_weights.split(" dtype=")[1]
    shape = model_weights_wo_weights.split(" shape=")[1].split(" dtype=")[0]

    print ("{:<80} {:<20} {:<10}".format(efficientnet_lite0_base_layer.weights[i].name, shape, dtype))

Output:
Thickness of the model: 245
Layer                                                                            Shape                Type
efficientnet-lite0/stem/conv2d/kernel:0                                          (3, 3, 3, 32)        float32
efficientnet-lite0/stem/tpu_batch_normalization/gamma:0                          (32,)                float32
efficientnet-lite0/stem/tpu_batch_normalization/beta:0                           (32,)                float32
efficientnet-lite0/stem/tpu_batch_normalization/moving_mean:0                    (32,)                float32
efficientnet-lite0/stem/tpu_batch_normalization/moving_variance:0                (32,)                float32
efficientnet-lite0/blocks_0/depthwise_conv2d/depthwise_kernel:0                  (3, 3, 32, 1)        float32
efficientnet-lite0/blocks_0/tpu_batch_normalization/gamma:0                      (32,)                float32
efficientnet-lite0/blocks_0/tpu_batch_normalization/beta:0                       (32,)                float32
efficientnet-lite0/blocks_0/tpu_batch_normalization/moving_mean:0                (32,)                float32
efficientnet-lite0/blocks_0/tpu_batch_normalization/moving_variance:0            (32,)                float32
efficientnet-lite0/blocks_0/conv2d/kernel:0                                      (1, 1, 32, 16)       float32
efficientnet-lite0/blocks_0/tpu_batch_normalization_1/gamma:0                    (16,)                float32
efficientnet-lite0/blocks_0/tpu_batch_normalization_1/beta:0                     (16,)                float32
efficientnet-lite0/blocks_0/tpu_batch_normalization_1/moving_mean:0              (16,)                float32
efficientnet-lite0/blocks_0/tpu_batch_normalization_1/moving_variance:0          (16,)                float32
.
.
.
efficientnet-lite0/blocks_15/conv2d/kernel:0                                     (1, 1, 192, 1152)    float32
efficientnet-lite0/blocks_15/tpu_batch_normalization/gamma:0                     (1152,)              float32
efficientnet-lite0/blocks_15/tpu_batch_normalization/beta:0                      (1152,)              float32
efficientnet-lite0/blocks_15/tpu_batch_normalization/moving_mean:0               (1152,)              float32
efficientnet-lite0/blocks_15/tpu_batch_normalization/moving_variance:0           (1152,)              float32
efficientnet-lite0/blocks_15/depthwise_conv2d/depthwise_kernel:0                 (3, 3, 1152, 1)      float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_1/gamma:0                   (1152,)              float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_1/beta:0                    (1152,)              float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_1/moving_mean:0             (1152,)              float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_1/moving_variance:0         (1152,)              float32
efficientnet-lite0/blocks_15/conv2d_1/kernel:0                                   (1, 1, 1152, 320)    float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_2/gamma:0                   (320,)               float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_2/beta:0                    (320,)               float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_2/moving_mean:0             (320,)               float32
efficientnet-lite0/blocks_15/tpu_batch_normalization_2/moving_variance:0         (320,)               float32
efficientnet-lite0/head/conv2d/kernel:0                                          (1, 1, 320, 1280)    float32
efficientnet-lite0/head/tpu_batch_normalization/gamma:0                          (1280,)              float32
efficientnet-lite0/head/tpu_batch_normalization/beta:0                           (1280,)              float32
efficientnet-lite0/head/tpu_batch_normalization/moving_mean:0                    (1280,)              float32
efficientnet-lite0/head/tpu_batch_normalization/moving_variance:0                (1280,)              float32

@akhorlin Hmmm..., I am wondering how I can get intermediate layer outputs from tf.Variables. Could you elaborate a bit?


Improve JS.ForEach

It would be good if JS.ForEach could drop the function call, it would shorten stack traces like this one. I think it should be doable by making JS.ForEach a macro.

This is also solvable more generally by an improvement in the optimizer. Local functions that are only used one place and it is valid to do a beta-reduction there should be inlined. This will be further be improved by the .ts output having let (block-scoped) variables, because currently the function may be needed for scoping, so beta-reduction is prevented.


hotfix for bonfire bootstrap script

Description

FIXES: The bootstrap error in bonfire that prevents from correctly importing the scripts from the new cicd-tools repository
Type of change
What is it?

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Documentation update
[ ] Tests update
[ ] Refactor

This is a temporary workaround to unblock us until a more permanent solution is provided.


EXISTS operator

is there an exists comparator for sql syntax
https://dev.mysql.com/doc/refman/5.0/en/exists-and-not-exists-subqueries.html
I'm using edges
I have " Dataset (vertex )  <-> DatasetData(edge)  <-> Data (vertex)
there is index on property "stringValue" of the Data Vertex class.
i need to search any datasets that have a data  (where stringValue like "bond%" and his context.itemDef is "Name")  and a data (where stringValue like "james%" and his context.itemDef is "Forname")
After reading responses from https://github.com/orientechnologies/orientdb/issues/4612 i cannot write sql conditions that contains a dot. (because it's not using indexes)
So i'm searching for exists clause. I don't see another solutions.

I've tried to contains but its not working with two contains.
SELECT FROM DatasetData WHERE
out contains(SELECT FROM Data WHERE stringValue = "BOND"  and context.itemDefId="NAME") AND
out contains(SELECT FROM Data WHERE stringValue = "JAMES" and context.itemDefId="FORNAME" )
GROUP BY in

Using in() method can help but i don't think that really it's the best request for performance.
Waiting the version 3 of orientDB seems to be the solution.
Select  from DataSet where inE().out in(Select from Data where stringValue LUCENE "bond" and context.itemDefId="NAME" ) AND  inE().out in(Select from Data where stringValue LUCENE "JAMES" and context.itemDefId="FORNAME" )

Better request -> contrains subquery of data to the datasetdata with key of dataset !
Select from DataSet LET $dataOdId=@rid where inE().out in(Select from Data where stringValue LUCENE "BOND" and context.itemDefId="NAME" and outE("DatasetData").in=$dataOdId  ) AND
inE().out in(Select from Data where stringValue LUCENE "GERTY" and context.itemDefId="FORNAME" and outE("DatasetData").in=$dataOdId  )

did optimization for this use case make it into v3?  we are having a similar issue to #4612 where we cannot say something like:
select Book where Author.name = xxx
because the (existing) index does not get evaluated via the dot notation. and because of the number of objects subqueries are prohibitive.


/es/newsletters/2023/02/15/

@bitschmidty involuntarily created another pull request

I assume that the new one is the one that has the latest version, right?


Feat like button

was a bit lazy with readme stuff

Thanks, I'll merge this (you have access to merge, too!) and fix that one line in the doc afterwards.. :)


feat: julia

P.S. What about reverse proxying missed files? Redirecting to the origin significantly degrades the end-user experience.

What about reverse proxying missed files?

Probably we can do that :)

You might also need to notify johnnychen94 before deploying this change. I remembered that some Julia infra relies on SJTUG and I'm not sure if this change would impact something.

@SnoozeThis wait until Friday

(https://snoozeth.is/gaURcctCiII) I will wait until Fri, 22 Dec 2023 11:57:40 UTC and then add a comment.

Resolved.

@SnoozeThis wait until tomorrow

(https://snoozeth.is/yWz2acxEgB0) I will wait until Sat, 23 Dec 2023 00:00:00 UTC and then add a comment.

Resolved.


Compile failure on ghc 7.6.3 - can't find B.unsafePackMallocCStringLen

See https://travis-ci.org/jgm/pandoc/jobs/64786014 for full log.
Preprocessing library hslua-0.4.0...
[1 of 2] Compiling Scripting.Lua.Raw ( dist/build/Scripting/Lua/Raw.hs, dist/build/Scripting/Lua/Raw.o )
[2 of 2] Compiling Scripting.Lua    ( dist/build/Scripting/Lua.hs, dist/build/Scripting/Lua.o )
src/Scripting/Lua.hsc:200:5:
    Not in scope: `B.unsafePackMallocCStringLen'
    Perhaps you meant one of these:
      `B.unsafePackMallocCString' (imported from Data.ByteString.Unsafe),
      `B.unsafePackCStringLen' (imported from Data.ByteString.Unsafe)

Sorry, the log doesn't show the bytestring version.  But I see from the git log that this function was added to bytestring on June 20, 2012.  So you probably need a bytestring >= 0.10 constraint.  Or, better, use some CPP so that hslua will still build against older versions of bytestring, which are still common.  One way or the other, could you push a fixed package? Otherwise builds that depend on hslua will fail.

We already had a bytestring >= 0.10 constraint, but apparently that's not enough because this function is added with <IP_ADDRESS>.
I updated Hackage version(without pushing a new version, just run a cabal update) and pushed fixed Cabal file to the repo.
Thanks for reporting.

Great, thanks!
+++ Ömer Sinan Ağacan [Jun 01 15 07:13 ]:

We already had a bytestring >= 0.10 constraint, but apparently that's not enough because this function is added with <IP_ADDRESS>.
I updated Hackage version(without pushing a new version, just run a cabal update) and pushed fixed Cabal file to the repo.
Thanks for reporting.

Reply to this email directly or view it on GitHub:
https://github.com/osa1/hslua/issues/28#issuecomment-107507619


Feature/new serialization system

@devhoffmann Could you take a look on the failed test? I give it a try, but might not have enough time though.


Dashed styling breaks when zoom enabled

When combining the zoom functionality with the "dashed" styling of a region the chart gets the styled X axis stuck on it and never redraws. Here is a jsfiddle showing the issue..
http://jsfiddle.net/ZpBdL/5/
This maybe related/similar to issue #1036. If you need any additional feedback please let me know.
Thanks!

+1 .. I'm having the same problem :S

:+1:

same issue, is there any fix ?
thanks

Fix is merged but no ETA for the next release.

The fix is released in v0.5.2.


save output as a file & multiple peak files check

@ypriverol PR for review

@ypriverol PR for review