Human as a tool

I see that the python repo had a new tool that is super useful, Human as a tool (https://github.com/hwchase17/langchain/tree/master/langchain/tools/human)
I'm trying to implement it myself but am wondering if this is something anyone can help with
this is what I have so far
`import { Tool } from "./base.js";

type PromptFunc = (text: string) => void;
type InputFunc = () => Promise<string>;

class HumanInput extends Tool {
  name = "Human";
  description =
    "You can ask a human for guidance when you think you got stuck or you are not sure what to do next. The input should be a question for the human.";
  promptFunc: PromptFunc;
  inputFunc: InputFunc;

  constructor(promptFunc: PromptFunc, inputFunc: InputFunc) {
    super();
    this.promptFunc = promptFunc;
    this.inputFunc = inputFunc;
  }

  async _call(query: string): Promise<string> {
    this.promptFunc(query);
    return await this.inputFunc();
  }
}

export { HumanInput };`

Came here to ask this question!

figure it out?


UI kit

Specific Demand
A components and utilities kit for Dioxus. Some specific feature ideas:

Styled components.
App settings. For example, this might include the theme.
Color palette, with a color palette designer like the material one.
Form tools. Maybe something like this (pseudocode):

enum ProductSearchSort {
    PriceAscending(String), // String here is the description of the sort
    PriceDescending(String),
    Relevance(String),
}

// We could make a custom `form_filter!` macro to make this more succint and automatically include ALL
bitflags! {
    #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
    struct ProductSearchFilter: u32 {
        const ALL = Self::COMPUTERS.bits() | Self::PHONES.bits();
        const COMPUTERS = 1 << 0;
        const PHONES = 1 << 1;
    }
}

// Form validation applied to the values
struct ProductSearchInputs<'a> {
    pub name: &'a str,
    pub max_price: u32,
    #[form_sort]
    pub sort: ProductSearchSort,
    #[form_filter]
    pub filter: ProductSearchFilter,
}

// Displayed as the default values in the form
impl<'a> Default for ProductSearchInputs<'a> {
    fn default() -> Self {
        ProductSearchInputs {
            name: "",
            max_price: u32::MAX, // Maybe hidden if MAX?
            sort: ProductSeachSort::PriceAscending("price lowest to highest"),
            filter: ProductSeachFilter::ALL,
        }
    }
}

#[component]
fn ProductSearch(cx: Scope) -> Element {
    render! {
        dxkit::Form {
            inputs: ProductSearchInputs::default(),
            instant_validation: true, // If an "invalid value" error message is shown right when typing or on submit
            oninput: move |e| { e.stop_kit_functions(); }, // e.stop_kit_functions could prevent the kit from doing anything on this input. For example, displaying a the "invalid value" message
            onsubmit: move |e| {}, // e.inputs: ProductSearchInputs
        }
    }
}

Maximize Lighthouse score. For example, a layout that has all landmark elements.
Animations. Maybe how Freya does it.
Custom types for commonly used HTML/CSS types. For example, instead of href being a string, it would be a Link, which would be constructed with a parse_str_to_link!() macro. This would enable compile-time validation of values for literals. Another example might be CSS units. So, "10px" turns into AbsoluteUnit::PX(10). This way we could implement operators and other goodies.

Implement Suggestion
I think for styling, we could use Bulma. It's purely-CSS, simple, extendable with Sass, and looks good.

Should this have a "custom" name like taffy and freya, or just something like "dxkit"? I prefer the "custom" route, unless this is going to be a core part of Dioxus, but I don't see it that way.

Should this have a "custom" name like taffy and freya, or just something like "dioxus-kit"? I prefer the "custom" route, unless this is going to be a core part of Dioxus, but I don't see it that way.

So far, anything that can be used outside of Dioxus has a custom name: Taffy (used in bevy), Blitz (used as a web renderer), plasmo (used as a standalone terminal html renderer), generational-box (Copy runtime). Fermi doesn't follow this pattern. Freya is an external library not an official Dioxus project. I think dioxus-ui, or dioxus-components would be a good name. dioxus-kit makes me think of a metaframework like svelte-kit.

"Kit" might not be ideal, but my issue with "UI" is that it's too ambiguous and "components" is perhaps too specific. This won't contain just components.

I'm on the fence about one of the potential features I mentioned, that being custom types for commonly used HTML/CSS types. I think this would be a very nice feature, but I'm wondering if it shouldn't be in dioxus-html instead.
If this UI library has an enum like CssUnit that allows calculating together a CssUnit::AbsoluteUnit or CssUnit::RelativeUnit, it would need to be converted to a string to be able to actually use it somewhere like height. That's not a problem, but arguably a better option could be having the CssUnit right in the height definition, thus there's also type safety for that attribute, since normally they're just strings. However, I don't know much about the internals of dioxus-html, so this might not be possible.

This would be awesome to incorporate with #1443 to create a global settings page!

https://github.com/matthunz/dioxus-material is a component library for dioxus based on material UI

This is being implemented in https://github.com/DioxusLabs/components


ad.concat ignores dtype of indptr and indices

For a very large dataset, we need to use int64 indexing for sparse matrix. When merging these dataset, ad.concat ignores dtype and defaults to int32
Reproducible code:
import anndata as ad, pandas as pd, numpy as np
from scipy import sparse

x = sparse.csr_matrix(np.array([[0, 1], [2, 3]]), dtype=np.float32)
x.indptr = x.indptr.astype(np.int64)
x.indices = x.indices.astype(np.int64)

a = ad.AnnData(x)
print("before", a.X.indptr.dtype)
print("before", a.X.indices.dtype)

merged = ad.concat([a, a])
print("after", merged.X.indptr.dtype)
print("after", merged.X.indices.dtype)

Output:
before int64
before int64
after int32
after int32

Yes, this is a known issue with scipy sparse matrices. See upstream discussion at:

https://github.com/scikit-learn/scikit-learn/issues/23653
https://github.com/scipy/scipy/issues/16774

I'm not sure we can work around this on our side without some changes in scipy.

So there’s nothing to do from here? Then we can just close this, no?

I don't know if we need to fix this for sparse matrices. They're gonna get downcast in so many ways, I don't think we can manage enforcing 'no downcasting' behavior.


Rust Engine write_deltalake Schema

Environment
Ubuntu 22.04, python3, reading events from Azure eventhubs and writing out to Azure blob storage.
Delta-rs version:
pip list returns version 0.15.3 for delta lake
Binding:
Environment:

loud provider: Azure
OS: Ubuntu
Other: python

Bug
What happened:
When using schema option on the write_deltalake function with the rust engine it fails to write out to delta lake on azure storage.
If i change to pyarrow engine then it works as expected.
What you expected to happen:
When using schema option on the write_deltalake function with the rust engine it writes out to delta lake on azure storage.
How to reproduce it:
Use the code sample as stated below,
Apologies for my code snippet if there are any errors.
`import pyarrow as pa
from azure.storage.blob import ContainerClient
from deltalake import write_deltalake,WriterProperties,DeltaTable,Schema,Field
from deltalake.schema import PrimitiveType
events = [{"appname":"AnthraX","geoip":{"long": 123,"lat":456},"facility":"ftp","hostname":"we.eus","message":"You're not gonna believe what just happened","msgid":"ID675","procid":5599,"severity":"notice","timestamp":"2024-02-19T12:01:40.777Z","version":1},
{"appname":"AnthraX","geoip":{"long": 123,"lat":456},"facility":"ntp","hostname":"random.rent","message":"You're not gonna believe what just happened","msgid":"ID898","procid":1335,"severity":"notice","timestamp":"2024-02-19T12:01:41.777Z","version":2},
{"appname":"shaneIxD","geoip":{"long": 123,"lat":456},"facility":"authpriv","hostname":"we.frogans","message":"There's a breach in the warp core, captain","msgid":"ID356","procid":3449,"severity":"warning","timestamp":"2024-02-19T12:01:42.778Z","version":2},
{"appname":"jesseddy","geoip":{"long": 123,"lat":456},"facility":"user","hostname":"for.is","message":"We're gonna need a bigger boat","msgid":"ID516","procid":445,"severity":"notice","timestamp":"2024-02-19T12:01:43.777Z","version":2},
{"appname":"shaneIxD","geoip":{"long": 123,"lat":456},"facility":"audit","hostname":"make.dad","message":"Great Scott! We're never gonna reach 88 mph with the flux capacitor in its current state!","msgid":"ID304","procid":2692,"severity":"err","timestamp":"2024-02-19T12:01:44.777Z","version":2},
{"appname":"b0rnc0nfused","geoip":{"long": 123,"lat":456},"facility":"local1","hostname":"up.realtor","message":"Maybe we just shouldn't use computers","msgid":"ID952","procid":8756,"severity":"emerg","timestamp":"2024-02-19T12:01:45.777Z","version":2},
{"appname":"KarimMove","geoip":{"long": 123,"lat":456},"facility":"local4","hostname":"some.store","message":"Maybe we just shouldn't use computers","msgid":"ID415","procid":9398,"severity":"crit","timestamp":"2024-02-19T12:01:46.778Z","version":2},
{"appname":"ahmadajmi","geoip":{"long": 123,"lat":456},"facility":"local1","hostname":"random.tushu","message":"Pretty pretty pretty good","msgid":"ID127","procid":8555,"severity":"info","timestamp":"2024-02-19T12:01:47.777Z","version":1},
{"appname":"b0rnc0nfused","geoip":{"long": 123,"lat":456},"facility":"alert","hostname":"names.tips","message":"There's a breach in the warp core, captain","msgid":"ID129","procid":5473,"severity":"debug","timestamp":"2024-02-19T12:01:48.777Z","version":2}
]
schema_json = '''{
"type": "struct",
"fields": [
{"name":"timestamp","type":"datetime","nullable":false,"metadata":{}},
{"name":"appname","type":"string","nullable":true,"metadata": {}},
{"name":"facility","type":"string","nullable":true,"metadata": {}},
{"name":"hostname","type":"string","nullable":true,"metadata": {}},
{"name":"message","type":"string","nullable":true,"metadata": {}},
{"name":"procid","type":"string","nullable":true,"metadata": {}},
{"name":"msgid","type":"string","nullable":true,"metadata": {}},
{"name":"severity","type":"string","nullable":true,"metadata": {}}
]
}'''
schema = Schema.from_json(schema_json )
data = pa.Table.from_pylist(events)
delta_path =<EMAIL_ADDRESS>delta_partition = ["severity"]
deltalake_writeprops = WriterProperties(compression='LZ4')
delta_storage_options = {
"AZURE_STORAGE_ACCOUNT_NAME": mystorageaccountname,
"AZURE_STORAGE_ACCOUNT_KEY": mysupersecretkey
}
write_deltalake(
delta_path,
data,
partition_by=delta_partition,
storage_options=delta_storage_options,
writer_properties=deltalake_writeprops,
schema=schema,
overwrite_schema=True,
engine="rust",
mode="append")
'''`
More details:
BACKTRACEFULL.txt

Okay i dont think this is a bug now i have unpicked it further, the timestamp field type is written to delta as an EPOCH timestamp, as such fields of type timestamp expect the time stamp field to be an integer and by default in microseconds.  I have updated my code to handle this scenario by setting any pyarrow timestamp fields to seconds using the below logic on initialisation,
for x, i in enumerate(pa_schema):
    if pa.timestamp('us').equals(i.type):
        #pa_schema.get_field_index('timestamp')
        pa_schema = pa_schema.remove(x)
        pa_schema = pa_schema.insert(x,pa.field(i.name,pa.timestamp(delta_timestamp, tz=delta_tz),nullable=i.nullable))

Then during even processing changing the timestamp field to posix time using datetime.timestamp() with below code logic
        for i in pa_schema:
            if pa.timestamp(delta_timestamp).equals(i.type):
                e[i.name] = parse(e[i.name]).timestamp()
        eventlist.append(e)

@tastyfrankfurt I do believe we are missing a case here, you passed a schema but we are not using it when writing with Rust engine, I'll try to address this

No worries, i guess the issue now for us if i have moved some data processing back into the python code base which i didn't want to do for performance reasons, anyway, i look forward to seeing what you can find.

Closing this since passing a schema is not really needed. Can always cast beforehand


Four new resources: AllergyIntolerance, Device, DiagnosticReport, and Immunization

This will round out the most interesting USCDI resources and let us do better quality metric analysis.
I'm looking at this now.

This got done and I forgot to close the issue! Whoops.


eamils

can you please add email too

Correct me if I am wrong but Google maps does not to list emails. So, only option you have is to use https://rapidapi.com/letscrape-6bRBa3QguO5/api/website-contacts-scraper/ to scrape emails from the website.

orrect me if I am wrong but Google maps does not to list emails.

i didn't know that Thank you also id odon't know how to use the API

I hope you know coding, You need to create an account on RapidApi and use their Api to get them.


Log levels don't work correctly with robot:flatten

This problems was initially reported as part of #4919. It can be reproduced, for example, with this keyword:
*** Keywords ***
Flatten with levels
    [Tags]    robot:flatten
    Log    INFO 1         INFO
    Log    DEBUG 1        DEBUG
    ${old} =    Set Log Level    DEBUG
    Log    INFO 2         INFO
    Log    DEBUG 2        DEBUG

When the above keyword is run, it produces the following messages:
15:04:58.453 	INFO 	INFO 1
15:04:58.453 	DEBUG 	DEBUG 1
15:04:58.453 	INFO 	Log level changed from TRACE to DEBUG.
15:04:58.453 	INFO 	${old} = TRACE
15:04:58.453 	INFO 	INFO 2
15:04:58.453 	DEBUG 	DEBUG 2

This is obviously wrong. The first debug message shouldn't be there and the message about the level change should show "from INFO to DEBUG".

The root cause is that we use a different XML logger when flattening than normally, and that logger has its log level set to TRACE regardless what is set from the command line. Initializing the flat logger correctly wouldn't be hard, but then we needed to add extra logic for keeping the loggers in sync in case Set Log Level is used. A much simpler alternative is changing flattening so that we have just one XML logger and that handles flattening itself. A big benefit of this approach is that it simplifies the code considerably.

Raised the priority because log levels not working properly may even disclose confidential information.


Move Up/Down context menu not available on folders, but Alt-Up/Down works

Move Up / Down is not available in the context menu when a folder is selected, however the functionality is there, so there seems no reason for removing it from the context menu.
Repro steps
Rightclick on a folder in the solution explorer:

Expected behavior
The options Move Up / Move Down should be available, similar to when you select a file:

Actual behavior
Move Up / Move Down are not available.
Known workarounds
Use the Alt + Up Arrow or Alt + Down Arrow key combinations.
Related information
The same applies to the functionality of Add Below, Add Above context menus.

Visual Studio 2015, Update 3

It looks like this is because QueryStatusOnNode is never being called on FSharpFolderNode with a command group of VSProjectConstants.guidFSharpProjectCmdSet. It is called with this command group on FSharpFileNode. I'm not sure where these commands get registered but I imagine this is the issue.

Tagging this for VS Updates; although it seems like a small enough change that we may be able to get it in for release.

I'll try to take a look at this. No guarantees though as I don't know much about the solution explorer APIs.

Interestingly, it looks like it no longer shows up at all as of <IP_ADDRESS>31702

@joshvarty that's the problem :) it may only work on folders and not files. There is a ticket but I'm on mobile so I'm struggling to find it!

@JoshVarty Shows up for me on <IP_ADDRESS>31702.  The options are greyed out, though.

Excellent work on this! The folder-support in VS2017, and adding/removing items from a folder, or moving them inside a folder or to/from a folder, including the much-anticipated copy/paste now all seem totally stable (it used to cripple the project file).


Fix over-generation of health & mana

My attempt at #117.
Adds methods to CombatData to interface with health and mana regeneration. These methods are constrained and will not over-heal/over-generate.
I'm assuming Food items won't be poisonous initially, so I've had the Food class invoke Heal() on the combat data object. If they were poisonous, perhaps there could be a bool we could use and then it can call the appropriate damaging methods on the combat data.
However, I'd like a couple set of eyes on this PR because this is my concern:
The private variables I use to track maximum health and mana are set on Awake(). I don't know how this transfers over from scene to scene.
Result: Player cannot overheal by binging on all of the delicious fishies.

@peschkaj: I tweaked the PR to include your changes. Would it be better if I actually provided methods as getters/setters to guard against setting max health to 0, etc?
see 51fa19415221fe557b898268e176581e6b99a780

Initially I was gonna be all "Unity is going to hate these" but then I realized that we can treat the initial value in the editor as the max value. There are some niche reasons why we might want this as a settable property, but that leads us down some really complex paths and let's skip that.

In direct answer to your query, I do think it would be helpful to prevent code from setting max values to <= 0. I can see some temporary effect lowering max health below current health, but that'll require some special work when we implement stuff like that.

Just to be clear, because this is approved -- should this be merged or would it be best if I added methods for adjusting max health, mana?

I'll merge it!


Enhancement: Subgroup projects searching and pagination

What does this PR do and why?
Describe in detail what your merge request does and why.
Added the following to the Group > Details > Subrgoups and Project tab:

Searching.  Search through all descendants for project and subgroups
Pagination.

Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other pull requests.

How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.

Login as user1
Go to http://localhost:3000/streptococcus
Create a new subgroup called Outbreak
Go back to http://localhost:3000/streptococcus
Search of outbreak, you should be presented with the new subgroup as well as a list of projects
Pagination should work at the bottom of this list

PR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

[X] I have evaluated the PR acceptance checklist for this PR.

This is awesome! When trying it out, I did a search, navigated to another page, then went back to search page. The search filter was still populated, but the results were not filtered. I think our sample page retains the filter and filtered results. Does this happen for you too?

Good catch on that, I believe that it is caused by "data-turbo-permanent": "true" in the search_form, which I'm guessing was set to preserve focus on search but I think the solution that @ChrisHuynh333 came up within namespace attachments might work here (See https://github.com/phac-nml/irida-next/commit/0b66dd336056d5064de901265780bece2e635991#diff-f425d445c1c9d738b4775d67d992b7bce03b95d96dbf12a404068625939c5f67R51)

This is awesome! When trying it out, I did a search, navigated to another page, then went back to search page. The search filter was still populated, but the results were not filtered. I think our sample page retains the filter and filtered results. Does this happen for you too?

Thanks @ksierks that is a good catch.  It should be working as expected in 8bdcf3f0b45255488acccec3bb2ea66c33a551f6


Update nan to ^1.8 to support io.js 2.0

Fix #68
R= @bnoordhuis  or @es128

I am intentionally switching to ^ carrot operator in order to track newer nan releases and thus get support for future io.js releases for free.

LGTM. Just re-running the Travis instance that timed out to make sure prior to merge.

Released as<EMAIL_ADDRESS>


bug: routerLink forwards does not unmount origin component

Bug Report
Ionic version:
[x] 4.11.5
Current behavior:

Using routerLink on for example IonCard with routerDirection="forward" does not unmount the origin component, thus not allowing clean up effect to take place.
Expected behavior:

When using router to navigate between pages I expect the page containing the link to be unmounted.
Steps to reproduce:

Create a Ionic/react starter project with sidemenu and add the following card to the Home component:
         <IonCard routerLink={"home/list"} routerDirection="forward">
            <IonCardHeader>
              <IonCardSubtitle>SUBHEADER</IonCardSubtitle>
              <IonCardTitle>CARD TITLE</IonCardTitle>
            </IonCardHeader>
            <IonCardContent>

CARD BODY HELLOOOOOOO

            </IonCardContent>
          </IonCard>
        </IonCard>

Then navigate back and forth using the side menu and the routerLink on the component.

Apparently IonRouterOutlet is keeping the state intact.


How to handle the toml-rb gem and TOML files in general

Use Case
currently this module installs the toml-rb gem, but the user can not decide

if he or she wants the module to do this (manage toml_rb_gem switch)
the version
needs a proxy or an other source for the gem

Describe the Solution You Would Like
One option would be to patch this module and the influxdb module it uses to do this
An other option would be to use to_toml() function from stdlib, and get rid of the dependency: https://github.com/puppetlabs/puppetlabs-stdlib/blob/main/REFERENCE.md#to_toml  (I would prefer this)
Describe Alternatives You've Considered
none
Additional Context
none

Unfortunately stdlib doesn't have a from_toml function equivalent to toml-rb's load function, which we need for reading the config files from disk and comparing to the desired state.  I know installing gems isn't the best option and we'd like to avoid it when possible, but I don't see a way around it currently.
The version can be specified in influxdb::profile::toml, but like you said it's not great when you have a proxy or airgap.
I guess if you are ok with not using SSL for InfluxDB and using Bolt, you could use the provision dashboard plan.  That sort of gets around the gem issue because you can install it locally with Bolt, so it doesn't have to be installed on the target machine.
/opt/puppetlabs/bolt/bin/gem install --user-install toml-rb

I'll leave this open for now so people can leave comments and suggestions, but I think toml-rb will probably remain a requirement.

the missing from_toml (stdlib has load$DATAFORMAT) and parse$DATAFORMAT for this case. I will have a look at these and figure out if these are a good guideline for a TOML implementation.

https://github.com/voxpupuli/puppet-telegraf/pull/207 for telegraf we're switching to the stdlib implementation to get rid of toml.

#219  Added option to specify yaml instead of toml, removing the dependency on the toml gem completely

@zoojar thanks for the PR!


RFC Indices and finer grained events

Here is my take on the data structures:

the file in this PR has a list of maps and back pointers to owning objects (I didn't care much about idiomatic go). I have also added conditions during which endpoints/clusters/routes/listeners will change with respect to the service registry
The focus is on cutting down redundant computation and trimming memory usage. So the tradeoff here has been to dynamically compose the xDS response using proto bits from the hash map, instead of caching the entire proto/json response to a single endpoint's XDS call.
With this fine grained information, we could have a more cleaner invalidation or update system
(i.e. in the AppendServiceHandler (which responds to changes in services), and in AppendInstanceHandler (which responds to changes in endpoints))
Instead of blowing up the entire cache as done here https://github.com/istio/istio/blob/master/pilot/pkg/proxy/envoy/v1/discovery.go#L330-L337
(out.clearcache()), we could use the information to selectively update the right parts (like listeners/clusters/etc/). Essentially, there is no cache. The cache is the hashmaps that we have. And the cache invalidation is updating parts of the hash map. We get to reuse the code from serviceregistry/model, etc.

Thoughts? (@ZackButcher / @costinm / @kyessenov / @ijsnellf / @GregHanson / @rosenhouse ) ?

One factor to consider is that in v2 we'll also have a map of active envoy configs, with the pending response. Since it's long poll, we can update only the structs we need, and it acts as a cache ( in particular for streaming grpc)

@costinm is there a design doc for that? What does "v2" refer here and what is this long poll?

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
89158f4df12378610989c22678a2c50efa8e0b43
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Design doc for what ? Long poll is what Envoy is doing in v2/REST
(apparently for both gRPC fetch and json), pilot keeps request waiting
until there is
an update.
For the data structures - I think the comments are in effect the design, no
need to copy them to a doc that will be out of date.
All 'long poll' implementations I know in go have a map of 'active'
requests and push to them.
On Fri, Feb 9, 2018 at 4:12 PM, Kuat<EMAIL_ADDRESS>wrote:

@costinm https://github.com/costinm is there a design doc for that?
What does "v2" refer here and what is this long poll?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/3378#issuecomment-364605710, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFI6skTZ6o5B2R3_zcYl7_7mUiIpXyMks5tTN79gaJpZM4SAqwA
.

https://en.wikipedia.org/wiki/Push_technology - AFAIK envoy json is pretty
good match for what is typically called long polling.
XDS_PROTOCOL.md uses the same term:
"Synchronous (long) polling via REST endpoints is also available for the
xDS singleton APIs"
On Fri, Feb 9, 2018 at 6:20 PM, Costin Manolache<EMAIL_ADDRESS>wrote:

Design doc for what ? Long poll is what Envoy is doing in v2/REST
(apparently for both gRPC fetch and json), pilot keeps request waiting
until there is
an update.
For the data structures - I think the comments are in effect the design,
no need to copy them to a doc that will be out of date.
All 'long poll' implementations I know in go have a map of 'active'
requests and push to them.
On Fri, Feb 9, 2018 at 4:12 PM, Kuat<EMAIL_ADDRESS>wrote:

@costinm https://github.com/costinm is there a design doc for that?
What does "v2" refer here and what is this long poll?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/3378#issuecomment-364605710, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFI6skTZ6o5B2R3_zcYl7_7mUiIpXyMks5tTN79gaJpZM4SAqwA
.

Well, you probably need a mutex for the big struct for sure.
Too fine-grained muxtexes may seem as an optimization - but sometimes they
hurt more then help ( if you consider low-level cpu behavior).
Agree we should ignore mutexes for now - once we have implementation and
bench tests we can evaluate.
On Fri, Feb 9, 2018 at 6:27 PM, Shriram Rajagopalan <
<EMAIL_ADDRESS>wrote:

@rshriram commented on this pull request.
In pilot/pkg/proxy/envoy/v2/datastructures.go
https://github.com/istio/istio/pull/3378#discussion_r167385882:

+// BoxedCluster holds the cluster proto representation, pointers to endpoints
+type BoxedCluster struct {

// The proto representation of this cluster
protoCluster *xdsapi.Cluster
// list of endpoints associated with this cluster
endpoints []*BoxedEndpoint
// List of labels associated with this cluster
labels []*model.Labels
+}

+// Indices is a set of reverse indices to quickly locate endpoints, clusters, routes
+// and listeners
+type Indices struct {

// Mutex guards guarantees consistency of updates to members shared across threads.
mu sync.RWMutex

Ignore the mutex please. It was a copy paste thing from an old struct
definition.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/3378#discussion_r167385882, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFI6nHM8-6yH7AnCzw-IG-ebvpOm75qks5tTP6FgaJpZM4SAqwA
.

@costinm I am familiar what the long poll is, I was merely referring to your authoritative statement: in v2 we'll also have a map of active envoy configs, with the pending response, which seems like 1) some decision has been made to use a long poll, 2) there is some plan to do that already.

@costinm Thanks for explaining. Yes, you are right, even ADS is essentially a long poll over gRPC bi-di in envoy. I was confused since I wasn't sure whether you are referring to changing legacy poll to long poll or to bi-di long poll. In any case, let's distract from the topic of this PR, which is only tangentially related to config transport mechanism.

@rshriram I find it easier to reason about efficiency of a data structure if we know the interface that the data structure implements. The indexes should be designed around the externally facing operations. Worst comes to worst, we could take an off-the-shelf database to optimize hashmaps, but the interface operations are a lot harder to change.

Long poll is closely related - it implies pilot will have a map with all
active sidecars that connect to this pilot, which can link to those structs

and back. It also has implications on caching - and on copy. Once a route
proto is serialized, it can be sent with 0 copy to all connected envoys,
and may not need to be saved.

It is important to be aware of the larger picture...
On Feb 10, 2018 13:05, "Kuat"<EMAIL_ADDRESS>wrote:

@costinm https://github.com/costinm Thanks for explaining. Yes, you are
right, even ADS is essentially a long poll over gRPC bi-di in envoy. I was
confused since I wasn't sure whether you are referring to changing legacy
poll to long poll or to bi-di long poll. In any case, let's distract from
the topic of this PR, which is only tangentially related to config
transport mechanism.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/3378#issuecomment-364692012, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFI6h_OF1jylSU5YFt_X4TzJmniJtCLks5tTgSNgaJpZM4SAqwA
.

Kuat - interfaces and abstractions have a role, but this PR is about
efficient data structures and minimal memory,  O(1) operations, and as
direct as possible implementation of the envoy interface.
On Feb 10, 2018 13:07, "Kuat"<EMAIL_ADDRESS>wrote:

@rshriram https://github.com/rshriram I find it easier to reason about
efficiency of a data structure if we know the interface that the data
structure implements. The indexes should be designed around the externally
facing operations. Worst comes to worst, we could take an off-the-shelf
database to optimize hashmaps, but the interface operations are a lot
harder to change.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/3378#issuecomment-364692173, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAFI6uMMQOGStHAkKsivejmUEewB14cCks5tTgUfgaJpZM4SAqwA
.

@costinm

One factor to consider is that in v2 we'll also have a map of active envoy configs, with the pending response. Since it's long poll, we can update only the structs we need, and it acts as a cache ( in particular for streaming grpc)

We have this sorted out already (or atleast a PoC) in go-control-plane, where we have the concept of a snapshot (an envoy config). You can set the snapshot in one operation and then the ADS server will take care of sequencing the updates to the consumer envoy.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
89158f4df12378610989c22678a2c50efa8e0b43
link
/test istio-pilot-e2e

prow/istio-presubmit.sh
72e6b790d79ab0dfb99ff93f800be91f6d914105
link
/test istio-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
89158f4df12378610989c22678a2c50efa8e0b43
link
/test istio-pilot-e2e

prow/istio-presubmit.sh
72e6b790d79ab0dfb99ff93f800be91f6d914105
link
/test istio-presubmit

prow/istio-unit-tests.sh
72e6b790d79ab0dfb99ff93f800be91f6d914105
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

updates:

Some finer grained locks
Added a rough sketch of how I envision this to work (by plumbing this into the model events and the ADS server). The proto struct copying is very bad, and I could use some help in fixing the go cp/finding work arounds to avoiding this copy
The ADS server could be very easily replaced by a REST server as well, and the code would not change much.

Essentially, we construct listeners/clusters/routes once when something changes, and ship the same stuff to every endpoint (we have to fine tune this logic to trim clusters and routes based on source match).
@kyessenov I believe this answers the interface questions. The stuff in go-cp could use a serious renaming pass, which I will get to soon.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
89158f4df12378610989c22678a2c50efa8e0b43
link
/test istio-pilot-e2e

prow/istio-presubmit.sh
72e6b790d79ab0dfb99ff93f800be91f6d914105
link
/test istio-presubmit

prow/istio-unit-tests.sh
8fb64365510c7c35cf82b6ddc7aea9b8f6e6502e
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
89158f4df12378610989c22678a2c50efa8e0b43
link
/test istio-pilot-e2e

prow/istio-unit-tests.sh
8fb64365510c7c35cf82b6ddc7aea9b8f6e6502e
link
/test istio-unit-tests

prow/istio-presubmit.sh
8fb64365510c7c35cf82b6ddc7aea9b8f6e6502e
link
/test istio-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
89158f4df12378610989c22678a2c50efa8e0b43
link
/test istio-pilot-e2e

prow/istio-unit-tests.sh
8fb64365510c7c35cf82b6ddc7aea9b8f6e6502e
link
/test istio-unit-tests

prow/istio-presubmit.sh
2f54c4db62d2c78718f6dce753b4f0a868952320
link
/test istio-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
89158f4df12378610989c22678a2c50efa8e0b43
link
/test istio-pilot-e2e

prow/istio-presubmit.sh
2f54c4db62d2c78718f6dce753b4f0a868952320
link
/test istio-presubmit

prow/istio-unit-tests.sh
2f54c4db62d2c78718f6dce753b4f0a868952320
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

With regard to the existing mesh.go - I find it to be a mix of functionalities for multi-zone mesh and pilot optimizations. I would like to focus solely on pilot optimizations and keep the multi-zone mesh separate. Mixing both leads to complex code.
It is also duplicating quite a bit of functionality that already exists (e.g., config change events, new platform registry interfaces, etc.).

The existing config watcher system (in model) and the set of event handlers has been tried and tested for a year, while the one in mesh.go is similar yet limited (it is not handling route rules, external services, gateways, etc.). I am unclear on the need to reinvent that piece.
I am unsure of the need to rewrite all of pilot registry adapters. Things like this:

// NewEndpoint is a boiler-plate function intended for environment specific registries to create a new Endpoint.
func NewEndpoint(address string, port uint32, socketProtocol SocketProtocol, labels []EndpointLabel) (*Endpoint, error) {

essentially ignores all the platform adapters that we have been using/testing for a year. It will require all platform adapters to be rebuilt causing more code churn than necessary, and in the end we will be deploying even more code that is less tested.
The memory/perf gains are from the data structures and the code here is a super set of the data structures in mesh.go, with lesser number of hash map lookups (aka more tlb friendly).
This code also uses the ADS server that we (istio team) asked the rest of the community to use (in go-control-plane). There are some gaps there (such as those unnecessary dispatches that you (costin) pointed out), which need to be fixed, but I am sure @kyessenov would be happy to squeeze every last ounce of performance from that library, which would benefit other users of that library as well.
I think if we add a REST server to that library, our code would be simpler.

@rshriram would you mind responding on this thread re next steps for multicloud: https://groups.google.com/forum/#!topic/istio-environments/3SZaZGhPk0U
I want to get moving on multicloud.  The current churn is typical, but would be helpful to have a path forward.
Cheers
-steve

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
527d4cf253a9785770f20a27b89df52f665b1673
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
6a7bd2afa6afc89da2f62665a25aca5f41a64a87
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

I don't think this is solely about performance. This is about the system design and how to structure the code, with comments that clearly explain the control/data flow, and not reinventing the wheel. It actually shows how to leverage most parts of pilot in the registry while reworking the underlying data structures.

So for the benefit of @sdake and myself, could you please let us know how this integrates with v2/mesh.go?

On Mon, Feb 12, 2018 at 5:02 PM Gautam Nirodi<EMAIL_ADDRESS>wrote:

So for the benefit of @sdake https://github.com/sdake and myself, could
you please let us know how this integrates with v2/mesh.go?

Mesh.go (multi cloud) is rapidly evolving and will undergo simplifications
over time.  However, I am unwilling to let that complexity and evolution
permeate all of Pilot, as it seems to be doing today.
This code is meant to cleanly separate what is a v2 implementation from
what is a multicloud thing. I am trying to keep large parts of pilot code
intact in an attempt to reach 1.0. Reimplementing the platform registry
interfaces, config event system etc as done by mesh.go seems
counterintuitive to reaching that goal.
The crux of the problem is the act of clearing the caches when platform
registry changes. That can be handled more easily as I have shown here. The
label handling can be optimized separately as well.
—

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/3378#issuecomment-365077911, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AH0qd-KB2jy6HQMLeZVfTBOSSVEGtWrzks5tULUHgaJpZM4SAqwA
.

closing as it does not align with work on multicloud.

@rshriram: I am trying to get a handle on this proposal. I can see that it makes a lot of sense.
A few questions: code in proxy/envoy/v1 will not be much of use for v2 with roughly the same design as in this PR, is that a fair assessment? I can imagine some changes in other parts of pilot to support this design as well, do you agree?
If I understand correctly, We'll end up having these caches in pilot:
C0: caches possibly maintained by the underlying api server client (such as k8s go-client)
C1: caches maintained in individual service registries as what exists today.
C2: caches as proposed in this PR: endpoints, clusters, routes, listeners that are lined up with envoy V2 quite well.
C3: snapshots (Corresponds to the existing cached responses in v1) tailored to envoy v2
I can imagine with this proposal, it can cut down the duplicate processing in building listeners, clusters, routes, etc that exist in proxy/envoy/v1 code today. By duplicate processing, I mean the logic that will be run for all the first requests (or when changes take place in platform registries as you pointed out in your above comments) although it'd result in the same results. And if done right, it can make it quite efficient. Some optimization in C1 might be useful. E.g., the big loop in GetSidecarServiceInstances(). But it's implementation details in terms of how C2 interacts with C1 for this design, or the methods required of C1 for V2 might be different from what's there. And it  might even be possible eventually C1 could be optimized out for V2.
To complete this design, details might need to be fleshed out as to how to build responses for xDS requests that come from individual envoy proxies. I can see that the majority parts of the responses have already been built in cache C2. What remains to be built is the parts that are specific to the sidecar that the requests are coming from, which is then combined with what's in the Cache C2, to have the complete response built. Let me know if this understanding is correct.
.

FYI I have decided to close this PR as it does not align with the work that @gnirodi is doing on the multi-cloud (mesh.go).
Answers here are just for posterity.

A few questions: code in proxy/envoy/v1 will not be much of use for v2 with roughly the same design as in this PR, is that a fair assessment?

Not exactly. The idea is to reuse the control flow in proxy/envoy/v1 (all the function call chain as is, and simply replace filling in the json structs with filling in proto structs).

I can imagine some changes in other parts of pilot to support this design as well, do you agree?

The idea behind this design is to keep code in pilot/pkg/serviceregistry/ stable, as well as code in pilot/pkg/model/ . The only code that will change is the code in pilot/pkg/envoy/v2/, and tests. The golden files will still be there (with different json schemas (v2 schema based on proto) )

If I understand correctly, We'll end up having these caches in pilot:
C0: caches possibly maintained by the underlying api server client (such as k8s go-client)

This is k8s client specific. Consul/eureka does not have C0

C1: caches maintained in individual service registries as what exists today.

We don't have caches in the service registries. The interfaces defined in model will remain as it is.

C2: caches as proposed in this PR: endpoints, clusters, routes, listeners that are lined up with envoy V2 quite well.

This is actually the only cache we have. In earlier model, we had per proxy response cache (that cached the entire JSON). This model has a shared cache that is finer grained.

C3: snapshots (Corresponds to the existing cached responses in v1) tailored to envoy v2

Yes. And this is actually just an array of pointers to the protos that are stored in C2. So its not much of a cache per se.

I can imagine with this proposal, it can cut down the duplicate processing in building listeners, clusters, routes, etc that exist in proxy/envoy/v1 code today. By duplicate processing, I mean the logic that will be run for all the first requests (or when changes take place in platform registries as you pointed out in your above comments) although it'd result in the same results.

Good observation. I am very happy to see that you caught on to this concept!

And if done right, it can make it quite efficient. Some optimization in C1 might be useful. E.g., the big loop in GetSidecarServiceInstances(). But it's implementation details in terms of how C2 interacts  with C1 for this design, or the methods required of C1 for V2 might be different from what's there. And it might even be possible eventually C1 could be optimized out for V2.

As I stated above, my intent is not to rewrite all of pilot, if we are shooting for a stable 1.0. The interfaces in C1 (platform registries) will stay as it is today. It is very hard to populate three level deep envoy proto struct with information from the platform registries. We could revisit this at some point, once we get the v2 xDS servers stable.

To complete this design, details might need to be fleshed out as to how to build responses for xDS requests that come from individual envoy proxies. I can see that the majority parts of the responses have already been built in cache C2. What remains to be built is the parts that are specific to the sidecar that the requests are coming from, which is then combined with what's in the Cache C2, to have the complete response built. Let me know if this understanding is correct.

Yep. Spot on. Thats the only part that needs to be worked out. Once we build the responses, the snapshots will contain a list of pointers to the right proto structs for a given envoy.

+istio-environments
On Tue, Feb 13, 2018 at 8:29 AM, Shriram Rajagopalan <
<EMAIL_ADDRESS>wrote:

FYI I have decided to close this PR as it does not align with the work
that @gnirodi https://github.com/gnirodi is doing on the multi-cloud
(mesh.go).

I still believe it is important for @gnirodi and other people working on
multicloud to check this PR and incorporate
parts of it into the new data structures, and I would expect optimizations
on the routing side (v1alpha2) will
use something similar.

Answers here are just for posterity.
A few questions: code in proxy/envoy/v1 will not be much of use for v2
with roughly the same design as in this PR, is that a fair assessment?
Not exactly. The idea is to reuse the control flow in proxy/envoy/v1 (all
the function call chain as is, and simply replace filling in the json
structs with filling in proto structs).

The short term plan - yes, of course, we need to iterate and get something
working so it can be fine tuned.
Longer term, and based on scale test results - we may replace parts of the
chain related to endpoints.
And as LDS/RDS/CDS get moved to v2 - I expect similar optimizations and
cleanups to be done, but
also gradually and safely.

I can imagine some changes in other parts of pilot to support this design
as well, do you agree?
The idea behind this design is to keep code in pilot/pkg/serviceregistry/
stable, as well as code in pilot/pkg/model/ . The only code that will
change is the code in pilot/pkg/envoy/v2/, and tests. The golden files will
still be there (with different json schemas (v2 schema based on proto) )
If I understand correctly, We'll end up having these caches in pilot:
C0: caches possibly maintained by the underlying api server client (such
as k8s go-client)
This is k8s client specific. Consul/eureka does not have C0

One of the proposals I had was to explore the lighter k8s client, which is
also no using C0
and appears better suited for us. A good (and safe) start would be to
explore its use for
fetching workloads/endpoints - as an alternative path to fill in the
multi-cloud structures.

C1: caches maintained in individual service registries as what exists
today.
We don't have caches in the service registries. The interfaces defined in
model will remain as it is.

C2: caches as proposed in this PR: endpoints, clusters, routes, listeners
that are lined up with envoy V2 quite well.
This is actually the only cache we have. In earlier model, we had per
proxy response cache (that cached the entire JSON). This model has a shared
cache that is finer grained.

Well - the v1 implementation has C3, in the new implementation we'll only
have C2.

C3: snapshots (Corresponds to the existing cached responses in v1)
tailored to envoy v2
Yes. And this is actually just an array of pointers to the protos that are
stored in C2. So its not much of a cache per se.
I can imagine with this proposal, it can cut down the duplicate processing
in building listeners, clusters, routes, etc that exist in proxy/envoy/v1
code today. By duplicate processing, I mean the logic that will be run for
all the first requests (or when changes take place in platform registries
as you pointed out in your above comments) although it'd result in the same
results.
Good observation. I am very happy to see that you caught on to this
concept!

I think it should go a bit further: since v2 is a push API, this logic
should also be able to process events ( endpoint
added/changed ) and directly send it to sidecars, without waiting for a
poll.
The pilot will have a map of long-polling sidecars, and one of the reasons
for switch RDS/LDS/CDS to v2 is to
propagate changes immediately (or gradually under server control), instead
of each sidecar polling every
few seconds. That's not in scope for multi-cloud, but will need to be done
in istio.

And if done right, it can make it quite efficient. Some optimization in C1
might be useful. E.g., the big loop in GetSidecarServiceInstances(). But
it's implementation details in terms of how C2 interacts with C1 for this
design, or the methods required of C1 for V2 might be different from what's
there. And it might even be possible eventually C1 could be optimized out
for V2.
As I stated above, my intent is not to rewrite all of pilot, if we are
shooting for a stable 1.0. The interfaces in C1 (platform registries) will
stay as it is today. It is very hard to populate three level deep envoy
proto struct with information from the platform registries. We could
revisit this at some point, once we get the v2 xDS servers stable.

+1 - we want to iterate and keep things stable.
It is possible to have an alternate 'direct path' for endpoints, for
example using the lighter k8s client to
directly fill the endpoint structs - but with a flag and as an option, with
the default as the stable (current)
version until the new code has similar stability.
To complete this design, details might need to be fleshed out as to how to

build responses for xDS requests that come from individual envoy proxies. I
can see that the majority parts of the responses have already been built in
cache C2. What remains to be built is the parts that are specific to the
sidecar that the requests are coming from, which is then combined with
what's in the Cache C2, to have the complete response built. Let me know if
this understanding is correct.
Yep. Spot on. Thats the only part that needs to be worked out. Once we
build the responses, the snapshots will contain a list of pointers to the
right proto structs for a given envoy.

https://kubernetes.io/docs/reference/client-libraries/ ->
github.com/ericchiang/k8s
I did a quick test - it is just a generated wrapper on top of the
libraries, with minimal 'opinions'.
Costin
On Tue, Feb 13, 2018 at 11:40 AM, 'Andra Cismaru' via Istio Environments <
<EMAIL_ADDRESS>wrote:

Shriram, can you send a pointer to the lighter k8s client you are talking
about?
On Tue, Feb 13, 2018 at 8:52 AM, Costin Manolache<EMAIL_ADDRESS>wrote:

+istio-environments
On Tue, Feb 13, 2018 at 8:29 AM, Shriram Rajagopalan <
<EMAIL_ADDRESS>wrote:

FYI I have decided to close this PR as it does not align with the work
that @gnirodi https://github.com/gnirodi is doing on the multi-cloud
(mesh.go).

I still believe it is important for @gnirodi and other people working on
multicloud to check this PR and incorporate
parts of it into the new data structures, and I would expect
optimizations on the routing side (v1alpha2) will
use something similar.

Answers here are just for posterity.
A few questions: code in proxy/envoy/v1 will not be much of use for v2
with roughly the same design as in this PR, is that a fair assessment?
Not exactly. The idea is to reuse the control flow in proxy/envoy/v1
(all the function call chain as is, and simply replace filling in the json
structs with filling in proto structs).

The short term plan - yes, of course, we need to iterate and get
something working so it can be fine tuned.
Longer term, and based on scale test results - we may replace parts of
the chain related to endpoints.
And as LDS/RDS/CDS get moved to v2 - I expect similar optimizations and
cleanups to be done, but
also gradually and safely.

I can imagine some changes in other parts of pilot to support this
design as well, do you agree?
The idea behind this design is to keep code in
pilot/pkg/serviceregistry/ stable, as well as code in pilot/pkg/model/ .
The only code that will change is the code in pilot/pkg/envoy/v2/, and
tests. The golden files will still be there (with different json schemas
(v2 schema based on proto) )
If I understand correctly, We'll end up having these caches in pilot:
C0: caches possibly maintained by the underlying api server client (such
as k8s go-client)
This is k8s client specific. Consul/eureka does not have C0

One of the proposals I had was to explore the lighter k8s client, which
is also no using C0
and appears better suited for us. A good (and safe) start would be to
explore its use for
fetching workloads/endpoints - as an alternative path to fill in the
multi-cloud structures.

C1: caches maintained in individual service registries as what exists
today.
We don't have caches in the service registries. The interfaces defined
in model will remain as it is.

C2: caches as proposed in this PR: endpoints, clusters, routes,
listeners that are lined up with envoy V2 quite well.
This is actually the only cache we have. In earlier model, we had per
proxy response cache (that cached the entire JSON). This model has a shared
cache that is finer grained.

Well - the v1 implementation has C3, in the new implementation we'll only
have C2.

C3: snapshots (Corresponds to the existing cached responses in v1)
tailored to envoy v2
Yes. And this is actually just an array of pointers to the protos that
are stored in C2. So its not much of a cache per se.
I can imagine with this proposal, it can cut down the duplicate
processing in building listeners, clusters, routes, etc that exist in
proxy/envoy/v1 code today. By duplicate processing, I mean the logic that
will be run for all the first requests (or when changes take place in
platform registries as you pointed out in your above comments) although
it'd result in the same results.
Good observation. I am very happy to see that you caught on to this
concept!

I think it should go a bit further: since v2 is a push API, this logic
should also be able to process events ( endpoint
added/changed ) and directly send it to sidecars, without waiting for a
poll.
The pilot will have a map of long-polling sidecars, and one of the
reasons for switch RDS/LDS/CDS to v2 is to
propagate changes immediately (or gradually under server control),
instead of each sidecar polling every
few seconds. That's not in scope for multi-cloud, but will need to be
done in istio.

And if done right, it can make it quite efficient. Some optimization in
C1 might be useful. E.g., the big loop in GetSidecarServiceInstances(). But
it's implementation details in terms of how C2 interacts with C1 for this
design, or the methods required of C1 for V2 might be different from what's
there. And it might even be possible eventually C1 could be optimized out
for V2.
As I stated above, my intent is not to rewrite all of pilot, if we are
shooting for a stable 1.0. The interfaces in C1 (platform registries) will
stay as it is today. It is very hard to populate three level deep envoy
proto struct with information from the platform registries. We could
revisit this at some point, once we get the v2 xDS servers stable.

+1 - we want to iterate and keep things stable.
It is possible to have an alternate 'direct path' for endpoints, for
example using the lighter k8s client to
directly fill the endpoint structs - but with a flag and as an option,
with the default as the stable (current)
version until the new code has similar stability.
To complete this design, details might need to be fleshed out as to how

to build responses for xDS requests that come from individual envoy
proxies. I can see that the majority parts of the responses have already
been built in cache C2. What remains to be built is the parts that are
specific to the sidecar that the requests are coming from, which is then
combined with what's in the Cache C2, to have the complete response built.
Let me know if this understanding is correct.
Yep. Spot on. Thats the only part that needs to be worked out. Once we
build the responses, the snapshots will contain a list of pointers to the
right proto structs for a given envoy.
--
You received this message because you are subscribed to the Google Groups
"Istio Environments" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to<EMAIL_ADDRESS>To post to this group, send email to<EMAIL_ADDRESS>To view this discussion on the web visit https://groups.google.com/d/ms
gid/istio-environments/CAP8-Fqmg%2BxfxneV57K2HGc1Gci889940-
Fa%3D34xy%2BHN6mTLHYg%40mail.gmail.com
https://groups.google.com/d/msgid/istio-environments/CAP8-Fqmg%2BxfxneV57K2HGc1Gci889940-Fa%3D34xy%2BHN6mTLHYg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"Istio Environments" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to<EMAIL_ADDRESS>To post to this group, send email to<EMAIL_ADDRESS>To view this discussion on the web visit https://groups.google.com/d/
msgid/istio-environments/CAGN0SZXtNB5coto4RQ5xju-cDKx%
3DXyMJZhiZ3jQ7gH8%2Bxksz%3DA%40mail.gmail.com
https://groups.google.com/d/msgid/istio-environments/CAGN0SZXtNB5coto4RQ5xju-cDKx%3DXyMJZhiZ3jQ7gH8%2Bxksz%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

From looking at its GitHub page, it's not clear to me whether or not it supports changes of services and endpoints in the cluster.

it does support watch.

It is possible to have an alternate 'direct path' for endpoints, for example using the lighter k8s client to directly fill the endpoint structs - but with a flag and as an option, with the default as the stable (current) version until the new code has similar stability.

What problem is this trying to solve?
FWIW, the recommendation from https://github.com/ericchiang/k8s#should-i-use-this-or-client-go says:

TL;DR - Use client-go if you're writing a controller.

On Tue, Feb 13, 2018 at 1:02 PM, Jason Young<EMAIL_ADDRESS>wrote:

It is possible to have an alternate 'direct path' for endpoints, for
example using the lighter k8s client to directly fill the endpoint structs

but with a flag and as an option, with the default as the stable
(current) version until the new code has similar stability.

What problem is this trying to solve?

Lower memory overhead. In multi-cluster the goal is to scale to 10s of
clusters, each with max number of endpoints.

FWIW, the recommendation from https://github.com/ericchiang/
k8s#should-i-use-this-or-client-go says:
TL;DR - Use client-go if you're writing a controller.

We are not writing a controller.
Istio does have an ingress controller - and it should keep using client-go

but for most of the code fetching data ( endpoints in particular )
the client-go caching and allocations may be a problem. Right now we are
seeing a pretty large GC overhead with pilot, that's one
of the things to explore - of course we're at early stage, just trying to
get heap dumps and find simpler solutions.

Costin

And to clarify: I am not proposing we switch the client.
Just to evaluate if it's even an option, and wait until we have more real
data on where the memory is spent.
There is another use case for the library - on agent side - since it has
fewer dependencies and lower footprint,
but I hope we can continue to avoid a dependency ( loading SNI certs
dynamically was one case for agent using
k8s API)
Costin
On Tue, Feb 13, 2018 at 1:49 PM, Costin Manolache<EMAIL_ADDRESS>wrote:

On Tue, Feb 13, 2018 at 1:02 PM, Jason Young<EMAIL_ADDRESS>wrote:

It is possible to have an alternate 'direct path' for endpoints, for
example using the lighter k8s client to directly fill the endpoint structs

but with a flag and as an option, with the default as the stable
(current) version until the new code has similar stability.

What problem is this trying to solve?

Lower memory overhead. In multi-cluster the goal is to scale to 10s of
clusters, each with max number of endpoints.

FWIW, the recommendation from https://github.com/ericchiang/
k8s#should-i-use-this-or-client-go says:
TL;DR - Use client-go if you're writing a controller.

We are not writing a controller.
Istio does have an ingress controller - and it should keep using client-go

but for most of the code fetching data ( endpoints in particular )
the client-go caching and allocations may be a problem. Right now we are
seeing a pretty large GC overhead with pilot, that's one
of the things to explore - of course we're at early stage, just trying to
get heap dumps and find simpler solutions.

Costin


DM uninitialized constant Boolean

When running a migration to add a Boolean column to one of my tables using DataMapper, I am receiving this error:
=> Executing Rake dm:migrate ...
== Performing Up Migration \#10: add_active_to_restaurant
rake aborted!
uninitialized constant Boolean
/Users/xxxxxx/.rvm/gems/ruby-1.8.7-p249/gems/padrino-core-0.9.9/lib/padrino-core/cli/rake.rb:9:in `init'

My migration looks like this:
migration 10, :add_active_to_restaurant do
  up do
    modify_table :restaurants do
      add_column :active, Boolean
    end
  end

  down do
    modify_table :restaurants do
      drop_column :active
    end
  end
end

Thats weird Sequel also had the very same bug! Didn't think at all while typing that this will solve it!

The fix does not work for and throws another error:
no implicit conversion of String into Hash

Is there a way to use boolean fields in sequel?


Improve error message when providing wrong Workspace id

As reported on Nextflow Slack in the #tower-help channel here

Closing this issue as the current tower-cli version already improves the error messages when:

Launch with wrong workspace id:

Launch with wrong pipeline name:

Launch with wrong pipeline URL:


Nudge users to go to web/dev shell once ALDS starts

Suggestion came up when orienting Han: Let's make developers lives a bit easier by adding small suggested next steps with the console output. This is a similar pattern to tools/setup, where we also suggest next commands to run.

Created GH Issue out of @piotrswigon 's enhancement. Going to merge approved incremental improvement.


Publication of code and datasets relative to experiments

Hello! Thank you for the interesting paper. I have a few questions regarding the code and datasets used in your experiments.
I noticed that the notebooks available are for training the models and plotting the cross-chain attention. Are you planning to publish the code for the other experiments shown in the paper?
Additionally, in the last paragraph of the results section, you finetune the models for three classification tasks using datasets containing 20,000, 2,000, and 3,000 antibodies. Are these datasets available for public use?
I am interested in using these tasks for comparison with another model and wanted to know if the code and datasets will be made public or if I will need to reimplement them.
Thank you for your time and assistance!

Thanks for reaching out! We're happy to publish the additional code and the classification datasets -- give me a couple of days to get everything together. For the code, is there an experiment/figure you are most interested in, so I can prioritize it for you?

I was especially interested in figure 3 and I saw that you have already uploaded the code! Thank you very much for the quick response!

Happy to help! The classification datasets have been uploaded on Zenodo as well: https://zenodo.org/records/12745725


Update dependencies

def_site was replaced by call_site because new proc-macro2 doesn't expose it anymore. Besides, prost-derive only used spans for literals which shouldn't depend on span information at the same degree as identifiers.

This was fixed by #89, but thanks for the PR!


Cannot discover Samsung SmartTV

Hey everyone. I am trying to conduct devices search on my physical device, but there are only apple devices found in the network, such as my two MacBoks. My Samsung SmartTV can never be found during discovery. com.apple.developer.networking.multicast entitlement is enabled. Any ideas how to solve it? Thanks a lot in advance

I had same issue

any one solve this issue

Hi @MuhammadAsad786 @tuanvucourse @volodymyr-koff
I need one help, i am working on ConnectSDK iOS Configuration for TV Remote, i am setup sdk and give permission for local network and other and Try to find smart TV devices usinf DiscoveryManager class but not find any device,
var discoveryManager = DiscoveryManager.shared() discoveryManager?.registerDefaultServices()         let videoCapabilities = [             kMediaPlayerPlayVideo,             kMediaControlAny,             kVolumeControlVolumeUpDown         ]
`    let imageCapabilities = [kMediaPlayerDisplayImage]
let videoFilter = CapabilityFilter(capabilities: videoCapabilities)
let imageFilter = CapabilityFilter(capabilities: imageCapabilities)
discoveryManager?.delegate = self
discoveryManager?.capabilityFilters = [videoFilter, imageFilter]
discoveryManager?.pairingLevel = DeviceServicePairingLevelOn
discoveryManager?.registerDeviceService(WebOSTVService.self, withDiscovery: SSDPDiscoveryProvider.self)
discoveryManager?.registerDeviceService(AirPlayService.self, withDiscovery: ZeroConfDiscoveryProvider.self)
discoveryManager?.registerDeviceService(CastService.self, withDiscovery: CastDiscoveryProvider.self)
discoveryManager?.registerDeviceService(DIALService.self, withDiscovery: SSDPDiscoveryProvider.self)
discoveryManager?.registerDeviceService(RokuService.self, withDiscovery: SSDPDiscoveryProvider.self)
discoveryManager?.registerDeviceService(DLNAService.self, withDiscovery: SSDPDiscoveryProvider.self)
discoveryManager?.startDiscovery()

extension ConnectingDeviceViewController: DiscoveryManagerDelegate {

func discoveryManager(_ manager: DiscoveryManager!, didFind device: ConnectableDevice!) {
// A ConnectableDevice was found
print(device.friendlyName ?? "")
}
func discoveryManager(_ manager: DiscoveryManager!, didLose device: ConnectableDevice!) {
// A ConnectableDevice was lost
print(device.friendlyName ?? "")
}
func discoveryManagerDidFailWithError(_ error: Error!) {
print("Discovery error: (error.localizedDescription)")
}`
Please me if i need to add permission or anything else

Add multicast permission in info.plist, your issue will be solved

@MuhammadAsad786 Thank you so much for help it's resolved now and i am able to find devices.. Thank you so much for your help and your valuable time

Hi @MuhammadAsad786 i need one small help from your side.. My device listing is done but when i am connected to device using Connect() method but i need to send pairing code to Connected Device.. Have you any idea about that?

Add multicast permission in info.plist, your issue will be solved

how can add multicast permission sir?


CI: Update tested K8S versions

Update tested k8s versions for GKE, AKS, AWS-CNI, and EKS
Workflow Runs with all supported versions:
aws-cni : https://github.com/cilium/cilium/actions/runs/11665193779
eks : https://github.com/cilium/cilium/actions/runs/11665195477

/ci-gke versions=all

/ci-aks

/ci-awscni

/ci-eks

/ci-eks

/ci-aks

/ci-gke versions=all


[NEW] Add share the app feature to share the Google Play link in other apps

Description
Currently there is no way available to share the app. Add  feature to share the app by all possible ways available to phone.
Devices and Versions
Your Rocket.Chat.Android version: 3.1.0 Build 2049 - 60b39dad-play
Would you like to work on the issue?

Yes

Duplicates #98.

The feature was intended to  share the chat app to other phones by means of sharing using facebook, whatsapp, gmail and other sharing apps

@GOVINDDIXIT Share what exactly?

I mean one person can share this app to his friends by sharing the links through whatsapp or facebook or any other sharing app like shareit or xender

@GOVINDDIXIT Gotcha, you mean like an option in "Settings" to "Share this App with my friends" and then opening an intent with the Google Play URL?

Exactly 😃

@GOVINDDIXIT Gotcha, reopened it.

Also I will add sharing the app through xender and Shareit which will directly share the app to near friends using hotspot


When did fairseq move the tensors into gpu?

I found that the device of samples' tensors in dataset's collate() is CPU, but in forward() of the model, I found the samples' tensors are on GPU;
When and how fairseq put them on GPU ?
Thank you !

It happens here: https://github.com/pytorch/fairseq/blob/a4356b1da2b19ebd2e1be5c12ff882026ea4d7d2/fairseq/trainer.py#L466


[Automation] Create clean up tasks when a pull request is closed

User Story

As a [user role below]:
IBM.com Library developer

I need to:
remove deploy previews that are no longer needed

so that I can:
reduce cost of cloud object storage in IBM Cloud

Additional information

Deploy previews are currently stored in IBM Cloud object storage, under "ibm-dotcom-library"
Deploy previews have multiple buckets, a bucket for each package, as well as the cupcake website
Should determine if there should be a period of time before previews are deleted, or if they should be torn down the moment a PR is closed.

Acceptance criteria

[ ] Mechanism in place to remove unused deploy preview folders in IBM Cloud

Closing this, set a retention policy in IBM Cloud object storage.


linux: add flavor metadata

Motivation for this change
part of #101863
This is what I had in mind @NeQuissimus
I chose usbip because of https://hydra.nixos.org/eval/1622654?filter=linuxPackages&compare=1622624&full=#tabs-still-fail . Just being used as an example.
Ideally i would backport the metadata commit to 20.09 and disable the remaining failing kernel modules
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

$ nix-env -f /home/jon/.cache/nixpkgs-review/pr-102005/nixpkgs -qaP --xml --out-path --show-trace --meta
Nothing changed
https://github.com/NixOS/nixpkgs/pull/102005

@NeQuissimus do you mind giving this a quick review?

I think this might have broken linuxManualConfig. We were using it like this, which doesn't work anymore now:
(pkgs.linuxPackagesFor (pkgs.linuxManualConfig {
    inherit (pkgs) stdenv;
    inherit version;
    src = /some-kernel/src;
    allowImportFromDerivation = false;
    configfile = ./some-kernel.config;
    config = ./that-same-kernel.config-as.nix;
  }));

this only added items to passthru. It should not have caused any regressions.
I can't reproduce any errors with the above code.

I just tested this on top of 6330a9a998ea822ea72e0ba208f86a38db5f543a. This is my test.nix:
let pkgs = import ./. {};
in pkgs.linuxPackagesFor (pkgs.linuxManualConfig {
  inherit (pkgs) stdenv;
  inherit (pkgs.linux) version;
  src = pkgs.linux.src;
  allowImportFromDerivation = false;
  configfile = ./test.nix;
  config = {
    "CONFIG_MODULES" = "y";
  };
})

Running nix-build test.nix results in:
error: --- EvalError ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix-build
at: (18041:68) in file: /home/andy/nixpkgs/pkgs/top-level/all-packages.nix

 18040|
 18041|   linuxPackagesFor = kernel: lib.makeExtensible (self: with self; {
      |                                                                    ^
 18042|     callPackage = newScope self;

attribute 'isHardened' missing

@ajs124 do you mind trying your configuration against #102395

Full disclosure, I was only thing about hydra. And there's no hydra jobs which use linuxManualConfig

#102395 seems to work. I mean, it fails to evaluate, because ixgbevf-4.6.1-5.4.73 is marked as broken, but that's probably to be expected. And building .kernel evaluates/starts building.
There is linuxPackages_custom_tinyconfig_kernel in nixpkgs, which uses linuxPackages_custom, which uses linuxManualConfig. That didn't break though, because… idk.

I'm not sure either, and I got a cache hit. Could be coming from master? not sure.


DDMathParser not found on CocoaPods

Searching https://cocoapods.org/?q=ddmathparser no longer turns up any results.

I had this issue, just try using this command

pod 'DDMathParser', :git => 'https://github.com/davedelong/DDMathParser.git', :branch => 'master'


Health Report UI not rendering correctly on device rotate

Rotate the device on health report screen, it doesn't render correctly.

I thought I had earlier disabled screen rotation in the manifest.

Obviously this is not a true fix, but is fine for now.

@justinklchan : Hmm, i was fixing this issue :/

I see, if you are working on it, feel free to go ahead.
But I would want to focus on testing the core end to end protocol, like writing unit tests or running the critical code in PullFromServerTask.java.
Things like making sure the power consumption is reasonable.


Fix #76

Describe what this PR does / why we need it
Fix #76
Describe how you did it

更正 prompts 目标里的 question_answer_context 和 qa_retrieved_documents 不一致的问题
删除和光合作用有关的内 容，使得默认模板能适用绝大多数场景
将知识库的上下文追加到usertext 里，而不是直接替换 systemtext。使得 DashScopeDocumentRetrievalAdvisor 可以和 PromptChatMemoryAdvisor 这类 Advisor 兼
容

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

LGTM.


requirement.txt fails to load...

... when there are extra linebreaks in it
Seems to have been broken with https://github.com/jfloff/alpine-python/commit/76c2abe472a0f3952a65c7ae9ad5f7686c00e575
This works:
requests

this does not
requests

@tprobinson Could you take a look?

I can't seem to reproduce this. @holiman , could you share what happens when it breaks, and what other flags you're using? I tried this with requests and extra line breaks plus a -p docopt flag, but I didn't see a problem.

Closing due to lack of feedback. @holiman feel free to reopen


Unable to connect to Fake Server

I am using Hibernate, but I have encountered a serious problem. When waitmin passed, Hibernate stopped the server, and the server also showed that the fake server started running. When I tried to ping the fake server in the game, it showed that it could not connect to the server. I made sure that my server was configured correctly. I also tested it on my Windows and Linux systems. The following is the server log:
最近在试试Hibernate这个插件，但是遇到了很严重的问题。开服10分钟过后Hibernate停了服，服务端也显示Fake server开始运行，不过在游戏中尝试刷新去连接到fake server却显示无法连接至服务器，直接连接服务器时fake server也没有监听到有人尝试连接。服务端配置应该是正确的。我也在我的Windows和Centos系统都测试过。以下是相关服务端日志截图：

[Server] [13:50:31] [Server thread/INFO]: Done (2.742s)! For help, type "help" [MCDR] [13:51:33] [Thread-2/INFO]: Hibernate! [Server] [13:51:33] [Server thread/INFO]: Stopping the server [Server] [13:51:33] [Server thread/INFO]: Stopping server [Server] [13:51:33] [Server thread/INFO]: Saving players [Server] [13:51:33] [Server thread/INFO]: Saving worlds [Server] [13:51:33] [Server thread/INFO]: Saving chunks for level 'ServerLevel[world]'/minecraft:overworld [Server] [13:51:55] [Server thread/INFO]: ThreadedAnvilChunkStorage (world): All chunks are saved [Server] [13:51:55] [Server thread/INFO]: Saving chunks for level 'ServerLevel[world]'/minecraft:the_nether [Server] [13:51:56] [Server thread/INFO]: ThreadedAnvilChunkStorage (DIM-1): All chunks are saved [Server] [13:51:56] [Server thread/INFO]: Saving chunks for level 'ServerLevel[world]'/minecraft:the_end [Server] [13:51:56] [Server thread/INFO]: ThreadedAnvilChunkStorage (DIM1): All chunks are saved [Server] [13:51:56] [Server thread/INFO]: ThreadedAnvilChunkStorage (world): All chunks are saved [Server] [13:51:56] [Server thread/INFO]: ThreadedAnvilChunkStorage (DIM-1): All chunks are saved [Server] [13:51:56] [Server thread/INFO]: ThreadedAnvilChunkStorage (DIM1): All chunks are saved [MCDR] [13:51:56] [MainThread/INFO]: 服务端进程返回代码: 0 [MCDR] [13:51:56] [Thread-3/INFO]: FakeServer start! [MCDR] [13:51:56] [Thread-3/INFO]: FakeServer is ready
[MCDR] [2021-01-30 10:47:21] [MainThread/INFO]: 语言已设置为 zh_cn [MCDR] [2021-01-30 10:47:21] [MainThread/INFO]: 编码 / 解码方式已设置为 utf-8 / cp936 [MCDR] [2021-01-30 10:47:21] [MainThread/INFO]: 插件文件夹列表: [MCDR] [2021-01-30 10:47:21] [MainThread/INFO]: - plugins [MCDR] [2021-01-30 10:47:21] [MainThread/INFO]: 解析处理器已设置为 vanilla_handler [MCDR] [2021-01-30 10:47:21] [TaskExecutor/INFO]: 刷新所有插件中 [MCDR] [2021-01-30 10:47:22] [TaskExecutor/INFO]: 插件<EMAIL_ADDRESS>已加载 [MCDR] [2021-01-30 10:47:22] [TaskExecutor/INFO]: 插件<EMAIL_ADDRESS>已加载 [MCDR] [2021-01-30 10:47:22] [TaskExecutor/WARNING]: Plugin ID of RegularPlugin[file=info.py,path=plugins\info.py,state=0] not found, use fallback id info instead [MCDR] [2021-01-30 10:47:22] [TaskExecutor/WARNING]: RegularPlugin[file=info.py,path=plugins\info.py,state=0] doesn't specific a version, use fallback version 0.0.0 [MCDR] [2021-01-30 10:47:22] [TaskExecutor/INFO]: 插件<EMAIL_ADDRESS>已加载 [MCDR] [2021-01-30 10:47:22] [TaskExecutor/INFO]: 插件<EMAIL_ADDRESS>已加载 [MCDR] [2021-01-30 10:47:22] [TaskExecutor/INFO]: 插件<EMAIL_ADDRESS>已加载 [MCDR] [2021-01-30 10:47:22] [TaskExecutor/INFO]: 共加载: §75§r 个插件; 已加载插件数: §76§r [MCDR] [2021-01-30 10:47:22] [MainThread/INFO]: 正在启动服务端，启动参数为 "java -Xms1G -Xmx1G -jar fabric-server-launch.jar nogui" [MCDR] [2021-01-30 10:47:22] [MainThread/INFO]: 服务端正在以 PID 8880 运行中 [MCDR] [2021-01-30 10:47:24] [CheckUpdate/INFO]: MCDR 已为最新版 [MCDR] [2021-01-30 10:57:42] [Thread-2/INFO]: Hibernate! [MCDR] [2021-01-30 10:57:43] [MainThread/INFO]: 服务端进程返回代码: 0 [MCDR] [2021-01-30 10:57:43] [Thread-5/INFO]: FakeServer start! [MCDR] [2021-01-30 10:57:43] [Thread-5/INFO]: FakeServer is ready
衷心希望你能解决这个问题，如需更多配置信息请联系我

Same issue for me.
我也遇到了相同的问题。

Same issue for me.
我也遇到了相同的问题。

我跳出來
Error in getResultNew: timed out
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Python39\lib\threading.py", line 954, in _bootstrap_inner
self.run()
File "C:\Python39\lib\threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "plugins\Hibernate.py", line 41, in query_playernum
playerCount = Client().getResultNew()["OnlinePlayers"]
File "plugins\Hibernate.py", line 218, in getResultNew
server_property["Protocol"] = status["Protocol"]
KeyError: 'Protocol'
求解

The same problem,fakeserver cannot be detected on the client
一样的问题，在客户端检测不到fakeserver

我也是一样我问题

相同的问题

请尝试一下 https://github.com/HIM049/MCDR_HibernateR 看看是否能够正常使用


Describe Security Concept

Hi,
I really like your approach to this but I think the security implications should be documented a bit more or there has to be some instructions how to secure it.
Because as it currently is, you cannot use this for a multi-tenancy cloud with untrusted containers, as everyone can read/ write to etcd and marathon -- or am I wrong?
The alternative would be to have a management network where maraton, etcd and vulcand are exposing their APIs.
Thanks

I've added the following comment:
This tool isn't "production ready" and was developped to show end to end automation.
You're right, there's clearly a lack of security if it's used as-is.


Implement Message for Box<T: Message>

Fix for #185. Implements the Message trait for Box<T: Message> to allow for Boxed publishers and subscribers.
You should now be able to do
    let publisher =
        node.create_publisher::<Box<std_msgs::msg::String>>("topic", rclrs::QOS_PROFILE_DEFAULT)?;
    let mut message = Box::new(std_msgs::msg::String::default());
    publisher.publish(&message)?;

and
    let _subscription = node.create_subscription::<Box<std_msgs::msg::String>, _>(
        "topic",
        rclrs::QOS_PROFILE_DEFAULT,
        move |msg: Box<std_msgs::msg::String>| {
            num_messages += 1;
            println!("I heard boxed: '{}'", (*msg).data);
            println!("(Got {} messages so far)", num_messages);
        },
    )?;

It's been 3 years since I last properly wrote Rust so I'm not sure my Cow handling is idiomatic. Also not sure if the trait implementation is in an appropriate location.

Hi @LaVieEstDure, this works, let me just think aloud a little bit whether this is the best way.
The nicest solution imo would be one where the user can simply pass in either a T callback or a Box<T> callback when constructing a Subscription<T>. But I don't know how that could be made to work, you can't write a trait that abstracts over both forms. Can you think of a way to do this?
With this impl, you can now also construct Publisher<Box<T>>, but for publishers I'd prefer to integrate the Box variant into the MessageCow mechanism. With the MessageCow mechanism, the user can decide for each individual message whether it should be published by reference or owned (or as a Box, if we add that), but with the Message mechanism, you have to decide at the beginning, when creating the publisher, whether you're ever going to publish Boxes or not.
Another possibility would be always taking a Box<T> in callbacks. That has the drawback of being a bit more verbose/unergonomic and costing a heap allocation each time (not sure if that matters though).
What I like about the solution in this PR is that it keeps things simple – no extra traits.
So I'm inclined to say we go forward with this change, but I'd like to see if you or @esteve and @jhdcs see a way to get an optimal solution. BTW, your use of Cow and the location is fine :)

But I don't know how that could be made to work, you can't write a trait that abstracts over both forms. Can you think of a way to do this?

I think the demo there is the wrong approach. You're trying to specialise a Fn trait which Rust doesnt support. Would it work if instead of having callback: CallbackTrait which we have to specialise, we had callback: Fn(CallbackArgTrait) and we implemented CallbackArgTrait for all generated Message types which forwarded Message function implementations, and manually implemented the CallbackArgTrait for Box where the associated type of the implementation was the type of what the box was holding

If I remember correctly from ROS 1, you had to specify whether a subscription was for a message or a constant pointer to said message, right? If that's the case, then this PR is bringing back functionality from ROS 1 that could be useful for when a Box is needed.
I feel like we can probably look at reviewing this PR as-is, and revisiting it later if/when we discover that optimizations are needed.

Another possibility would be always taking a Box<T> in callbacks. That has the drawback of being a bit more verbose/unergonomic and costing a heap allocation each time (not sure if that matters though).

Personally, I would vote against this approach. I feel like it adds a "hidden tax" on the user by forcing them to make more heap allocations. This may reduce performance, and prevent rclrs from being used in the sort of high-speed high-throughput applications that Rust is normally good at.

One option, that is a bit more complicated than just simply implementing Message for Box is adding a second generic parameter over Subscription and constraining them by their associated type
impl<T, Arg> Subscription<T, Arg = T>
where
    T: Message,
    Arg: Message<RmwMsg = T::RmwMsg>
{
    /// Creates a new subscription.
    pub(crate) fn new<F>(
        node: &Node,
        topic: &str,
        qos: QoSProfile,
        callback: F,
    ) -> Result<Self, RclrsError>
    // This uses pub(crate) visibility to avoid instantiating this struct outside
    // [`Node::create_subscription`], see the struct's documentation for the rationale
    where
        T: Message,
        Arg: Message<RmwMsg = T::RmwMsg>,
        F: FnMut(Arg) + 'static + Send,

Which I think should work and I think you'll be able to change node to
    pub fn create_subscription<T, Arg, F>(
        &mut self,
        topic: &str,
        qos: QoSProfile,
        callback: F,
    ) -> Result<Arc<Subscription<T>>, RclrsError>
    where
        T: Message,
        Arg: Message<RmwMsg = T::RmwMsg>,
        F: FnMut(Arg) + 'static + Send,
    {
        let subscription = Arc::new(Subscription::<T, Arg>::new(self, topic, qos, callback)?);
        self.subscriptions
            .push(Arc::downgrade(&subscription) as Weak<dyn SubscriptionBase>);
        Ok(subscription)
    }

Or if we're trying real hard, we might be able to extend the Message trait to implement an UnderlyingMessage associated type, which is set to the same type for Message types and to the contained type for containers, and do
        let subscription = Arc::new(Subscription::<Arg::UnderlyingMsg, Arg>::new(self, topic, qos, callback)?);

But now we're getting into uglier territory. Feel free point out where I'm dumb and forgetting something that'll make make the compiler reee at me or if there's a simpler solution to what I just proposed

Also regarding

If I remember correctly from ROS 1, you had to specify whether a subscription was for a message or a constant pointer to said message, right? If that's the case, then this PR is bringing back functionality from ROS 1 that could be useful for when a Box is needed.

I think all the messages were passed around as const shared pointers. The template specification didnt change anything, it was just improved over time to be more hygienic to write. Subscriber still took a callback that had took, in effect, a Boost const shared ptr but you could pass in a callback that took const Msg& and it turned implicitly into that semantically equivalent type.

I think the demo there is the wrong approach. You're trying to specialise a Fn trait which Rust doesnt support. Would it work if instead of having callback: CallbackTrait which we have to specialise, we had callback: Fn(CallbackArgTrait) and we implemented CallbackArgTrait for all generated Message types which forwarded Message function implementations, and manually implemented the CallbackArgTrait for Box where the associated type of the implementation was the type of what the box was holding

That's definitely better, but it also has its problems: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b5d797bbde4864f45b0e9a33635e8b22
Another approach would be to simply have two functions, create_subscription() and create_subscription_for_heap_message() or something. Then we can store the callback internally as
enum Callback {
    Regular: Box<dyn Fn(T) + 'static + Send>,
    Heap: Box<dyn Fn(Box<T>) + 'static + Send>
}

But I don't like that so much either. So, let's go ahead with this PR, and revisit it when needed. I have just one small change to request.


Bump to 0.34.0

@bors: r+

:pushpin: Commit 1b886f70729f04e7e7c5acf5ec3af6838ddad219 has been approved by alexcrichton

:hourglass: Testing commit 1b886f70729f04e7e7c5acf5ec3af6838ddad219 with merge 316182ebced80bfa0033a91f0202b28c4c730a8c...

:broken_heart: Test failed - status-appveyor

Oh dear did I break it?  I might not get until tomorrow a chance to look into it on a laptop.

For reference, it looks like this was caused by rust-lang/rust#56392.  Before that PR, the entries like use cargo; did not issue an unused warning.

:hourglass: Testing commit 1b886f70729f04e7e7c5acf5ec3af6838ddad219 with merge c4b5a6368654d80fb22519f548124751aca9c05b...


refactor!(core): remove CMapError

This enum had a single variant: CMapError::UndefinedVertex. Removing the error and replacing Results with Options simplifies the code and makes the vertex/attribute metho signatures consistent.
Scope

[x] Code: core crate, other changes are just fixes

Type of change

[x] Refactor

Other

[x] Breaking change

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.01%. Comparing base (c98aebe) to head (be82241).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #171      +/-   ##
==========================================
- Coverage   82.02%   82.01%   -0.02%
==========================================
  Files          32       32
  Lines        4513     4504       -9
==========================================
- Hits         3702     3694       -8
+ Misses        811      810       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Document getopt output on stderr.

Document defaultGetoptFormatter as an alternative to defaultGetoptPrinter for printing on stderr.

The opinion of stackoverflow and stackexchange: options should go to stdout when requested by -h/--help. On error, a message to stderr is important, but options would normally not be part of that -- a reference to "--help" would suffice.
Closing.


Click on portfolio images not working

link for portfolio images is written as "#" takes you to top, doesn't open the images in modal like similar BS templates.

The path to the images is in the .css files type _masthead.scss and _callout.scss

There is nothing connected to the portfolio image links at this point, it is something for the designer to wire up if they want it to point to anything like a modal or lightbox.


License

Thank you for implementing this. Could you please add a LICENSE file that specifies the license your code is available under?

Thanks for your interest!
Added a readme file with pointer to apache license.


[test] migrate Http2ProtocolOptions test stubs to v2

Description: Chipping away at #6362 again, converting test stubs to the v2 API to remove usage of v1 code.
Risk Level: Low
Testing: Existing
Docs Changes: N/A
Release Notes: N/A
Signed-off-by: Derek Argueta<EMAIL_ADDRESS>

Thank you! Needs format fix.
/wait


Audio PR 7 : Last Batch Policy, stick_to_shard, shard_size & auto_reset

Audio PR 7 : Last Batch Policy, stick_to_shard, shard_size & auto_reset
Changes for last batch policy is introduced in this PR for audio pipeline
The variables shard_size, auto_reset have been introduced and incorporated in the iterator
The variable stick_to_shard and shard_size has been added as a part of the reader_cfg to reset() the data when shard_size >0 and stick_to_shard is False (to increase the overall entropy of the dataset across different shards) for audio pipeline

Changed the branch name to swbs/audio/pr10. Hence closing the PR


Suggestion - Be able to manage servers and use key per server

In controlled environments, it's useful to be able to reuse server setting.
What do you think?

It is something a few people have asked for, something I want to look into but it will require quite a bit of refactoring at present

Maybe it's possible to store the settings and then use some kind of hook to create the servers. This way you wouldn't have to refactor it all.
The SSL keys is another think that might be worth looking into using just the one. What was the logic behind using a unique key per project?

Security, if a key gets exposed you'd only need to invalidate it on one server rather than possibly dozens. However, you can provide your own key to use instead from the "SSH Key" tab when creating a project. A key manager is also on the list of improvements (I have a branch I started a long time ago but I want to get all tests written before I do any more refactoring)

Yeah I use same server for a number of projects so I have to keep adding it to new projects.
What about using the template section to add servers in? Maybe select server on project creation?

In my experience rarely one app runs on one server. If one key gets exposed the most likely is that it's in use on several servers. For me particularly the least servers I have running an app is 2.
If a key got exposed I would still have to go through several servers to delete it.
Do you mind if I have a go on changing it to make it the admin choice to use a generic key, a key per server or current behaviour?

Feel free. As I said, you can currently provide your own SSH key when you set up a project from the SSH key tab. Take a look at the features/key-manager branch, I had started on a feature to allow this so there is a section in the admin tools to show a list of keys and which projects they are used on, and was planning to change the project management so when you create it you get a list of existing keys you can use, the option to generate a new key and the option to enter your own key. It is next on my list of things to work on after I finish the test coverage (currently at 93%, not much more to go...)

I'm having some time off next week, I'll look into it then.
Cheers

I've looked into your feature branch, but at the moment it's so far behind master that merge is causing to many conflicts in to many key peaces of code. What do you think about a complete rewrite?

that is a fair point. Probably easier yeah, don't think there was much there to begin with

I've split this feature in 2.

Server manger
Key manager
for the first one I've pretty much finished, pushed the code to branch feature/server_manager. I haven't run any of the automation tools neither have build tests. Will do both before the pull request.

Feel free to browse around https://github.com/cfpinto/deployer/tree/feature/server_manager

👍 I shall take a look

You know you've committed the .idea folder right?

Yeah, it was really late last night. I've removed already

Looks good.