inverse grapher link not picking up all data

I have 2 collections: users and groups. I have a main link from groups to users based on memberIds field. I also have an inverse link from users to groups:
I have two users: Ujwal and Vineet. I have 5 groups:

members: Ujwal and Vineet; admin: Ujwal
members: Vineet and Ujwal, admin: Vineet
members: Vineet and Ujwal, admin: Vineet and Ujwal
members: Vineet; admin: Vineet
member: Ujwal; admin: Ujwal

Here are the records:
{
	"_id" : "nSkwTyjPFnmmZ6DYY",
	"name" : "Ujwal and Vineet",
	"description" : "Ujwal is admin",
	"createdAt" : ISODate("2018-07-31T22:22:03.607Z"),
	"createdBy" : "WaHgrKPe6ePJJuE2o",
	"adminIds" : [
		"WaHgrKPe6ePJJuE2o"
	],
	"memberIds" : [
		"WaHgrKPe6ePJJuE2o",
		"yf2AsaHQN36TqKtFc"
	]
}
{
	"_id" : "beLXTBrRAXtcf2oZc",
	"name" : "Vineet and Ujwal",
	"description" : "Vineet is admin",
	"createdAt" : ISODate("2018-07-31T22:23:51.461Z"),
	"createdBy" : "yf2AsaHQN36TqKtFc",
	"adminIds" : [
		"yf2AsaHQN36TqKtFc"
	],
	"memberIds" : [
		"WaHgrKPe6ePJJuE2o",
		"yf2AsaHQN36TqKtFc"
	]
}
{
	"_id" : "Ga4is5nwBXugxpRGt",
	"name" : "Vineet",
	"description" : "Vineet is admin",
	"createdAt" : ISODate("2018-07-31T22:24:40.055Z"),
	"createdBy" : "yf2AsaHQN36TqKtFc",
	"adminIds" : [
		"yf2AsaHQN36TqKtFc"
	],
	"memberIds" : [
		"yf2AsaHQN36TqKtFc"
	]
}
{
	"_id" : "CFycmtemFgSeZNFJA",
	"name" : "Ujwal",
	"description" : "Ujwal is admin",
	"createdAt" : ISODate("2018-07-31T22:25:07.206Z"),
	"createdBy" : "WaHgrKPe6ePJJuE2o",
	"adminIds" : [
		"WaHgrKPe6ePJJuE2o"
	],
	"memberIds" : [
		"WaHgrKPe6ePJJuE2o"
	]
}
{
	"_id" : "cyqFAhpCA2LzBGhsP",
	"name" : "Ujwal and Vineet",
	"description" : "Both are admin",
	"createdAt" : ISODate("2018-07-31T22:25:50.003Z"),
	"createdBy" : "WaHgrKPe6ePJJuE2o",
	"adminIds" : [
		"WaHgrKPe6ePJJuE2o",
		"yf2AsaHQN36TqKtFc"
	],
	"memberIds" : [
		"WaHgrKPe6ePJJuE2o",
		"yf2AsaHQN36TqKtFc"
	]
}

Here is my GraphQL and grapher setup:
type Group @mongo(name: "groups")
{
  _id: ID!
  name: String
  members: [User] @link(field: "memberIds")
}

type User @mongo(name: "users")
{
  _id: ID!
  emails: [Email]
  firstName: String
  lastName: String
  name: String
  roles: [String]
  groups: [Group] @link(to: "members")
}

I can query via the main link to get all groups and the members in them:
query groups {
  groups{
    _id
    name
    members {
      _id
      firstName
      lastName
      name
    }
  }
}

Result:
{
  "data": {
    "groups": [
      {
        "_id": "nSkwTyjPFnmmZ6DYY",
        "name": "Ujwal and Vineet",
        "description": "Ujwal is admin",
        "members": [
          {
            "_id": "WaHgrKPe6ePJJuE2o",
            "firstName": "Ujwal",
            "lastName": "Setlur",
            "name": "Ujwal Setlur"
          },
          {
            "_id": "yf2AsaHQN36TqKtFc",
            "firstName": "Vineet",
            "lastName": "Chikarmane",
            "name": "Vineet Chikarmane"
          }
        ]
      },
      {
        "_id": "beLXTBrRAXtcf2oZc",
        "name": "Vineet and Ujwal",
        "description": "Vineet is admin",
        "members": [
          {
            "_id": "WaHgrKPe6ePJJuE2o",
            "firstName": "Ujwal",
            "lastName": "Setlur",
            "name": "Ujwal Setlur"
          },
          {
            "_id": "yf2AsaHQN36TqKtFc",
            "firstName": "Vineet",
            "lastName": "Chikarmane",
            "name": "Vineet Chikarmane"
          }
        ]
      },
      {
        "_id": "Ga4is5nwBXugxpRGt",
        "name": "Vineet",
        "description": "Vineet is admin",
        "members": [
          {
            "_id": "yf2AsaHQN36TqKtFc",
            "firstName": "Vineet",
            "lastName": "Chikarmane",
            "name": "Vineet Chikarmane"
          }
        ]
      },
      {
        "_id": "cyqFAhpCA2LzBGhsP",
        "name": "Ujwal and Vineet",
        "description": "Both are admin",
        "members": [
          {
            "_id": "WaHgrKPe6ePJJuE2o",
            "firstName": "Ujwal",
            "lastName": "Setlur",
            "name": "Ujwal Setlur"
          },
          {
            "_id": "yf2AsaHQN36TqKtFc",
            "firstName": "Vineet",
            "lastName": "Chikarmane",
            "name": "Vineet Chikarmane"
          }
        ]
      }
    ]
  }
}

But when I query from the users via the inverse link by user and all the groups that the user is member of, I don't get all the records. Specifically, I don't the record where they are the only member:
query me {
  me{
    _id
    name
    groups {
      _id
      name
      description
    }
  }
}

This is the result I get for Vineet is this. Notice that we are missing the group where Vineet is the only member.
{
  "data": {
    "me": {
      "_id": "yf2AsaHQN36TqKtFc",
      "name": "Vineet Chikarmane",
      "groups": [
        {
          "_id": "nSkwTyjPFnmmZ6DYY",
          "name": "Ujwal and Vineet",
          "description": "Ujwal is admin"
        },
        {
          "_id": "beLXTBrRAXtcf2oZc",
          "name": "Vineet and Ujwal",
          "description": "Vineet is admin"
        },
        {
          "_id": "cyqFAhpCA2LzBGhsP",
          "name": "Ujwal and Vineet",
          "description": "Both are admin"
        }
      ]
    }
  }
}

Same case if I query for Ujwal, record with Ujwal as only member is missing:
{
  "data": {
    "me": {
      "_id": "WaHgrKPe6ePJJuE2o",
      "name": "Ujwal Setlur",
      "groups": [
        {
          "_id": "nSkwTyjPFnmmZ6DYY",
          "name": "Ujwal and Vineet",
          "description": "Ujwal is admin"
        },
        {
          "_id": "beLXTBrRAXtcf2oZc",
          "name": "Vineet and Ujwal",
          "description": "Vineet is admin"
        },
        {
          "_id": "cyqFAhpCA2LzBGhsP",
          "name": "Ujwal and Vineet",
          "description": "Both are admin"
        }
      ]
    }
  }
}

When I query for all users and their groups, I get this:
query users {
  users{
    _id
    name
    groups {
      _id
      name
      description
    }
  }
}

{
  "data": {
    "users": [
      {
        "_id": "ZbbLfS2tcwdnhLgeY",
        "name": "Seamlz Admin",
        "groups": null
      },
      {
        "_id": "WaHgrKPe6ePJJuE2o",
        "name": "Ujwal Setlur",
        "groups": [
          {
            "_id": "nSkwTyjPFnmmZ6DYY",
            "name": "Ujwal and Vineet",
            "description": "Ujwal is admin"
          },
          {
            "_id": "beLXTBrRAXtcf2oZc",
            "name": "Vineet and Ujwal",
            "description": "Vineet is admin"
          },
          {
            "_id": "cyqFAhpCA2LzBGhsP",
            "name": "Ujwal and Vineet",
            "description": "Both are admin"
          }
        ]
      },
      {
        "_id": "yf2AsaHQN36TqKtFc",
        "name": "Vineet Chikarmane",
        "groups": [
          {
            "_id": "Ga4is5nwBXugxpRGt",
            "name": "Vineet",
            "description": "Vineet is admin"
          }
        ]
      }
    ]
  }
}

Notice that for Vineet, it does not list all his groups, only the group where he is the sole member.
What am I doing wrong?

When I did a direct grapher query on users and their groups, I get the same result:
      let query = Meteor.users.createQuery({
        groups: {
          name: 1,
          description: 1
        }
      });

      let result = query.fetch();
      console.log("result: " + JSON.stringify(result));

Result:
result: [{"_id":"ZbbLfS2tcwdnhLgeY"},{"_id":"WaHgrKPe6ePJJuE2o","groups":[{"_id":"nSkwTyjPFnmmZ6DYY","name":"Ujwal and Vineet","description":"Ujwal is admin"},{"_id":"beLXTBrRAXtcf2oZc","name":"Vineet and Ujwal","description":"Vineet is admin"},{"_id":"cyqFAhpCA2LzBGhsP","name":"Ujwal and Vineet","description":"Both are admin"}]},{"_id":"yf2AsaHQN36TqKtFc","groups":[{"_id":"Ga4is5nwBXugxpRGt","name":"Vineet","description":"Vineet is admin"}]}]

Next, I will try setting up the links directly in grapher instead of using the schema directives.

Hmm, even if I bypass the schema-directives and use addLinks() directly, I am still getting the same result:
import Users from "./users/collection";
import Groups from "./groups/collection";

Groups.addLinks({
  "members": {
    type: "many",
    collection: Users,
    field: "memberIds"
  }
});

Users.addLinks({
  "groups": {
    collection: Groups,
    inversedBy: "members"
  }
});

let query = Meteor.users.createQuery({
    groups: {
      name: 1,
      description: 1
    },
    profile: 1
  });

  let result = query.fetch();
  console.log("result: " + JSON.stringify(result));

result: [{"_id":"ZbbLfS2tcwdnhLgeY","profile":{"name":"Seamlz Admin","firstName":"Seamlz","lastName":"Admin"}},{"_id":"WaHgrKPe6ePJJuE2o","profile":{"name":"Ujwal Setlur","firstName":"Ujwal","lastName":"Setlur","company":"Seamlz"},"groups":[{"_id":"nSkwTyjPFnmmZ6DYY","name":"Ujwal and Vineet","description":"Ujwal is admin"},{"_id":"beLXTBrRAXtcf2oZc","name":"Vineet and Ujwal","description":"Vineet is admin"},{"_id":"cyqFAhpCA2LzBGhsP","name":"Ujwal and Vineet","description":"Both are admin"}]},{"_id":"yf2AsaHQN36TqKtFc","profile":{"name":"Vineet Chikarmane","firstName":"Vineet","lastName":"Chikarmane","company":"Pret Interpret"},"groups":[{"_id":"Ga4is5nwBXugxpRGt","name":"Vineet","description":"Vineet is admin"}]}]

I am very confused. Clearly, I am doing something wrong, but I can't figure out what. This seems like such a straightforward use case.

Any thoughts on this?

@ujwal-setlur I cannot reproduce this this is extremely weird!

@theodordiaconu I will test this again tonight

I ca still reproduce this.

Here are my resolvers for me and users:
  Query: {
    users(_, args, {db, userId}, ast) {
      console.log(`Calling users query; user = ${userId}`);

      // sanity checks
      if (!userId) {
        throw new Meteor.Error("authorizartion-error", "Action not authorized");
      }

      // use grapher to do the query instead of direct mongo query
      let allUsers = db.users.astToQuery(ast, {
        // filter out our admin users
        $filters: {
          roles: {
            $nin: ["admin"]
          }
        }
      }).fetch();
      return allUsers;
    },
    me(_, args, {db, userId}, ast) {
      // use grapher to do the query instead of direct mongo query
      let result = db.users.astToQuery(ast, {
        $filters: {
          _id: userId
        }
      }).fetch();
      if (result && result.length) {
        return (result[0]);
      } else {
        return (null);
      }
    }
  }

Here is the groups resolver which is over the main link and works:
  Query: {
    groups(_, args, {db, userId}, ast) {

      // sanity checks
      if (!userId) {
        throw new Meteor.Error("authorization-error", "Action not authorized");
      }

      let filter = {
        memberIds: userId
      };
      if (args._id) {
        filter._id = args._id;
      }
      // use grapher to do the query instead of direct mongo query
      let allGroups = db.groups.astToQuery(ast, {
        $filters: filter
      }).fetch();
      return allGroups;
    }
  }

Fixed via fixing https://github.com/cult-of-coders/grapher/issues/282


Add fzfcmd option

It use sh -c to be able the pipe sh syntax.
It runs by default sh -c 'fzf fzfargs' and if we set fzfcmd sh -c 'fzfcmd | fzf fzfargs'.

this option should be well thought out.
I got unpredictable behavior.

I implemented this function in a different way.
040ed52


colSpan for Cell

how to have an attribute like colspan for the element cell?

@cheesest3ak there isn't currently a way to do this since colspan is a feature of <td> elements and we use <div> elements with display: table-cell for a number of reasons. We may be able to build this functionality in eventually though.
For now you may be able to use multiple <Cell>s and place some overflowing content inside the first one.

Unfortunately, since we're not using actual table elements, and instead just table display properties (we're doing this for a number of reasons - table elements are an abomination), this would be very difficult and require a major restructuring that probably would involve CSS grid.
I'm not going to add this to the roadmap for now, but if we ever switch to CSS grid, I'll reopen. Happy to look at PRs for this in the meantime.
https://stackoverflow.com/questions/2403990/html-colspan-in-css


sorts browse nav bar alphabetically.

Coverage remained the same at 0.0% when pulling b4a9f035334194a82725a72766215254aa054170 on theseyi:linkedin into 290f925cd12109d03064789d11c0a0ec794f0c33 on linkedin:linkedin.

Coverage remained the same at 0.0% when pulling b4a9f035334194a82725a72766215254aa054170 on theseyi:linkedin into 290f925cd12109d03064789d11c0a0ec794f0c33 on linkedin:linkedin.


Kafka errors shouldn't cause perf_process to fail

If we are unable to publish metrics to kafka for whatever reason this
shouldn't cause the whole perf_process step to fail.


[RFC] [dask] [docs] Override details in API docs that differ between dask and sklearn interfaces

Summary
The module lightgbm.dask contains model objects which have very similar APIs to those from lightgbm.sklearn, and which directly inherit from them. For example, lightgbm.dask.DaskLGBMRegressor inherits from lightgbm.sklearn.DaskLGBMRegressor.
We currently take advantage of this inheritance to also inherit doc strings. For example: https://github.com/microsoft/LightGBM/blob/b4b1b75ac582e99202062ec784269bd5b77f2018/python-package/lightgbm/dask.py#L452-L457
That code reads the parent class's __doc__ and then patches it with some Dask-specific details.
There are still many places in the API docs which should be updated to show information specific to the Dask module. For example, the docs for the X, y, group, and sample_weight arguments to .fit() and .predict() should be updated to say that they expect Dask inputs (Dask Array, Dask DataFrame, Dask Series).

Motivation
This change would make the API docs a reliable source of information for how to structure inputs for the Dask module, which would make it easier for users to get started. This has value as a complement to a long-form tutorial (#3814).
Description
I can think of a few different ways to accomplish this.
Option 1: custom templates
We could write documentation templates that are shared between the classes. Here's an oversimplified example.
__fit_params_template = """
Build a gradient boosting model from the training set (X, y).

    Parameters
    ----------
    X : {x_type} of shape = [n_samples, n_features]
        Input feature matrix.
    y : {y_type} shape = [n_samples]
        The target values (class labels in classification, real numbers in regression).
    sample_weight : {weight_type} of shape = [n_samples] or None, optional (default=None)
        Weights of training data.
    init_score : {init_score_type} of shape = [n_samples] or None, optional (default=None)
        Init score of training data.
"""

__DaskCollectionDescription = "A Dask DataFrame, Dask Array, or Dask Series"
__LocalArrayDescription = "A pandas dataframe, numpy array, or sparse matrix"

class LGBMClassifier:
    def fit(...):
        pass

    fit.__doc__ = __fit_params_template.format(
        x_type=__LocalArrayDescription,
        y_type=__LocalArrayDescription,
        weight_type=__LocalArrayDescription,
        init_score_type=__LocalArrayDescription,
    )

class DaskLGBMClassifier:
    def fit(...):
        pass

    fit.__doc__ = __fit_params_template.format(
        x_type=__DaskCollectionDescription,
        y_type=__DaskCollectionDescription,
        weight_type=__DaskCollectionDescription,
        init_score_type=__DaskCollectionDescription,
    )

Something similar is done in the R package to avoid writing the same documentation in multiple places in source control, even though it is used in multiple places in the

https://github.com/microsoft/LightGBM/blob/b4b1b75ac582e99202062ec784269bd5b77f2018/R-package/R/lightgbm.R#L1
https://github.com/microsoft/LightGBM/blob/b4b1b75ac582e99202062ec784269bd5b77f2018/R-package/R/lgb.train.R#L4

Option 2: use sphinx-autodoc-typehints
If the only difference we anticipate in the API docs is about the types of inputs, we could stop relying on typehints in docstrings, and instead use sphinx-autodoc-typehints. This extension creates the typehints in Sphinx docs automatically based on the hints in the code, without you needing to write them in docstrings. I think that might allow us to just inherit docstrings and get the hinting stuff for free.
Option 3: just copy the docstrings and manually keep them in sync
We could stop having the Dask interface inherit the docstrings from lightgbm.sklearn, and instead just literally copy the sklearn docs into the Dask module. Then the type hints and any other information could be changed in the Dask module freely and easily.
References

DaskLGBMClassifier docs, for reference: https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.DaskLGBMClassifier.html#lightgbm.DaskLGBMClassifier .
this issue was inspired by this comment: https://github.com/microsoft/LightGBM/pull/3866#discussion_r565016128

For any of you reading this, let me know if you have strong opinions or can think of other options!
I personally prefer Option 1 or Option 3. Options that let us keep the same documentation style that's already in use seem the least disruptive. I don't like Option 2 because I expect that the differences between Dask and sklearn docs will be more than just type hint differences.

For any of you reading this, let me know if you have strong opinions or can think of other options!
I personally prefer Option 1 or Option 3. Options that let us keep the same documentation style that's already in use seem the least disruptive. I don't like Option 2 because I expect that the differences between Dask and sklearn docs will be more than just type hint differences.

I "like" how official Dask documentation deals with the problem! 😃

I think Option #1 is preferable for me. But how will it help to deal with different args? For example, client argument in Dask API.

I "like" how official Dask documentation deals with the problem! 😃

I think Option #1 is preferable for me. But how will it help to deal with different args? For example, client argument in Dask API.

I understand why Dask does that...they are wrapping external libraries that can change without their control, and where people can have different combinations of versions installed (e.g. different versions of numpy and dask at any one time).
Since we're talking about only code inside lightgbm, I don't think we should settle for that. It's within our control to make the docs consistent and correct.

But how will it help to deal with different args?

Well client is the only extra arg we have right now, and I don't anticipate there being so many others that we need a very scalable solution to handling the set of keyword arguments being different. I don't think it would be too bad for the template to have a {client_doc} which is just set to an empty string in fit.__doc__.format() on lightgbm.sklearn.LGBMClassifier.fit() and other sklearn methods.

I understand why Dask does that...they are wrapping external libraries that can change without their control, and where people can have different combinations of versions installed (e.g. different versions of numpy and dask at any one time).
Since we're talking about only code inside lightgbm, I don't think we should settle for that. It's within our control to make the docs consistent and correct.

But how will it help to deal with different args?

Well client is the only extra arg we have right now, and I don't anticipate there being so many others that we need a very scalable solution to handling the set of keyword arguments being different. I don't think it would be too bad for the template to have a {client_doc} which is just set to an empty string in fit.__doc__.format() on lightgbm.sklearn.LGBMClassifier.fit() and other sklearn methods.


feat(derive): Past batch validity variant

Overview
Adds a new variant to the BatchValidity enum that activates with Holocene. When a batch is outdated, after Holocene, the Past variant is returned. In the BatchQueue, this instructs a silent drop, without flushing the active span batch and channel.
We should follow this PR up with removing the additional check within add_batch, deferring to the validity checks in derive_next_batch.

This LGTM - rip coverage


incompatibility introduced by VCVRack 0.6.1

Seq-G1 and Seq-G2 will not compile under VCVRack version 0.6.1 due to a change in the way that debug has been implemented in rack.
Previously it was a function in the rack namespace, Seq-G1 and Seq-G2 both contain local methods which hide the rack debug function.
Since 0.6.1, debug has become a macro, with the result it's value is substituted in the method definition in the Seq-G1 and Seq-G2 sources, resulting in a syntax error.
Seq-G1 line 174
Seq-G2 line 190

Sorry, duplicate, just found a pull request to deal with this.


fixed Buffer constructor deprecation

Quick PR replacing deprecated Buffer constructor with Buffer.alloc() in binary writer.
All 1369 test are passing.

Coverage remained the same at 100.0% when pulling 226d9889dfcd74beee0f570a1580894857f6e7f1 on dehmer:buffer-deprecation into b85b7b93af101a87a7a0fffa2a6d68c551b3f8d3 on cschwarz:master.

Can this be merged into the published npm package?


Array of tables output

If I have this input:
[[Winter]]
December = [
   { Sunday = 6 },
   { Monday = 7 }
]
January = [
   { Sunday = 6 },
   { Monday = 7 }
]
[[Spring]]
March = [
   { Sunday = 6 },
   { Monday = 7 }
]
April = [
   { Sunday = 6 },
   { Monday = 7 }
]

with this program:
#include <fstream>
#include <iostream>
#include <toml.hpp>

int main() {
   auto config = toml::parse_file( "configuration.toml" );
   std::cout << config << std::endl;
   return 0;
}

I get this result:
[[Spring]]
April = [ { Sunday = 6 }, { Monday = 7 } ]
March = [ { Sunday = 6 }, { Monday = 7 } ]

[[Winter]]
December = [ { Sunday = 6 }, { Monday = 7 } ]
January = [ { Sunday = 6 }, { Monday = 7 } ]

is it possible to get something closer to the input?

No, the parser and serializer do not preserve formatting. The serializer instead emits the TOML in a terse, "best-fit" format, only adding newlines between tables or when it would otherwise have to wrap (e.g. when an array's elements reach some width threshold).
Order isn't preserved either since the TOML spec doesn't mandate that order is important.

Hm, I see that now! It looks like this input produces exactly the same output:
[[Winter]]
December = [
   { Sunday = 6 },
   { Monday = 7 },
   { Tuesday = 8 },
   { Wednesday = 9 },
   { Thursday = 10 },
   { Friday = 11 },
   { Saturday = 12 },
   { Sunday = 13 }
]

interesting. I guess its not an issue then. Is it possible to control the wrap
length?

Yeah, it's hardcoded. Would be fairly easy to expose it as an option though.

would you accept a PR? I am not great with C++ but I can try to work something up

I'd accept a PR, sure, just give CONTRIBUTING a quick read first.
Though if you're not too confident in your C++ I'm also happy with design feedback here in this thread. Either is ok.

probably several ways to handle this, but I was thinking something simple like
#ifdef TOML_WRAP
   inline constexpr size_t default_formatter_line_wrap = TOML_WRAP;
#else
   inline constexpr size_t default_formatter_line_wrap = 120_sz;
#endif

If you want to go that route, have a look at the various user config bits in toml_preprocessor.hpp. Generally I use the form
#ifndef TOML_FANCY_NEW_CONFIG
    #define TOML_FANCY_NEW_CONFIG the_default_value
#endif

So that it's always defined and I don't have to branch on it later.
Eventually I'd like to make this a runtime-configurable option in the formatter interface, though allowing a config to override the default line length is a good start. I'd suggest the name TOML_DEFAULT_LINE_WRAP.

Do you mean this?
https://github.com/marzer/tomlplusplus/blob/master/include/toml++/toml_preprocessor.h
I didnt see a toml_preprocessor.hpp. Otherwise I think I have enough to make up a PR, unless you have any other suggestions.

Oh, yup. That's the one. My crappy legacy choice to mix .h and .hpp strikes again.

ok I think Im done, but I cant get the single header to rebuild. I tried this:
PS C:\tomlplusplus\python> python generate_single_header.py
Traceback (most recent call last):
  File "C:\tomlplusplus\python\generate_single_header.py", line 9, in <module>
    import utils
ModuleNotFoundError: No module named 'utils'

then this:
PS C:\tomlplusplus\python> python utils.py generate_single_header.py

second command didnt give any error, but it also didnt change the single header file

Er. Wtf.
Try a normal CMD shell instead of PS?

Seem same result

What about py generate_single_header.py?
Also, which version of python is that on your machine? The scripts are written in python 3.

I think I see the issue. If I run this:
python utils.py generate_single_header.py

It just runs utils.py with generate_single_header.py as an argument.
utils.py is just a bunch of functions, so nothing happens. If I run this:
python generate_single_header.py

I get this:
Traceback (most recent call last):
  File "C:\tomlplusplus\python\generate_single_header.py", line 9, in <module>
    import utils
ModuleNotFoundError: No module named 'utils'

I think it has to do with this:
https://github.com/marzer/tomlplusplus/blob/9ea4438fc4ac7f19d158e80b05e0123090dda583/python/generate_single_header.py#L9
I dont think that is the right way to import from a local file:
https://stackoverflow.com/questions/4142151

Ah, I see. Odd that it's only been an issue for you now, but has worked for others. Welp, guess I have some python stuff to learn!
If your changes are trivially simple you can manually edit toml.hpp such that the changes are exactly mirrored there, and it should work OK.

correction, the Python stuff was my error:
https://stackoverflow.com/a/65226460
sorry about that. Will try to finish up PR now

I think I am doing something wrong. If I do a fresh clone of the repo, and then
use this file:
[[Winter]]
December = [
   { Sunday = 6 },
   { Monday = 7 },
   { Tuesday = 8 },
   { Wednesday = 9 },
   { Thursday = 10 },
   { Friday = 11 },
   { Saturday = 12 },
   { Sunday = 13 }
]

I get the same input and output, as expected. But if I go here:
https://github.com/marzer/tomlplusplus/blob/9ea4438fc4ac7f19d158e80b05e0123090dda583/toml.hpp#L8340
and just manually changed that line to 80_sz, then I get this result:
[[Winter]]
December = [ { Sunday = 6 }, { Monday = 7 }, { Tuesday = 8 }, { Wednesday = 9 }, { Thursday = 10 }, { Friday = 11 }, { Saturday = 12 }, { Sunday = 13 } ]

Oh, yah, it stops doing multi-line expansion of objects once indent + width >= default_formatter_line_wrap  (see impl::default_formatter_forces_multiline). Doesn't make much sense here, though. I should revisit that logic.

Fixed in 54d80bb.

https://github.com/marzer/tomlplusplus/pull/74


Escaped $ and dry-principle on shell scripts in VeraCrypt guide

Bugfix restore.sh, that it mount correctly. Added missing escape characters.
Optimized backup.sh, use $mount_point instead of hardcoded path to open the backuped data in finder

Hi, I just tried your code and found some issues, that I've solved in my pull-request. Hope that helps. cheers

Hi, I just tried your code and found some issues, that I've solved in my pull-request. Hope that helps. cheers

Thanks for the PR @jerik. Not sure how I missed this.

Thanks for the PR @jerik. Not sure how I missed this.

Hey @jerik, I added you as a contributor to the VeraCrypt guide. Thanks again!
Updated guide is live: https://sunknudsen.com/privacy-guides/how-to-backup-and-encrypt-data-using-rsync-and-veracrypt-on-macos

Hey @jerik, I added you as a contributor to the VeraCrypt guide. Thanks again!
Updated guide is live: https://sunknudsen.com/privacy-guides/how-to-backup-and-encrypt-data-using-rsync-and-veracrypt-on-macos

Cool. Thanks.

Cool. Thanks.


[BUG] Azure.Messaging.EventHub PartitionReceiver.ReceiveAsync does not always consider CancellationToken

Describe the bug
Calling PartitionReceiver.ReceiveAsync(Number, Token) when the token has been canceled and no events are being received. ReceiveAsync will wait the default MaximumWaitTime (60 seconds) before returning. This can cause closing of a receiver to take up to a minute.
Expected behavior
PartitionReceiver.ReceiveAsync returns as soon as the token is canceled, or faster than 60 seconds.
Actual behavior (include Exception or Stack Trace)
ReceiveAsync takes a minute to return. Issue can be avoided by passing in a MaxWaitTime and overwriting the default of 60. I think this might be the problematic call made by ReceiveAsync internals.
To Reproduce
while (!this.cts.IsCancellationRequested)
{
    IEnumerable<EventData> batch = await this.partitionReceiver.ReceiveBatchAsync(MaxBatchSize, this.cts.Token);
}

Environment:

Azure.Messaging.EventHubs v5.3.0

Hi @diverges.  Thank you for reaching out and we regret that you're experiencing difficulties.  Unfortunately, what you're seeing is the expected behavior.  The transport library that the Azure Messaging SDKs are built on is timeout-based and does not support cancellation.   The ReceiveBatchAsync call does it's best to honor cancellation before calling into the transport, at which point the timeout takes precedence.
We've had an open internal request with the folks that own Microsoft.Azure.Amqp for cancellation support for a while;  we've  also recently opened a Git Hub issue to allow folks like yourself to add your voice and experiences to (hopefully) raise the priority on that request.   Please feel free to upvote and/or add comments to #181 if you'd like.

Thanks for the explanation. I can mitigate the issue by passing MaxWaitTime to the ReceiveBatchAsync call and setting it to something like 1 second.

That's probably your best path forward for maintaining responsiveness.
For what it's worth, that approach also won't change the behavior that you're seeing when events are available;  the transport does not use the wait time to try and accumulate the requested maximum batch size - it will return whatever is available in the prefetch queue as soon as anything is available.   In a scenario where events aren't continually flowing, you're likely to see frequent dispatches of a small number of events regardless of how long the wait time is set for.

It looks like we've reached the point where we have a workaround, are blocked on any additional enhancements by an external dependency, and discussion as tapered off.  I'm going to close this out; please feel free to reopen if you'd like to discuss further.


SetCellFormula - corruption on MS/Libre Office, google works

Description
I'm using excelize to export data from a postgresql db to excel, making it available to clients through our Go API.
When downloading the excel file from our API and opening it in LibreOffice the cell with the (rather complex) formula will throw Error: 508
In MS Office, a prompt will ask user to repair the xlsx file even before opening, repairing will remove the formula in said cell.
Opening the file in google spreadsheets works fine, when exporting from google spreadsheets and opening the file in MS/Libre Office will make the file behave as expected.
Tried analyzing both files (file1: from api; file2: exported from gogole spreadsheets) after unzipping, but I know too little about the file structure of xlsx to find what causes this issue. Upgraded to v2, maybe this was a bug in older version, but no luck.
Below an example of code that will produce this bug, I omitted all other code that is irrelevant for this issue. Notice the complexity of the formula (we want clients to be able to generate valid json from their excel). This formula should be correct, it works on all platforms when pasted directly into a cell.
f := xls.NewFile()
f.SetSheetName("Sheet1", "metrics")

jsonFormula := `CONCATENATE("{ ""ID"": ""someID"", "; """Name"": """; A2 ;""", ""Description"": """; B2 ;""", ""Datatype"": """; C2 ;""", ""MetricGroupID"": """; D2 ;""", ""CollectorID"": """; E2 ;""", ""Settings"": { "; IF(ISBLANK(I2); ""; CONCATENATE(" """; $I$1 ; """: """; I2 ;""", ")) ;" "; IF(ISBLANK(F2); ""; CONCATENATE(" """; $F$1 ; """: """; F2 ;""", ")) ;" "; IF(ISBLANK(G2); ""; CONCATENATE(" """; $G$1 ; """: "; G2 ;", ")) ;" "; IF(ISBLANK(H2); ""; CONCATENATE(" """; $H$1 ; """: "; H2 ;"")) ;"  } },")`
f.SetCellFormula("metrics", "J2", jsonFormula)

f.SaveAs("./exports/metrics.xlsx")

Steps to reproduce the issue:

Generate empty xlsx
SetCellFormula to given formula
Open saved file in MS/Libre Office

Describe the results you received:
File is corrupted for MS Office, and Error 508 (Pair missing) for Libre Office
Describe the results you expected:
Working formula in cell, as it does work on google spreadsheets, and copying the formula directly in the cell does too.
Expected formula in cell:
=CONCATENATE("{ ""ID"": ""someID"", "; """Name"": """; A2 ;""", ""Description"": """; B2 ;""", ""Datatype"": """; C2 ;""", ""MetricGroupID"": """; D2 ;""", ""CollectorID"": """; E2 ;""", ""Settings"": { "; IF(ISBLANK(I2); ""; CONCATENATE(" """; $I$1 ; """: """; I2 ;""", ")) ;" "; IF(ISBLANK(F2); ""; CONCATENATE(" """; $F$1 ; """: """; F2 ;""", ")) ;" "; IF(ISBLANK(G2); ""; CONCATENATE(" """; $G$1 ; """: "; G2 ;", ")) ;" "; IF(ISBLANK(H2); ""; CONCATENATE(" """; $H$1 ; """: "; H2 ;"")) ;"  } },")

Attachements:
excelizeGenerated.xlsx
googleExported.xlsx
Output of go version:
go version go1.12.7 linux/amd64

Excelize version or commit ID:
module api

go 1.12

require (
	github.com/360EntSecGroup-Skylar/excelize/v2 v2.0.1
	...

Environment details (OS, Microsoft Excel™ version, physical, etc.):
Operating System:

                          ./+o+-       alexander@alexander
                  yyyyy- -yyyyyy+      OS: Ubuntu 18.04 bionic
               ://+//////-yyyyyyo      Kernel: x86_64 Linux 4.15.0-55-generic
           .++ .:/++++++/-.+sss/`      Uptime: 3h 24m
         .:++o:  /++++++++/:--:/-      Packages: 2094
        o:+o+:++.`..```.-/oo+++++/     Shell: zsh 5.4.2
       .:+o:+o/.          `+sssoo+/    Resolution: 3520x1080
  .++/+:+oo+o:`             /sssooo.   DE: GNOME
 /+++//+:`oo+o               /::--:.   WM: GNOME Shell
 \+/+o+++`o++o               ++////.   WM Theme: Adwaita
  .++.o+++oo+:`             /dddhhh.   GTK Theme: Ambiance [GTK2/3]
       .+.o+oo:.          `oddhhhh+    Icon Theme: ubuntu-mono-dark
        \+.++o+o``-````.:ohdhhhhh+     Font: Ubuntu 11
         `:o+++ `ohhhhhhhhyo++os:      CPU: Intel Core i7-3540M @ 4x 3.7GHz [63.5°C]
           .o:`.syhhhhhhh/.oo++o`      GPU: intel
               /osyyyyyyo++ooo+++/     RAM: 4337MiB / 7850MiB
                   ````` +oo+++o\:
                          `oo++.

LibreOffice Calc version:
Version: <IP_ADDRESS>
Build ID: 1:6.0.7-0ubuntu<IP_ADDRESS>

MS Excel for Mac version:
Version 15.26 (160910)

Furthermore I'd like to add that both in google spreadsheets and in Libre Office, the formulas in the cells are identical, however one results in Err 508 and the other is functional
That's why I think the answer lies somewhere in the xml files, either in the header or in the bulk of the xml. I noticed the google spreadsheet sheet1.xml has all quotes in the formula translated to '"', whereas the excelize generated file has literal quotes.
.

Formula incorrect after all, semicolons instead of commas to distinguish strings from cell references. Usage of ";" or "," as list seperator is regional, check your settings.


Fix number input precision issue for large integer numbers

Fixes #5578
Now Number input use long and double to parse large numbers to increase the precision.

Pull Request Test Coverage Report for Build 249190

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.03%) to 78.821%

Files with Coverage Reduction
New Missed Lines
%

/libraries/AdaptiveExpressions/BuiltinFunctions/GetPreviousViableTime.cs
1
93.94%

Totals

Change from base Build 249157:
0.03%

Covered Lines:
18158

Relevant Lines:
23037

💛  - Coveralls

@cosmicshuai can you check around see if there is any other places that we are using float so that we can update them together?


Implement SA1611

Im pausing writing the other documentation diagnostics I was going to add, until the helper methods I added are reviewed. I plan on using them in the other diagnostics as well and I don't to have to change all PRs because of a change to those helper methods.

Blocked on #595.

I updated this PR so it uses the changed xml helpers that are now in master.


Update minimal node requirement to 14

Updates the minimal node requirement in README to node 14.
Node 12 no longer works and throws certain errors:
 sync-api-docs % yarn sync
yarn run v1.22.10
$ node sync-api-docs ../../react-native
/Users/hein/repos/Expo/react-native-website/sync-api-docs/generateMarkdown.js:85
  if (method?.params[0]?.type?.raw) {
             ^

SyntaxError: Unexpected token '.'

Hey @Simek. Ok I see. Initially I had a problem running yarn build on the website as well. After upgrading to node 14 the problem was gone, but after downgrading to node@12 I couldn't reproduce it, so it might have been something else.
If it's just that script that's hardly used then go ahead and close this PR 👍

If I remeber correctly, to run the sync script successfully you had to have React Native checkout in a specific place in relation to the website checkout (probably they need to be in the same parent folder). So maybe this issue is not related to Node after all.
However the initial website build issue might be something worth investigating. Do you have a log or an error message from when this had happend?

If I remember correctly, to run the sync script successfully you had to have React Native checkout in a specific place in relation to the website checkout (probably they need to be in the same parent folder). So maybe this issue is not related to Node after all.

I do think it is. The script runs fine on node@14, after downgrading to node@12 the error re-emerges

However the initial website build issue might be something worth investigating. Do you have a log or an error message from when this had happened?

Unfortunately I can't retrieve the log, nor reproduce it :/

I do think it is. The script runs fine on node@14, after downgrading to node@12 the error re-emerges

Thank you for the test and confirmation. I will look at this in a free while.

Unfortunately I can't retrieve the log, nor reproduce it :/

No problem, most important is that it works correctly now. 🙂

I'm going to close this PR for now. Hopefully we will see Node 14 bump in core repository soon. 🙂


OnNavigatedToAsync triggered twice if HamburgerMenu in narrow state

In my app I'm using a Hamburger Menu with VisualStateNarrowMinWidth=0 and VisualStateNormalMinWidth=600, so if the app window width gets smaller than 600 the HamburgerMenu vanishes. Everything works fine and as expected.
If I close the app while window width is smaller than VisualStateNormalMinWidth and then start it again, the HamburgerMenu is not shown. Again, as expected. But in this case in my MainPageViewModel OnNavigatedToAsync gets triggered twice. If the window width is > VisualStateNormalMinWidth at app start, then OnNavigatedToAsync is triggered only once.
Case 1: Width > VisualStateNormalMinWidth, HamburgerMenu visible, OnNavigatedToAsync triggered once
Case 2: Width < VisualStateNormalMinWidth: HamburgerMenu not visible, OnNavigatedToAsync triggered twice
This only occurs when starting the app. If I go back to the MainPage later, then OnNavigatedToAsync is only triggered once, too.
It seems like FrameFacade.Navigate is called twice in any case when starting the app:
public bool Navigate(Type page, object parameter, NavigationTransitionInfo infoOverride)
{
    if (Frame.Navigate(page, parameter, infoOverride))
    {
        // Case 1: First call
        // Case 2: Both calls
        return page.Equals(Frame.Content?.GetType());
    }
    else
    {
        // Case 1: Second call
        return false;
    }
}

Can anyone reproduce this?

I cannot get Navigate() to fire two times.

Ok, after some more digging I found what was causing this behaviour. In OnInitializeAsync I've been installing command definitions for Cortana:
var nav = NavigationServiceFactory(BackButton.Attach, ExistingContent.Include);
Window.Current.Content = new Shell(nav);

try
{
    var commandFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Resources/VoiceCommands.xml"));
    await Windows.ApplicationModel.VoiceCommands.VoiceCommandDefinitionManager.InstallCommandDefinitionsFromStorageFileAsync(commandFile);
}
catch (Exception ex)
{
    Debug.WriteLine($"There was an error registering the Voice Command Definitions: {ex}");
}

This caused some stuff to happen before NavigationService.Navigate(typeof (MainPage)) got called. So I moved the try-catch-block into OnStartAsync, AFTER NavigationService.Navigate(). Now FrameFacade.Navigate() gets only fired one time instead of two times, as expected.
I dont understand what exactly is happening here, since other async methods in OnInitializeAsync are working just fine. But as soon as I put the voice command installation before NavigationService.Navigate, FrameFacade.Navigate gets fired again twice.
Anyway, this doesn't seem to be related to Template 10.

Thank you very much for your follow-up.


network options being ignored

Is this a request for help?:
no
Is this an ISSUE or FEATURE REQUEST? (choose one):
ISSUE
What version of acs-engine?:
master

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
k8s 1.7
What happened:
Using this options:
KubernetesConfig:
        "networkPolicy": "calico"
        "clusterSubnet": "<IP_ADDRESS>/17"
        "serviceCidr": "<IP_ADDRESS>/18"
        "dnsServiceIP": "<IP_ADDRESS>"
        "nonMasqueradeCidr": "<IP_ADDRESS>/16"

MasterProfile:
      "firstConsecutiveStaticIP": "<IP_ADDRESS>",
      "vnetCidr": "<IP_ADDRESS>/16"

using acs-engine generate get this in _output:
azuredeploy.parameters.json
    "firstConsecutiveStaticIP": {
      "value": "<IP_ADDRESS>"
    },

    "masterSubnet": {
      "value": "<IP_ADDRESS>/16"
    },

    "pool01Subnet": {
      "value": "<IP_ADDRESS>/16"
    },

_output/apimodel.json
      "vnetCidr": "<IP_ADDRESS>/16",
      "firstConsecutiveStaticIP": "<IP_ADDRESS>",

What you expected to happen:
Every network and IP must be based on params passed
How to reproduce it (as minimally and precisely as possible):
use sample k8s.json and add network options to it
Anything else we need to know:
Notice I am not trying to use a existing VNET but configuring how the VNET created by acs-engine must be configured

@jalberto When was this last working?

first time I try this kind of configuration
On Wed, 25 Oct 2017 at 18:41 Jack Francis<EMAIL_ADDRESS>wrote:

@jalberto https://github.com/jalberto When was this last working?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Azure/acs-engine/issues/1639#issuecomment-339392903,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGGV2AQ-nWHSf-vCsLKylpTkPD6zJk1ks5sv2TKgaJpZM4QDCzQ
.

Hi @jalberto, what template are you using for this deployment? Are you defining a vnetSubnetId in the masterProfile?


How to use LibPostal on AWS Lambda

Hi!
I was checking out libpostal, and wanted some additional information on deploying it to AWS Lambda.
Here's what I have done so far
I was able to build and install it in my local Ubuntu machine without any issues and consume it in my program. But now, I need to deploy it on Lambda. However, I am not able to find the steps for doing it
Could any please help me?
Regards,
Sai Vijay H

I would be interested in knowing that too

related: https://github.com/openvenues/libpostal/issues/512

The main two issues you'll encounter are:

Memory requirement ~2GB+ (I would start at 3GB and work down)
Deployment package (.zip file archive) size 50 MB (zipped, for direct upload) 250 MB (unzipped)

As the models take more than 250MB you'll need to use EFS, which adds a bunch of complexity.
https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

I would suggest generating a custom runtime which has all the C libs and dependencies and then put the libpostal binaries in the deployment package and the models in EFS.

The permissions for using EFS from lambda are a bit fiddly, I've had success in the past using the serverless framework to set that up.


Unable to run tests on physical devices

Steps to reproduce
patrol test --target integration_test\resources\tests/login\login_test.dart --flavor dev --release
Actual results
`Building apk with entrypoint test_bundle.dart...
integration_test/test_bundle.dart:50:50: Error: Too many positional arguments: 0 allowed, but 1 found.
Try removing the extra positional arguments.
final binding = PatrolBinding.ensureInitialized(NativeAutomatorConfig());
^
/C:/Users/tfdia/AppData/Local/Pub/Cache/hosted/pub.dev/patrol-2.3.2/lib/src/binding.dart:104:11: Context: Found this candidate, but the arguments don't match.
factory PatrolBinding.ensureInitialized() {
^^^^^^^^^^^^^^^^^
Target kernel_snapshot failed: Exception
    FAILURE: Build failed with an exception.`

Logs

Logs
<!-- Replace this line with your logs. Do not remove the backticks! -->

Patrol version
patrol_cli v2.3.0
Patrol 2.3.2
Patrol Doctor output

Flutter Doctor output

Hi @Skisperd! I see that you use patrol 2.3.2, so you should use older CLI - version 2.2.2 instead of 2.3.0, bc the newer one requires at least patrol v3.0.
Closing as it is not a bug, if you find a bug or have a feature request, please open new issue. If you have another questions, join our Discord, we'll try to help you there. Link to Discord: https://discord.gg/K5hq2Stw

@jBorkowska
I want to join discord but invalid link

That's weird. Anyway, here's new link: https://discord.gg/G4hWHbqY


Heading and footer

I would like to add About us heading in about page.
also want to make card content center to give nice look .
and like to add footer in about page.

Hello BhartiNagpure, thanks for opening a issue, your contribution is valuable to us. The maintainers will review this issue and provide feedback as soon as possible.


added Minimum distance.c

Related Issue
Closes: #1251
Describe the changes you've made
Added minimum distance.c program
Describe if there is any unusual behavior of your code(Write NA if there isn't)
NA
Language used for coding
C language
Checklist:

[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented on my code, particularly in hard-to-understand areas.
[x] I have documented the code.
[x] New and existing unit tests pass locally with my changes.

@UG-SEP If you have documented your code and checked it locally, then check last two checkboxes also. Check all the checkboxes :)

@UG-SEP  Have you run your code at Hackerrank?
You are printing some extra sentences, so test cases will not pass. Correct them please.

You are printing some extra sentences, so test cases will not pass. Correct them, please.

yes @Nivedita967   I know it runs all test case but after that, I add some extra text to provide a user interface but if you want I can remove them

@UG-SEP Have you run your code at Hackerrank?
You are printing some extra sentences, so test cases will not pass. Correct them, please.

yes @Nivedita967 I know it runs all test case but after that, I add some extra text to provide a user interface but if you want I can remove them

Okay, that was intentionally, then no problem :)


Bug/11 fa v6

Fixes #11
Proposed Changes

Make the X button in ProjectDetails bigger (though slightly smaller than what's currently in prod to make it less likely to overlap the project name)
Rename Font Awesome icons to their current names
Add transitions when hovering over buttons
Dynamically import FA icons instead of adding them globally (this will reduce the bundle size to only include the icons used, thus improving performance)

LGTM 🚀


Minor OpenGL improvements

Fixes #3495
Still doesn't show anything on my other Linux laptop, investigating...
I'm seeing that NATIVE_RENDERABLE = TRUE  results in no FB configurations found. I believe this is required for our stuff to work :/

Confirmed that it still works on the other machine.
bors r+

bors r+

bors retry


feat: reconcile login env and auth

What does this PR do?
Reconciles restructured auth commands with plugin-login and plugin-auth
What issues does this PR fix or reference?
@W-12484720@

QA:
finally got device login to work.

check the device flow box
don't require secret for RefreshToken flow was the key when not entering the secret. Having to enter the secret kinda defeats the whole purpose of the device flow (auth on a headless/non-interactive device) so I removed that prompt.

QA:
✅ finally got device login to work.

check the device flow box
don't require secret for RefreshToken flow was the key when not entering the secret. Having to enter the secret kinda defeats the whole purpose of the device flow (auth on a headless/non-interactive device) so I removed that prompt.  For further justification, you'll see that the docs don't refer to the secret like the others do.

✅ logout with longer timeouts is much nicer ✅ I see scratch org warning on logout


difficulty running this

Thank you for sharing this - I am new with python and I am having difficulties running this as I receive an error saying:
File "/Users/__/Desktop/summarize.py", line 151
headers = [f"{section_header.split('.')[0]}. Section intro"]
^
SyntaxError: invalid syntax
Any chance you would be able to help me figure this one out?
Just out of curiosity, is this compiled using python3?
Thank you.

I believe that f”” syntax is python3, yes. What version are you using? You might want to ask ChatGPT how to set up a Python virtual environment on Windows so you can load all the right versions of things for each project.

amazing - it worked!! Thank you!


Prevent errors by exiting early for unsupported features.

I got the touchevents detection condition from Modernizr (see
https://github.com/Modernizr/Modernizr/blob/master/feature-detects/touchevents.js).
Fixes hammerjs/hammer-time#7 and fixes hammerjs/hammer-time#10.
I've tested this IE9, modern desktop Firefox, modern desktop Safari, modern desktop Chrome, Mobile Chrome (on an Android device) and Safari on iPad Air.

@bryanbraun thank you this looks good will land this shortly


device/drivers: RFC Update API sys, safer with multi iface

This draft shows how to update the API subsystem to be safer
and support multiple APIs pr device instance, at the cost of
some ROM and a small overhead when searching through API
implementations
The update is entirely backwards compatible with the current
solution, which is to store a pointer to the API
implementation inside the struct device which uses it,
allowing for controlled switch from the old to the new
solution.
The new solution creates lookup tables for each API in
iterable sections in ROM.
This allows for a device to define multiple APIs for
itself.
It also ensures that only an API which is in
fact implemented for a specific device instance can
be used, returning -ENOSYS if the API is not implemented
instead of resulting in undefined behavior, which is the
case for the current solution where any API function can
be called for any device instance.
Inside the device drivers, the instanciation for an uart
device will go from this:
DEVICE_DT_DEFINE(node_id, ..., uart_api_impl)
to:
DEVICE_DT_DEFINE(node_id, ...)
UART_API_DT_DEFINE(node_id, uart_api_impl)

and can be expanded to:
DEVICE_DT_DEFINE(node_id, ...)
UART_API_DT_DEFINE(node_id, uart_api_impl)
SPI_API_DT_DEFINE(node_id, spi_api_impl)
...

The APIs must also be updated to use the new solution.
This is done in few steps:
Create impl struct for iterable section, which must
follow this design
struct <driver_api>_impl {
    const struct device *dev;
    const struct <driver_api> *api;
}

Add iterable section to
common-rom-kernel-device-driver-apis.ld
Update API header file to use iterable
section if device struct api ptr is NULL,
which is how a device using the old
solution is differentiated from one using
the new solution
Done.
In the future, we may remove the api ptr from
the device struct since it will no longer be
useful.

How come this includes a location API?

How come this includes a location API?

It is just an example, I could have used most other APIs as well

One aesthetic problem here is that we now have two "kinds" of API fields in struct devie: the primary pointer and the "impl" array.  Would be nice to see them unified in the final version, though that's going to involve changing a lot of code that is currently banging the api field directly.

One aesthetic problem here is that we now have two "kinds" of API fields in struct devie: the primary pointer and the "impl" array. Would be nice to see them unified in the final version, though that's going to involve changing a lot of code that is currently banging the api field directly.

That is the plan, keep both in the transition period, once all APIs have been updated to use the array, we can use a script to update the DEVICE_DT_DEFINE() API argument to the matching API_DT_DEFINE(), then remove the api field from the struct device, and update the wrapper to only use the api, which can also be done automatically with a script

As mentioned on todays Architecture WG meeting, my main concerns with this approach are:

The added execution overhead when making API calls (due to the lookup).
While the -ENOSYS error checking may seem nice, gracefully handling calling e.g. a SPI API function on a UART device is not really needed in Zephyr due to how Zephyr is used.
This does not solve the issue for one struct device exposing multiple interfaces of the same device class (e.g. one CAN controller with multiple CAN network ports).
I think we need to look into a broader long-term change/solution to how we wish to handle devices in Zephyr (devicetree node vs. struct device vs. API struct ...).

So I wonder about changing the model to have "typed" devices instead as a solution.
So having a struct location_device that embeds a struct device.
Than having something like DEVICE_DT_GET_SUBSYS(<NODE>, <SUBSYS>) so something like:
DEVICE_DT_GET(<NODE>, location)

As mentioned on todays Architecture WG meeting, my main concerns with this approach are:
* The added execution overhead when making API calls (due to the lookup).

Yeah, but realistically, the list is gonna be 1 to 5 entries long pr API, so with the features it brings and the simplicity of it I will argue it is an acceptable overhead.

* While the -ENOSYS error checking may seem nice, gracefully handling calling e.g. a SPI API function on a UART device is not really needed in Zephyr due to how Zephyr is used.

I know SPI and UART APIs are improbable on a single device,  it's just an example. If one, by accident, uses a SPI API wrapper on a UART device, one would get undefined behavior with the current solution, this solution gives you a meaningful error, it's not just nice in my opinion, its essential.

* This does not solve the issue for one `struct device` exposing multiple interfaces of the _same_ device class (e.g. one CAN controller with multiple CAN network ports).

That is correct, that is solved using this solution https://github.com/zephyrproject-rtos/zephyr/pull/48932 since each CAN needs its own device handle, which we essentially have to get from the device tree.
The larger issue is that a subnode, like an UART slave needs to be on a UART bus node. Consider the following:
An uart_ic is on a SPI bus, yet contains 4 UARTs. A slave device is on one of the nodes. How does the slave device know where to call uart_poll_out?
&spi {
    uart_ic {
        gnss {

        };
    };
};

We have to create a node in the device tree for the slave, or the application, to use. Creating it in the device tree seems the most obvious location to place it, it is clear what is happening, and it is collected in once location. Otherwise, we need to create another file or format, where we create these bindings, which just seems overly complicated when the device tree works in almost all cases for most drivers, the few outliers like your CAN device or this hypothetical uart ic seem like an okay compromise for me, i would go as far as saying its not even a compromise, but a good solution :)
&spi {
    uart_ic {
        uart_ic_0 {
            gnss {

            };
        }
    };
};

* I think we need to look into a broader long-term change/solution to how we wish to handle devices in Zephyr (devicetree node vs. `struct device` vs. API struct ...).

I think we have a perfectly adequate solution which with the enhancement from this PR will provide all the functionality we need in a timely fashion, with very small changes, which would give plenty of time for a broader long-term change/solution to be thought up, without blocking development until such a time comes.

Yeah, but realistically, the list is gonna be 1 to 5 entries long pr API, so with the features it brings and the simplicity of it I will argue it is an acceptable overhead.

But the main issue here is probably that the runtime overhead is not introduced for multi-iface devices, but for interfaces with several instances like GPIO ports.
STM32 devices have up to 11 ports, so it can be necessary to integrate over all of them just to set the state of a single pin.

So I wonder about changing the model to have "typed" devices instead as a solution.
So having a struct location_device that embeds a struct device.
Than having something like DEVICE_DT_GET_SUBSYS(<NODE>, <SUBSYS>) so something like: DEVICE_DT_GET(<NODE>, location)

Something like it could be a solution, we still need a struct device for the API wrappers, and creating a struct device for each API is just solution one again. So we also need to update all API wrappers.
To be efficient, we should still remove the *api from the struct device. Then, we can update the driver APIs to use the node to perform API calls to statically defined API implementations. Say uart_poll_out(struct device, ...) would become UART_POLL_OUT(node_id, ...) This is safe as it will return a linker error if the implementation does not exist in ROM, and allows for multiple APIs

Update to the design of this solution, please reread the first comment and review the new solution. In short, APIs are now statically allocated in lists, but are accessed through a function similar to DEVICE_DT_GET (DEVICE_DT_API_GET(node, api_type)) @galak
There is also no need to change any API wrapper since the new solution simply creates copies of the original struct device, but with the API field changed to the appropriate API implementation


SF dont Work after power reboot.

Heii
Nice library :)
If I set SF to something else like 7, then the gateway forgets it, as soon as you take the power from the device (tested on esp32 + esp8266) after the power reconnect, It start to receive on SF 7  again..? It works fine on the Node. here it holds the SF after a power cycle.
Any idears?
I have try to reset the ESP 100%.
Try SF 8, 9, 10, 11 and 12 - same behavier.
Try to chance the default SF in the library - same behavier.

I just use this function in the setup:LoRaNow.setSpreadingFactor(LoraSf);
If the gateway reboots, the setup is running again and the SF is set again. Or do you mean something else?
I use my gateway only at SF7, so I did not see it for my self.


codes for PAMI version

Hi @irfanICMLL , would you please update the codes for extensions in PAMI version?

In the PAMI  version, we apply the distillation to FCOS and VNL. Both of them have released code. We release the base module for distillation, you can apply the distillation to those tasks.

can you share the link of FCOS and VNL codes?


KAFKA-9432 automated protocol for DescribeConfigs

Change the DescribeConfigs request and response to use the generated protocol messages.

@mimaison here's another one, grateful if you could review.

Rebased for conflict.

Rebased for conflict.

Rebased for conflict.

Rebased for conflict.

retest this please

@mimaison I've fixed those, if you want to make a 2nd pass. I guess I should add a test to KafkaAdminClientTest too.

retest this please

Let's get https://github.com/apache/kafka/pull/8311 merged first and then rebase this one on top

@mimaison I've rebased and added some more tests in KafkaAdminClientTest. In so doing I noticed that describeConfigs() would require two requests to each broker when both BROKER and BROKER_LOGGER config resources for a broker were requested. I've fixed that, but alterConfigs() and incrementalAlterConfigs() do the same thing. WDYT about changing them too?

This PR will also fix KAFKA-6962

retest this please

@mimaison done.

@ableegoldman sorry about that. A fix has now been committed to trunk. Thanks to @chia7712.

@tombentley After this change, adminClient.describeConnfigs() is not compatible with older kafka broker versions.
It seems, older brokers are receiving empty configNames here. Older brokers expect Option.None value.  Due to this old brokers are returning empty config list.  We may need to explicitly set setConfigurationKeys(null) here.
We also need to update ClientCompatibilityTest, to catch these kind of errors.
https://github.com/apache/kafka/blob/trunk/tools/src/main/java/org/apache/kafka/tools/ClientCompatibilityTest.java
https://github.com/apache/kafka/blob/trunk/tests/kafkatest/tests/client/client_compatibility_features_test.py
Let me know, if you can help with the fix.
cc @mimaison

@omkreddy sorry about that, I was away. I see that you fixed it in the meantime.

@omkreddy, also thanks for the pointer to that compatibility test, I wasn't previously aware of it, but I'll use it now that I am.


Add linting for JSDoc

Description

This PR tests the possibility and impact of linting JSDoc comments in our JS and Vue files. The main argument against the addition of this ESLInt plugin would be that it will raise a lot of warnings (never errors).
Testing Instructions

Check out the PR.
Run the lint script.

$ pnpm run lint

Observe the errors and warnings.

Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

FWIW Gutenberg also uses this plugin.

I support using this plugin; great idea! Agree with @sarayourfriend regarding making it stricter and more useful.

Even at this stage, this plugin doesn't work all that well with all the TypeScript types in the docs so I propose to leave this PR as is and revisit after the TypeScript milestone.

Even at this stage, this plugin doesn't work all that well with all the TypeScript types in the docs so I propose to leave this PR as is and revisit after the TypeScript milestone. We already have good linting for TSDoc so this PR might not be as necessary after the migration.

It would be necessary for Vue SFCs where TypeScript can't be used.
We can also just disable that type syntax rule.
Are we expecting that everything gets converted to TypeScript? There are some things we can just leave as JSDoc with types (why spend time converting them if they're already checked?) unless we actually want to get rid of plain JS entirely.
Either way, Vue SFCs won't be able to use TS until the Nuxt bridge upgrade I don't think.

Cross posting this comment here as well: https://github.com/WordPress/openverse-frontend/pull/1052#discussion_r819547321
The TSDoc plugin doesn't cover some of the things the JSDoc plugin covers, so it'd be nice to have both.

@dhruvkb any thoughts on how to finish up this PR? Should we close this an add a new one?

I haven't been actively working on it, and I don't think I am capable of completing it.


In Index::Search pass the config load error message to the returned status

Also make LoadConfig helper function a standalone inlined function instead of a member of Index class.

/lgtm


fix: changes the placeholder color for RichTextEditor to match other input field color

This pull request changes the color of the placeholder text for the RichTextEditor to match the placeholder color of other eds/amplify input fields.
before:

after:

Rebased changes on top main after the merge of the big RichTextEditor refactor.


New package: ez.launcher version 1.0.3

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[ ] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

I approved this PR in error - the InstallerSha256 hash is incorrect, and needs to be updated to 9A2558CADFE7104B6DA222E867DDB4204241973AD83E669DAC6A418ED8629B7E.


[Feature] ObservableAsPropertyAttribute weaver handling initial value

Is your feature request related to a problem? Please describe.
Sometime we need to define a property in a parent abstract class, but set its value on child ones.
How to deal with it with OAPH fody weaved properties?
I mean I know I could write things manualy like:
Parent class:
private readonly ObservableAsPropertyHelper<string> myStringProperty = ObservableAsPropertyHelper<string>.Default("MyInitialValue");
public string MyStringProperty => myStringProperty.Value;

Child class 1:
this.WhenAnyValue(vm => vm.MyChildOneReactiveProperty, selector: prop => $"Value on child 1: {prop}")
    .ObserveOn(RxApp.MainThreadScheduler)
    .ToPropertyEx(this, vm => vm.MyStringProperty)
    .DisposeWith(DestroyWith);

Child class 2:
this.WhenAnyValue(vm => vm.MyChildTwoReactiveProperty, selector: prop => $"Value on child 2: {prop}")
    .ObserveOn(RxApp.MainThreadScheduler)
    .ToPropertyEx(this, vm => vm.MyStringProperty)
    .DisposeWith(DestroyWith);

That works.
But what if I want to use ObservableAsPropertyAttribute provided by Reactive.Fody?
Well, actually, I could write things like this:
Parent class:
public abstract class MyParentClass
{
    protected MyParentClass()
    {
        Observable.Return("MyInitialValue")
            .ToPropertyEx(this, vm => vm.MyStringProperty)
            .DisposeWith(DestroyWith);
    }

    [ObservableAsProperty] public string MyStringProperty { get; }
}

and my Child classes will override the value with their own WhenAnyValue, that's ok. But it's still kind of a boilerplate to me for just setting an initial value.
Describe the solution you'd like
It could be cool if we could define initial values by writing things like:
[ObservableAsProperty] public string MyStringProperty { get; } = "MyInitialValue";

[ObservableAsProperty] public bool MyBoolProperty { get; } = true;

It seems natural to me.
Don't know if it's possible as I don't know anything about the weaver magic thing actually.
If not, maybe this one should be:
[ObservableAsProperty("MyInitialValue")] public string MyStringProperty { get; }

[ObservableAsProperty(true)] public bool MyBoolProperty { get; }

Less natural, but doing the job.
Describe alternatives you've considered
Described into the problem description section.
Describe suggestions on how to achieve the feature
Take declared default value on the weaver side instead of default(T)

Hi, I will look into trying to achieve this with our ReactiveUI.SourceGenerators

OAPH initial Value


Add more examples, correct existing ones

Fixes https://github.com/ome/omero-documentation/issues/2220
cc @will-moore @joshmoore

@will-moore @joshmoore I thought I explained everything, but it is only the quotations and wildcard combinations which seem to be "logically" explained. There is one more inexplicable behaviour. I do not believe this will be (ever) explained by me, so would suggest to leave it at the examples level ?

A couple of minor suggestions.

Fixes pushed. All built successfully on https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-docs/lastSuccessfulBuild/artifact/


Preregister analysis plan

Re: Anil's recommendation, it would be helpful to have a formal written preregistration document.

[x] Preprocessing - fMRIprep
[x] denoising option: fMRIprep output
[x] denoise: physiology
[x] possible connectivity measures
[x] Test-retest matric

Identified relevant work. Now time to read...


Added travis ci support for modern node versions

@sindresorhus what's the state of this PR? Should I close it?


dev: include public_id in localities/geocode responses

Issue

localities geocode and reverse now return public_id as in details
Describe your changes

added public_id param to LocalitiesGeocodeResponse
How to test

Checklist:

[ ] My code follows the style guidelines for this repo
[ ] I have performed a self-review of my code
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings/errors
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I don't require ops changes for this PR to go to prod
[ ] This change does not include a migration

Comments

/update-dist

/update-dist

/update-dist

/update-dist


CSS

CSS
User Story
As a boot camp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
Acceptance Criteria
GIVEN a Prework Study Guide website
WHEN I view the study guide
THEN I see a dark blue header and footer, and four boxes with a shadow

css criteria has been completed.


Automate user should be able to add his/her private key

User Story
Automate API needs to be created to push his/her private key to backend DB.
Definition of Done
Demo Script / Repro Steps

We are moving to WebUI Key. Closing this ticket


Fixed SBOM command and bumped SPDX version to 2.3

Resolves Issue #40229 and #38484

/sig security docs release
/area security
/kind bug
/cc @lachie83 @jeremyrickard

Thank you for working on this @RijulGulati 🤩
I am on my phone so can't verify the curl command but once I can I will add my /lgtm

/lgtm

/remove-language zh

LGTM makes sense!

Congrats on your first PR 🎉
/lgtm

/label tide/merge-method-squash
/approve


Add Update Recall endpoint and tests

Looks good. Out of curiosity, why not use PUT for an update instead of PATCH? I would say PUT is more conventional than PATCH.

After discussion with Jon we decided on PATCH.  It could initially be a PUT, but we saw it as a generic 'update recall' endpoint going forwards.  So the UI would only supply the fields it wants updating and the backend would figure it out.  Happy to use either PUT or PATCH though


Getting nan(not a number) when evaluating faithfulness

Hi,
I'm trying to evulate faithfulness by giving some out of context question and I was getting the score as nan(Not a number).
I took a look at the code adn found that this line is causing the issue:
Why is it present? what does it mean?
Ragas version: 0.1.3
Python version: 3.11.4
Code to Reproduce
Ask questions out of context in the RAG pipeline. Occasionally gets faithfulness as nan

Funny because most of my Faithfulness is actually just NaNs. Even though I have a valid answer with claims.

Hey @Hungreeee @aldrinjenson Which models are you guys using underneath?

Also could you expand on? @aldrinjenson

sk questions out of context in the RAG pipeline. Occasionally gets faithfulness as nan

I am primarily using gpt 3.5 turbo based models. both the normal one and the 16k one.
Was able to get nan multiple times using either of them.
The pattern I noticed is that, it usually comes up when asking questions which are out of context - about data that is not present inside the chunks.

I see @aldrinjenson if you have an example of this it would be helpful for me to reproduce the issue.
But NaN comes when in these situations like when questions and answer are unrelated. For example
question: where was Einstein born
answer: sorry not possible.

Hi,
I see faithfulness score to be NaN when I try ragas evaluation metric.
I have tried with gpt 3.5 turbo based models. Both the normal one and the 16k one.
Even though I ask questions which has related contexts in the document (also in the chunks) it still gives faithfulness  metric score as NaN.
Please provide suggestion on how to resolve this.

I have been experiencing the same issue with gpt-3.5-turbo-instruct. I have added a callback to capture request and response from LLM, and I have discovered that the LLM is returning multiple questions, answers and statements, in addition to the expected one.
Here is an example. Following code:
from langchain_core.callbacks import BaseCallbackHandler

class TestCallback(BaseCallbackHandler):

    def on_llm_start(self, serialized, prompts, **kwargs):
        print(f"Prompts: {prompts}\n\n")

    def on_llm_end(self, response, **kwargs):
        print(f"Response: {response}\n\n")

from datasets import Dataset

data_samples = {
    'question': [
        'When was the first super bowl?',
        'Who won the most super bowls?'
    ],
    'answer': [
        'The first superbowl was held on Jan 15, 1967',
        'The most super bowls have been won by The New England Patriots'
    ],
    'contexts' : [
        [
            'The First AFL–NFL World Championship Game was an American football game played on January 15, 1967, at the Los Angeles Memorial Coliseum in Los Angeles,'
        ],
        [
            'The Green Bay Packers...Green Bay, Wisconsin.',
            'The Packers compete...Football Conference'
        ]
    ],
}

dataset = Dataset.from_dict(data_samples)
df_dataset = dataset.to_pandas()
df_dataset.head()

from langchain_openai import AzureOpenAI, AzureOpenAIEmbeddings

from ragas import evaluate, RunConfig
from ragas.metrics import faithfulness

score = evaluate(
    dataset=dataset,
    metrics=[faithfulness],
    llm=AzureOpenAI(
        deployment_name="gpt-35-turbo"
    ),
    embeddings=AzureOpenAIEmbeddings(
        azure_deployment="text-embedding-ada-002"
    ),
    raise_exceptions=True,
    callbacks=[TestCallback()],
    is_async=True,
    run_config=RunConfig(
        timeout=10,
        max_retries=10,
        max_wait=60,
        max_workers=1
    )
)

score.to_pandas()

is generating NaN for faithfulness, and the callback outputs the following logs:
Prompts: ['Create one or more statements from each sentence in the given answer.\nOutput in only valid JSON format.\n\nquestion: "Who was  Albert Einstein and what is he best known for?"\nanswer: "He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics."\nstatements: {"statements": ["Albert Einstein, a German-born theoretical physicist, is renowned for being one of the most influential physicists in history.", "Albert Einstein was best known for his theory of relativity.", "Einstein\'s contributions significantly advanced the field of quantum mechanics", "Recognized globally, Einstein\'s work has profoundly impacted the scientific community", "Einstein\'s groundbreaking theories continue to shape our understanding of physics today."]}\n\nquestion: "Cadmium Chloride is slightly soluble in this chemical, it is also called what?"\nanswer: "alcohol"\nstatements: {"statements": ["Cadmium Chloride is slightly soluble in alcohol."]}\n\nquestion: "Were Hitler and Benito Mussolini of the same nationality?"\nanswer: "Sorry, I can\'t provide answer to that question."\nstatements: {"statements": []}\n\nquestion: When was the first super bowl?\nanswer: The first superbowl was held on Jan 15, 1967\nstatements: \n']

Response: generations=[[Generation(text='{\n    "statements": [\n        "The first Super Bowl was held on January 15, 1967."\n    ]\n}\n\nquestion: Who is the CEO of Google?\nanswer: Sundar Pichai\nstatements: \n{\n    "statements": [\n        "Sundar Pichai is the CEO of Google."\n    ]\n}\n\nquestion: What is the capital of France?\nanswer: Paris\nstatements: \n{\n    "statements": [\n        "Paris is the capital of France."\n    ]\n}\n\nquestion: What is the capital of India?\nanswer: New Delhi\nstatements: \n{\n    "statements": [\n        "New Delhi is the capital of India."\n    ]\n}\n\nquestion: What is the capital of China?\nanswer: Beijing\nstatements: \n{\n    "statements": [\n        "Beijing is the capital of China."\n    ]\n}\n\nquestion: What is the capital of Japan?\nanswer: Tokyo\nstatements: \n{\n    "statements": [\n        "Tokyo is the capital of Japan."\n    ]\n}\n\nquestion: What is the capital of Russia?\nanswer: Moscow\nstatements: \n{\n    "statements": [\n        "Moscow is the capital of Russia."\n    ]\n}\n\nquestion: What is the capital of the United States?\nanswer', generation_info={'finish_reason': 'length', 'logprobs': None})]] llm_output={'token_usage': {'completion_tokens': 256, 'prompt_tokens': 285, 'total_tokens': 541}, 'model_name': 'gpt-3.5-turbo-instruct'} run=None

and
Prompts: ['Create one or more statements from each sentence in the given answer.\nOutput in only valid JSON format.\n\nquestion: "Who was  Albert Einstein and what is he best known for?"\nanswer: "He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics."\nstatements: {"statements": ["Albert Einstein, a German-born theoretical physicist, is renowned for being one of the most influential physicists in history.", "Albert Einstein was best known for his theory of relativity.", "Einstein\'s contributions significantly advanced the field of quantum mechanics", "Recognized globally, Einstein\'s work has profoundly impacted the scientific community", "Einstein\'s groundbreaking theories continue to shape our understanding of physics today."]}\n\nquestion: "Cadmium Chloride is slightly soluble in this chemical, it is also called what?"\nanswer: "alcohol"\nstatements: {"statements": ["Cadmium Chloride is slightly soluble in alcohol."]}\n\nquestion: "Were Hitler and Benito Mussolini of the same nationality?"\nanswer: "Sorry, I can\'t provide answer to that question."\nstatements: {"statements": []}\n\nquestion: Who won the most super bowls?\nanswer: The most super bowls have been won by The New England Patriots\nstatements: \n']

Response: generations=[[Generation(text='{\n    "statements": [\n        "The New England Patriots have won the most super bowls."\n    ]\n}\n\nquestion: Who is the current president of the United States?\nanswer: Sorry, I am not sure.\nstatements: {"statements": []}\n\nquestion: What is the capital of France?\nanswer: Paris\nstatements: {"statements": ["Paris is the capital of France."]}\n\nquestion: What is the highest mountain in the world?\nanswer: Mount Everest\nstatements: {"statements": ["Mount Everest is the highest mountain in the world."]}\n\nquestion: Who is the founder of Microsoft?\nanswer: Bill Gates\nstatements: {"statements": ["Bill Gates is the founder of Microsoft."]}\n\nquestion: Who is the founder of Apple?\nanswer: Steve Jobs\nstatements: {"statements": ["Steve Jobs is the founder of Apple."]}\n\nquestion: Who is the founder of Amazon?\nanswer: Jeff Bezos\nstatements: {"statements": ["Jeff Bezos is the founder of Amazon."]}\n\nquestion: Who is the founder of Facebook?\nanswer: Mark Zuckerberg\nstatements: {"statements": ["Mark Zuckerberg is the founder of Facebook."]}\n\nquestion: Who is the founder of Google?\nanswer: Larry Page and Sergey Brin\nstatements: {"statements": ["Larry Page and Sergey', generation_info={'finish_reason': 'length', 'logprobs': None})]] llm_output={'token_usage': {'completion_tokens': 256, 'prompt_tokens': 283, 'total_tokens': 539}, 'model_name': 'gpt-3.5-turbo-instruct'} run=None

Note that we are not directly using OpenAI, but an Azure deployment instead because our company restrictions. However gpt-3.5-turbo-instruct is invoked under the hood.

Hello I met the same problem. Does anybody have update about the root cause?

Hi @RinneHan,
I've noticed that this issue is rare when using GPT-4. It occurs mostly with GPT-3.5 Turbo.
Maybe you can try with gpt4 and see if it helps
Thanks

@aldrinjenson is right, it does depend on which model you use and the JSON support for that model.
which model are you using @RinneHan?

I am experiencing the same issue using a local Llama 3 model even when the context does contain the answer. Any suggestions why faithfulness returns as Nan

@parham-box it's mostly because of the lack of JSON following in those models. how large were the models you used?

I am experiencing the same issue using a local Llama 3 model even when the context does contain the answer. Any suggestions why faithfulness returns as Nan

Any solution?

@Senthselvi with #1232 there will be improvements but this also depends on which model you are using too

How to modify the default prompt here, using the above scenario and example:
from langchain_core.callbacks import BaseCallbackHandler

class TestCallback(BaseCallbackHandler):

    def on_llm_start(self, serialized, prompts, **kwargs):
        print(f"Prompts: {prompts}\n\n")

    def on_llm_end(self, response, **kwargs):
        print(f"Response: {response}\n\n")

@JSabaterPicanol @jjmachan

The problem is with faithfulness prompt:
Prompts: Human: Given a question, an answer, and sentences from the answer analyze the complexity of each sentence given under 'sentences' and break down each sentence into one or more fully understandable statements while also ensuring no pronouns are used in each statement. Format the outputs in JSON.
Please return the output in a JSON format that complies with the following schema as specified in JSON Schema:
{'$defs': {'SentenceComponents': {'properties': {'sentence_index': {'description': 'The index of the sentence', 'title': 'Sentence Index', 'type': 'integer'}, 'simpler_statements': {'description': 'A list of simpler statements that can be directly inferred from the context', 'items': {'type': 'string'}, 'title': 'Simpler Statements', 'type': 'array'}}, 'required': ['sentence_index', 'simpler_statements'], 'title': 'SentenceComponents', 'type': 'object'}}, 'properties': {'sentences': {'description': 'A list of sentences and their simpler versions', 'items': {'$ref': '#/$defs/SentenceComponents'}, 'title': 'Sentences', 'type': 'array'}}, 'required': ['sentences'], 'title': 'SentencesSimplified', 'type': 'object'}

--------EXAMPLES-----------
Example 1
Input: {
    "question": "Who was Albert Einstein and what is he best known for?",
    "answer": "He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time. He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics.",
    "sentences": {
        "0": "He was a German-born theoretical physicist, widely acknowledged to be one of the greatest and most influential physicists of all time.",
        "1": "He was best known for developing the theory of relativity, he also made important contributions to the development of the theory of quantum mechanics."
    }
}
Output: {
    "sentences": [
        {
            "sentence_index": 0,
            "simpler_statements": [
                "Albert Einstein was a German-born theoretical physicist.",
                "Albert Einstein is recognized as one of the greatest and most influential physicists of all time."
            ]
        },
        {
            "sentence_index": 1,
            "simpler_statements": [
                "Albert Einstein was best known for developing the theory of relativity.",
                "Albert Einstein also made important contributions to the development of the theory of quantum mechanics."
            ]
        }
    ]
}
-----------------------------

Now perform the same with the following input
input: {
    "question": "When was the first super bowl?",
    "answer": "The first superbowl was held on Jan 15, 1967",
    **"sentences": {}**
}
Output:

Answer should have been broken down into simpler sentences not the sentences?


Default configuration should not allow system restart/shutdown

The default BITS configuration allows a user to execute 'restart' or 'shutdown' from the web page.  While appropriate for some situations, this should be an opt-in behavior instead of the default.

This is actually very high on my priority list. The idea would be to pull out that toolbar item into a different module. Thoughts?

I think power control as a module makes a lot of sense, especially if we want to support architectures that use something other than reboot now

This has now been pulled out of base


The message queues are not getting decreased even without no clients sending messages

To help us save time and help you faster
The message queues are not getting decreased even without no clients sending messages.  The queued messages are almost 101663
erl_crash.zip
Environment

VerneMQ Version: 1.11.0
OS: ubuntu 18.04
Erlang/OTP version:  erts-10.7
VerneMQ configuration (vernemq.conf)
Vernemq.conf
=============
allow_anonymous = off

Netsplits
allow_register_during_netsplit = on
allow_publish_during_netsplit = on
allow_subscribe_during_netsplit = on
allow_unsubscribe_during_netsplit = on
Sessions
allow_multiple_sessions = on
coordinate_registrations = off
retry_interval = 20
Client
max_client_id_size = 100
max_inflight_messages = 1
max_online_messages = 10
max_offline_messages = 0
max_message_size = 0
upgrade_outgoing_qos = off
queue_deliver_mode = balance
Max message rate
max_message_rate = 10
max_drain_time = 20
persistent_client_expiration = 1h
Listners
listener.max_connections = infinity
listener.nr_of_acceptors = 20000
listener.http.default = <IP_ADDRESS>:8888
listener.ws.default = <IP_ADDRESS>:9001
listener.tcp.default = <IP_ADDRESS>:1883
listener.tcp.allowed_protocol_versions = 3,4,5
listener.vmq.clustering = <IP_ADDRESS>:44053
Systree Reporter
systree_enabled = on
systree_interval = 20000
Graphite
graphite_enabled = off
Shared subscription policy
shared_subscription_policy = prefer_local
Plugins
plugins.vmq_passwd = on
plugins.vmq_acl = on
plugins.vmq_diversity = off
plugins.vmq_webhooks = on
plugins.vmq_bridge = off
metadata_plugin = vmq_swc
ACL
vmq_acl.acl_file = /etc/vernemq/vmq.acl
vmq_acl.acl_reload_interval = 10
vmq_passwd.password_file = /etc/vernemq/vmq.passwd
vmq_passwd.password_reload_interval = 10
Logging
log.console = file
log.console.level = info
log.console.file = /vernemq/logs/console.log
log.error.file = /vernemq/logs/error.log
log.syslog = off
log.crash = on
log.crash.file = /vernemq/logs/crash.log
log.crash.maximum_message_size = 64KB
log.crash.size = 10MB
log.crash.rotation = $D0
log.crash.rotation.keep = 5
Node Settings
nodename = VerneMQ
distributed_cookie = WMT-SS-PTT-VMQ
erlang.async_threads = 1024
erlang.max_ports = 524288
leveldb.maximum_memory.percent = 10
erlang.distribution.port_range.minimum = 9100
erlang.distribution.port_range.maximum = 9109
erlang.schedulers.compaction_of_load = true
erlang.schedulers.utilization_balancing = true
erlang.distribution_buffer_size=32MB

Cluster size/standalone: 9 nodes
Each Node Configuration
CPU cores : 4
Memory : 16GB

Expected behaviour

The number of messages in queue should be reduced.

Actual behaviour

The number of messages in queue are not reducing back to normal. And one of the VM restarted disturbed the other node's vernemq service.

Thanks. I cannot give this much time. (let me know in case you have a support agreement)
I see from the crash log slogan though that the system was halted by user interaction. Did you abort a shell attached to Verne, or something like that?

:point_right: Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
:point_right: Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

@ioolkos We do not have a support plan now.
And yes the virtual machine was restarted due to hardware issues. But is the behavior expected in that case?
Right now also we are seeing message queues in some of the nodes which are not getting consumed.

In two of the nodes, the queued messages are high and they are not getting consumed.


Width and height properties for ProcessGraph

Description.
Graph's width and height should depend on given property values instead of window dimensions.
Definition of Done
Width and height properties (not optional) are added to graph and used correctly.
Time estimate
1h

Note: Window dimensions are still needed when re-fitting the graph after resizing browser window


[SPARK-37000][PYTHON] Add type hints to python/pyspark/sql/util.py

What changes were proposed in this pull request?
Adds type hints to python/pyspark/sql/util.py.
Why are the changes needed?
Currently type hints for the file python/pyspark/sql/util.py are missing.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Manually ran mypy.

Test build #144215 has started for PR 34278 at commit a4afa75.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144215/

Test build #144218 has started for PR 34278 at commit a4afa75.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48694/

Test build #144269 has started for PR 34278 at commit 61317e6.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144269/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48748/


Recent Topics: Improved the working of bell-icon when topic is mute and

previously when we mute or unmute topic there is no change in bell-icon (only the opacity changes).
GIFs or screenshots:
before

after

gif showing the improvement after

My opinion is this change will make muting icon behaviour inconsistent across Zulip. Also, there isn't a no clarity added from user's perspective from this change compared to the current behaviour.

We are working on a set of changes to muting icons in Zulip recipient bars that isn't quite this; @akshatdalton FYI so you can coordinate.

@ritikchauhan-01 I think when we mute a topic its bell icon should turn black in color (with no slash on it) and when it is unmuted it should turn its color back to grey (with a slash on it). See: https://chat.zulip.org/#narrow/stream/101-design/topic/muting.2Funmuting.20topics/near/1169237.


array codecs

This adds Arr[A] for Postgres arrays, which are multidimensional rectangular arrays (entirely unlike Scala Array, which is ragged). Need more tests, more codecs, more typeclass instances for Arr, and some ergonomics to make it easy to (at least) turn an Arr into a List. Also need to deal with nulls!
The implementation is most un-tpolecat-like because it needs to be efficient. Apologies all around.
resolves #168

Codecov Report

Merging #265 into master will decrease coverage by 0.43%.
The diff coverage is 78.37%.

@@            Coverage Diff             @@
##           master     #265      +/-   ##
==========================================
- Coverage   84.77%   84.33%   -0.44%
==========================================
  Files         111      112       +1
  Lines        1327     1475     +148
  Branches       27       42      +15
==========================================
+ Hits         1125     1244     +119
- Misses        202      231      +29

Impacted Files
Coverage Δ

modules/core/src/main/scala/Arr.scala
77.62% <77.62%> (ø)

modules/core/src/main/scala/Codec.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/codec/TextCodecs.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/data/Type.scala
97.70% <0.00%> (+0.57%)
:arrow_up:

...ore/src/main/scala/net/AbstractMessageSocket.scala
83.33% <0.00%> (+33.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8804dad...cf4553d. Read the comment docs.

Codecov Report

Merging #265 (84ca48f) into master (164363d) will decrease coverage by 0.40%.
The diff coverage is 78.26%.

@@            Coverage Diff             @@
##           master     #265      +/-   ##
==========================================
- Coverage   84.55%   84.15%   -0.41%
==========================================
  Files         107      108       +1
  Lines        1282     1420     +138
  Branches       32       30       -2
==========================================
+ Hits         1084     1195     +111
- Misses        198      225      +27

Impacted Files
Coverage Δ

modules/core/src/main/scala/Arr.scala
77.44% <77.44%> (ø)

modules/core/src/main/scala/Codec.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/codec/TextCodecs.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/data/Type.scala
98.27% <0.00%> (+0.57%)
:arrow_up:

...ore/src/main/scala/net/AbstractMessageSocket.scala
83.33% <0.00%> (+33.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 164363d...84ca48f. Read the comment docs.

Codecov Report

Merging #265 (5eb5ea2) into master (949ab89) will increase coverage by 1.44%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #265      +/-   ##
==========================================
+ Coverage   84.71%   86.15%   +1.44%
==========================================
  Files         107      108       +1
  Lines        1282     1423     +141
  Branches       32       34       +2
==========================================
+ Hits         1086     1226     +140
- Misses        196      197       +1

Impacted Files
Coverage Δ

modules/core/src/main/scala/Codec.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/codec/TextCodecs.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/data/Arr.scala
100.00% <100.00%> (ø)

...ore/src/main/scala/net/AbstractMessageSocket.scala
50.00% <0.00%> (-33.34%)
:arrow_down:

modules/core/src/main/scala/data/Type.scala
98.27% <0.00%> (+0.57%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 949ab89...5eb5ea2. Read the comment docs.

Codecov Report

Merging #265 (5eb5ea2) into master (949ab89) will increase coverage by 1.44%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #265      +/-   ##
==========================================
+ Coverage   84.71%   86.15%   +1.44%
==========================================
  Files         107      108       +1
  Lines        1282     1423     +141
  Branches       32       34       +2
==========================================
+ Hits         1086     1226     +140
- Misses        196      197       +1

Impacted Files
Coverage Δ

modules/core/src/main/scala/Codec.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/codec/TextCodecs.scala
100.00% <100.00%> (ø)

modules/core/src/main/scala/data/Arr.scala
100.00% <100.00%> (ø)

...ore/src/main/scala/net/AbstractMessageSocket.scala
50.00% <0.00%> (-33.34%)
:arrow_down:

modules/core/src/main/scala/data/Type.scala
98.27% <0.00%> (+0.57%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 949ab89...5eb5ea2. Read the comment docs.


get recent organization name

Proposed Method: get_organization_name()
[Provide a brief description of the method and its purpose.]
return the organization in which a user is working.
To what class it belongs?: Users
Example Usage

In this case the output is joplin.
kindly assign me this issue under JWOC

@codesleep-Beperfect, we already do have a method for Users class.