Consider adding a "Thank you" route to end every call

This is minor, but I might consider adding a "goodbye" stage that happens at the end of each path through the IVR.
I felt caught off guard a little by the app hanging up on me after the directions ending. And right now the planet selection endings play once, then play again, then hang up - I was expecting them to either play once and hang up, or loop forever.
A final route for each path that makes it clear the IVR is over. Maybe something like:
"Thank you for calling the ET Phone Home Service - the adventurous alien's first choice in intergalactic travel."

You might also include an option to restart from the top menu for people who were intrigued enough (like me) to want to hear every option

Fixed. 0e9f8ce


Calling chat.unfurl with user_auth_blocks resulting in error invalid_arguments

Reproducible in:
mvn dependency:tree | grep com.slack.api
gradle dependencies | grep com.slack.api
java -version
sw_vers && uname -v # or `ver`

The Slack SDK version
+--- com.slack.api:bolt-socket-mode:1.14.0
|    +--- com.slack.api:slack-api-model:1.14.0
|    +--- com.slack.api:slack-api-client:1.14.0
|    |    +--- com.slack.api:slack-api-model:1.14.0 (*)
|    +--- com.slack.api:slack-app-backend:1.14.0
|    |    +--- com.slack.api:slack-api-model:1.14.0 (*)
|    |    +--- com.slack.api:slack-api-client:1.14.0 (*)
|    \--- com.slack.api:bolt:1.14.0
|         +--- com.slack.api:slack-api-model:1.14.0 (*)
|         +--- com.slack.api:slack-api-client:1.14.0 (*)
|         \--- com.slack.api:slack-app-backend:1.14.0 (*)
+--- com.slack.api:bolt-socket-mode:1.14.0 (n)

Java Runtime version
openjdk version "11.0.12" 2021-07-20 LTS
OpenJDK Runtime Environment Corretto-<IP_ADDRESS>.2 (build 11.0.12+7-LTS)
OpenJDK 64-Bit Server VM Corretto-<IP_ADDRESS>.2 (build 11.0.12+7-LTS, mixed mode)

OS info
ProductName:    macOS
ProductVersion: 11.6.1
BuildVersion:   20G224
Darwin Kernel Version 20.6.0: Tue Oct 12 18:33:42 PDT 2021; root:xnu-7195.141.8~1/RELEASE_X86_64

Steps to reproduce:
The issue occurs when I use chatUnfurl() with the userAuthBlocks argument:
var app = new App();
var client = app.getClient();
app.event(LinkSharedEvent.class, (payload, ctx) -> {
  var event = payload.getEvent();
  var links = event.getLinks();
  var unfurls = links.stream().collect(Collectors.toMap(LinkSharedEvent.Link::getUrl, link -> {
    var details = new ChatUnfurlRequest.UnfurlDetail();
    details.setBlocks(Blocks.asBlocks(
        Blocks.section(s -> s.text(BlockCompositions.plainText("An link unfurling")))
    ));
    return details;
  }));
  var authBlocks = Blocks.asBlocks(
      Blocks.section(s -> s.text(BlockCompositions.plainText("An auth blocks")))
  );
  var unfurlReq = ChatUnfurlRequest.builder()
      .token(BOT_TOKEN)
      .unfurlId(event.getUnfurlId())
      .source(event.getSource())
      .unfurls(unfurls)
      .userAuthRequired(true)
      .userAuthMessage("An auth message")
      .userAuthBlocks(authBlocks)
      .build();
  var unfurlRes = client.chatUnfurl(unfurlReq);
  ctx.logger.info("unfurl response: error={}", unfurlRes.getError());
  return ctx.ack();
});

Expected result:
The call to chat.unfurl should result in no error.
Actual result:
The error I got is:
unfurl response: error=invalid_arguments

I tried to call the endpoint directly using curl with the same arguments and it worked. I think the reason is because the SDK doesn't transform req.getUserAuthBlocks() into JSON string:
https://github.com/slackapi/java-slack-sdk/blob/b54a628006905c02140cd8d9a68c94cf6dd90160/slack-api-client/src/main/java/com/slack/api/methods/RequestFormBuilder.java#L1220-L1237

Hi @erathorus, thanks for taking the time to report this issue! I'll check the details tomorrow in my timezone but at first glance, this sounds like a bug in this SDK.

Thank you so much for the quick fix @seratch! Our team is working on a feature using this user_auth_blocks argument so I wonder if there is a rough estimate about when version 1.15 of the SDK is released? 😄

@erathorus will do later today 💪


style nit fix

My fault..

Checklist

[ ] documentation is added or updated
[ ] tests are added or updated

Merged, thanks.


Automatic primitive templates don't show in the Thingpedia API

The primitive templates generated from the function #_[canonical] are not added to the Thingpedia database, so they do not show up in the getExamplesByKinds or getExamplesByKey API. Which in turns means they don't show as suggestions when the command is not understood, don't show up in the cheatsheet, don't show up in the device page.
@rayslxu ideas of how to solve this?

Add examples by applying one filter a time? I think users can infer how all filters can be composed.

I'm not worried too much about examples with filters. Rather the problem is not having examples at all...

Mh this issue is misplaced, it should be in almond-cloud.


Lunch ticket qr code

An ARKAD-funktionär (volunteer) should be able to see the QR code of their lunch ticket when logged in to the app. A button could be added to the profile page which displays the code when clicked.

Won't fix since feature is already implemented


Fix minio connection issue

Description
What is the current behavior?
Currently, we were not able to connect to minio(https://min.io/) as a replacement for S3.
Impact: load_file(), export_file() and minio as custom xcom backend
related: #750
and slack thread - https://astronomer.slack.com/archives/C02B8SPT93K/p1666289425280609
What is the new behavior?
Now we are passing all the required parameters to open_smart() to be able to communicate to minio server.
Does this introduce a breaking change?
Nope

@phanikumv @tatiana

Minio works very similarly to S3, so I think we can do away without adding an integration test
Also since Minio is locally hosted we need to have a server of it running in our CI env for us to have an integration test.

I think just having a UT which verifies that we are creating the transport params correctly should be enough.


v4  yields invalid hook call

When I render <VisualEditing /> in root we get the classic
Warning: Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:
1. You might have mismatching versions of React and the renderer (such as React DOM)
2. You might be breaking the Rules of Hooks
3. You might have more than one copy of React in the same app
See https://reactjs.org/link/invalid-hook-call for tips about how to debug and fix this problem.
TypeError: Cannot read properties of null (reading 'useSyncExternalStore')

node_modules/@sanity/visual-editing/node_modules/react/cjs/react.development.js:1676:21
    "react": "^18.2.0",
    "hydrogen-sanity": "^4.0.2",

What does npm ls react output? (https://react.dev/warnings/invalid-hook-call-warning#duplicate-react)
If you see multiple versions of react (18.2.x and 18.3.x), then try running npm dedupe

Are you on hydrogen v2024-04 and if so how do you avoid this issue?
https://github.com/Shopify/hydrogen/discussions/2073

Now that<EMAIL_ADDRESS>has been released, you may want to use:
   "react": "~18.2.0",

approximate equivalence is not a bad idea but the point I am trying to make is that<EMAIL_ADDRESS>has a dependency tree that includes as a peer dependency<EMAIL_ADDRESS>Nonetheless, this change simply moves the hook error to the browser
    "react": "~18.2.0",
    "react-dom": "~18.2.0",

When I encountered a similar error, I ran npm ls react and noticed both versions were installed. I then ran npm dedupe and they all resolved to<EMAIL_ADDRESS>Another option might be to override the dependency? I haven't tested this though, so your mileage may vary
  "overrides": {
    "react": "~18.2"
  }

Since this isn't a dependency issue caused by this particular package, I'll close this issue.


Add 'namespaced' property to support flat vs deep directory structure

Defaults to 'no' to use the preferred flat directory structure.

Note that the $if(namespaced.truthy)$FOO$else$.$endif$ pattern says if the namespaced config property is true, then the directory should exist and its name should be FOO (which could also be a template variable). Otherwise, the directory should not exist and any child directories/files become children of the parent.
I also find that it's a little easier to make sense of the new template directory names in tree form:
src/main/g8/
├── $if(namespaced.truthy)$src$else$.$endif$
│   ├── $if(namespaced.truthy)$main$else$src$endif$
│   │   └── $if(namespaced.truthy)$resources$else$.$endif$
│   │       └── $if(namespaced.truthy)$$package$$else$.$endif$
│   │           └── $if(namespaced.truthy)$$name__camel$$else$.$endif$
│   │               └── $if(namespaced.truthy)$xsd$else$.$endif$
│   │                   └── $name__camel$.dfdl.xsd
│   └── test
│       ├── $if(namespaced.truthy)$resources$else$.$endif$
│       │   └── $if(namespaced.truthy)$$package$$else$.$endif$
│       │       └── $if(namespaced.truthy)$$name__camel$$else$.$endif$
│       │           ├── Test$name__Camel$.tdml
│       │           ├── data
│       │           │   └── test_01.$name__camel$
│       │           └── infosets
│       │               └── test_01.$name__camel$.xml
│       └── $if(namespaced.truthy)$scala$else$.$endif$
│           └── $if(namespaced.truthy)$$package$$else$.$endif$
│               └── $if(namespaced.truthy)$$name__camel$$else$.$endif$
│                   └── Test$name__Camel$.scala
├── README.md
├── build.sbt
├── default.properties
└── project
    └── build.properties

Ok, I think all comments are addressed, please test and let me know if any other changes are required.

I checked out your fork. How do I get sbt new to use this checkout, not the one found online?

You can actually test it out it without even checking out my fork, you can point it to the right branch on my github repo:
sbt new stevedlawrence/daffodil-schema.g8 --branch namespaced-property

Or to with a local checkout, you can also run:
sbt new file:///path/to/cloned/repo

Tried it.
Fine, but I was surprised that the dummy data and expected infoset files are test_01.myformat and test_01.myformat.xml
Using the format name as an extension in the file-names is unexpected. Most format names will not be suitable for this. E.g. "link16Subset".
I think the script should ask for a data file suffix with ".dat" as the default

Makes sense, updated to add a new "extension" property which is used when creating the test files.

I noticed you updated the workflow.  This raises an interesting question: is it less work to manually sync this repo once or twice a year than to enable Dependabot and Scala Stewart for this repo?  Actually, Dependabot's updates can be scheduled by intervals of  1 month, and Scala Stewart's pull requests can be spaced by intervals of  days, so we could limit Dependabot PRs to the first day of each month and Scala Stewart PRs to a few times a year.  Scala Stewart will read a configuration file as well as Dependabot.

I was wondering why everything was out of date, I forgot that config files had to be added to enabled depenabot and scala steward. Yeah, we should just add those config files to enable dependency PR. I don't even mind if it's configred to open a PR as soon as possible rather than once a month. Those PR's are generally pretty easy to review, we have a lot less dependencies in this repo, and the PR's will probably be very similar to ones opened for Daffodil so it would be fresh in our mind.


Refactor: Tech Radar Code Quality & Style

Hey, I just made a Pull Request!
Closes #1222
I have still work to do on this one but I wanted to create a draft to show the progress of my work and to be able to ask questions about the existing codebase.
Improvements I was able to work on:

Removed « _ » from function names
Removed FC from react (related to https://github.com/spotify/backstage/issues/1376)
Reduce some function size
Add unit tests
Add utils wrapper component WithLink
Rename some variables
Update some types

:heavy_check_mark: Checklist

[x] All tests are passing yarn test
[ ] Screenshots attached (for UI changes)
[ ] Relevant documentation updated
[x] Prettier run on changed files
[ ] Tests added for new functionality
[ ] Regression tests added for bug fixes

@ayshiff Want to mark this as ready for review?

@ayshiff Added some comments! Looks fantastic, so much improvements to the readability 😁


[IBM VPC] Container not always containing

In the attached example, the zone and subnet are containers, then the 2 VSIs icons were dragged into the subnet container.
The expected outcome is that the 2 VSIs are now included with the subnet container and will move and duplicate whenever the subnet container is moved or duplicated.
The actual outcome is that the 2 VSIs are not included with the container when the container is moved, the VSIs remain where they are.  This is currently reproducible in the attached example by dragging the VSIs out of the container and then dragging the VSIs back into the container.  As far as I can tell I'm doing this correctly but let me know if I'm missing something.
This scenario seems to happen a lot and eventually I can get the VSIs to be part of the container.  I wouldn't expect that I would need to manually group the VSIs and containers.
Thanks,
Jay
ContainerNotContainingExample.drawio.zip

Changes published this weekend look awesome! I converted several secondary diagrams using Edit Style with cut-and-paste to get them quickly converted (realizing that the containers won't be set up properly since the objects were placed before some of the boxes were containers). The Subnet:ACL worked out well. This week I will convert the template published to draw.io from scratch so it will be created properly with all containers and will test out: 1. expansion problem (issue 723), 2. make sure objects stay in container (issue 724), and 3) revisit whether the 3 dashed groups should be a container. These changes are really excellent! Thanks!

Using the new boxes I was not able to reproduce this issue with icons not staying in a container box after an icon is dragged into the container box, so will close out and keep on eye on whether it happens.  Thanks!

I just ran across this issue with the new container boxes so reopening while I try to create a reproducible scenario.  When it happens it happens consistently, but then eventually the icon stays in the container.

Attached zip DragIconIntoContainerExmpleWithNewBoxes shows this issue with the new boxes.
The scenario is:

There is an icon underneath all the boxes by itself.
Drag that icon and drop inside the nearest Subnet:ACL container directly above it.
Move that Subnet:ACL container around or move any of the surrounding containers to a different position.
The expected outcome is that the icon moves with the containers but the actual outcome is that the icons stays stationary and doesn't move indicating that the icon is not staying in the container.

DragIconIntoContainerExampleWithNewBoxes.drawio.zip

The scenario I just mentioned is consistently repeatable.  However, if I move the Subnet container outside all of the boxes, then drag the icon into the Subnet container, the icon remains in the container, then move the Subnet container back into place and the icon remains in the container.
Therefore, the problem appears to only happen with nested boxes where the icon doesn't stay in the Subnet container.

The scenario for icon text background color not changable from white to none didn't happen in a previous diagrams, but it is a problem with the attached zip when I add an icon to the canvas which has a different style.
Working Style:
aspect=fixed;perimeter=ellipsePerimeter;html=1;align=center;shadow=0;dashed=0;image;image=img/lib/ibm/vpc/PublicGateway.svg;fontColor=#000000;labelPosition=center;verticalLabelPosition=bottom;verticalAlign=top;fontSize=12;flipH=1;labelBackgroundColor=none;spacingTop=-5;
Nonworking Style:
aspect=fixed;perimeter=ellipsePerimeter;html=1;align=center;shadow=0;dashed=0;fontColor=#000000;labelBackgroundColor=none;fontSize=12;spacingTop=3;image;image=img/lib/ibm/vpc/PublicGateway.svg;flipH=1;
The nonworking style has the lableBackgroundColor=none but the format panel has checked the background color of white that can't be unchecked.
The working style has additional settings and doesn't have a white label background.
For now I can give users the working style that they can replace in the nonworking icon style.
I'm working on updating the ibm_vpc_architecture template to replace on draw.io but I want to make sure that there is no label background color and that icons stay in containers.
Thanks!

The previous diagram was probably created a year ago and when I added the shaded boxes they happened to work already with no label background color.  Therefore, could something have changed in the default icon styles (that I didn't change) since then that caused this problem on new diagrams now that I'm recreating the IBM template from scratch.  Seems to happen on all icons and I don't usual change the icon styles that I recall anyway.

Clarification: Adding one of our icons to a standalone Subnet container or standalone Zone container works consistently with the icon staying in the container, but any other box does not.  So to get the other boxes to retain icons I start out with a Subnet or Zone container, add icons, then change the style to another box such as the Region container, then nest boxes as the final step.

I couldn't reproduce this issue. The children of containers move correctly even by moving parents of containers. Please make sure you do not use any layers while testing this.

Testing showed that this happens in Firefox but not in Chrome, so if possible, please use Chrome in the meantime.

Will be fixed in next release.

I do use FF on Mac and Windows which it looks like I didn't mention in the description (will include browser next time), I haven't tried Safari or Chrome or IE to reproduce this problem, looking forward to the fix in next release, and will use Chrome in the meantime.  Thanks!


[SPARK-16060][SQL] Vectorized Orc reader

What changes were proposed in this pull request?
Currently Orc reader in Spark SQL doesn't support vectorized reading. As Hive Orc already support vectorization, we can add this support to improve Orc reading performance.
Benchmark 1
Benchmark code:
test("Benchmark for Orc") {
  val N = 500 << 12
    withOrcTable((0 until N).map(i => (i, i.toString, i.toLong, i.toDouble)), "t") {
      val benchmark = new Benchmark("Orc reader", N)
      benchmark.addCase("reading Orc file", 10) { iter =>
        sql("SELECT  sum(_1), count(_2), sum(_3), avg(_4) FROM t").collect()
      }
      benchmark.run()
  }
}

Before this patch (version 1, no column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_71-b15 on Linux 3.19.0-25-generic
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                              1405 / 1593          1.5         686.1       1.0X

After this patch (version 1, no column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_71-b15 on Linux 3.19.0-25-generic
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                               944 / 1126          2.2         460.8       1.0X

Before this patch (version 2, column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_102-b14 on Linux 4.4.27-moby
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                               900 /  962          2.3         439.3       1.0X

After this patch (version 2, column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_102-b14 on Linux 4.4.27-moby
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                               286 /  330          7.2         139.8       1.0X

Notice: After support Spark's column batch, the performance is largely improved as the operators can benefit from batch processing.
Benchmark 2
Benchmark code:
test("Benchmark for Orc") {
  val N = 500 << 12
  withOrcTable((0 until N).map(i => (i, i.toString, i.toLong, i.toDouble)), "t") {
    val benchmark = new Benchmark("Orc reader", N)
    benchmark.addCase("reading Orc file", 10) { iter =>
      sql("SELECT * FROM t").count()
    }
    benchmark.run()
  }
}

Before this patch (version 1, no column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_71-b15 on Linux 3.19.0-64-generic
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                               813 / 1018          2.5         397.1       1.0X

After this patch (version 1, no column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_71-b15 on Linux 3.19.0-64-generic
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                               508 /  693          4.0         248.2       1.0X

Before this patch (version 2, column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_102-b14 on Linux 4.4.27-moby
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                               208 /  235          9.9         101.4       1.0X

After this patch (version 2, column batch support):
Java HotSpot(TM) 64-Bit Server VM 1.8.0_102-b14 on Linux 4.4.27-moby
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
Orc reader:                              Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------
reading Orc file                               102 /  128         20.1          49.7       1.0X

Notice: For simply counting operation, batch processing doesn't have as much improvement as benchmark 1. It should be reasonable. But the improvement is also significant.
How was this patch tested?
Existing tests.

This is still wrong unfortunately --- count(*) is going to prune all the columns ...

@rxin oh, right...I will update this later. Thanks!

@hvanhovell @rxin I've updated the benchmark. Please let me know if this time it is appropriate. Thanks!

Test build #61037 has started for PR 13775 at commit 855bcfd.

Test build #61037 has finished for PR 13775 at commit 855bcfd.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61037/
Test FAILed.

retest this please.

Test build #61075 has started for PR 13775 at commit 855bcfd.

Test build #61075 has finished for PR 13775 at commit 855bcfd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61075/
Test PASSed.

Test build #61088 has started for PR 13775 at commit 66ab632.

Test build #61088 has finished for PR 13775 at commit 66ab632.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61088/
Test PASSed.

Test build #61384 has started for PR 13775 at commit 4c14278.

Test build #61384 has finished for PR 13775 at commit 4c14278.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61384/
Test PASSed.

@rxin @hvanhovell Available to review this? Or wait for after 2.0 release?

also cc @liancheng @yhuai

ping @hvanhovell @rxin @liancheng @yhuai Could you review this now? Thanks.

Test build #63498 has started for PR 13775 at commit b067658.

Test build #63498 has finished for PR 13775 at commit b067658.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63498/
Test PASSed.

for the benchmark, how about we just test the scan operation?

Test build #63585 has started for PR 13775 at commit 06066eb.

@yhuai You mean just using sql("SELECT * FROM t").count()?

@yhuai I ran another benchmark for the scan operation. The results are updated in the pr description.

Test build #63585 has finished for PR 13775 at commit 06066eb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63585/
Test PASSed.

Merged build finished. Test PASSed.

ping @yhuai Any chance you can review this? Thank you.

ping @yhuai @liancheng @hvanhovell @cloud-fan Can you take a look at this? Thanks.

ping @yhuai @liancheng @hvanhovell @cloud-fan again...

ping @rxin @yhuai @liancheng @hvanhovell @cloud-fan Can you take a look? This is waiting for review for a while. Thanks!

Test build #66424 has started for PR 13775 at commit ed780f6.

Test build #66424 has finished for PR 13775 at commit ed780f6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66424/
Test FAILed.

retest this please.

Test build #66436 has started for PR 13775 at commit ed780f6.

Test build #66436 has finished for PR 13775 at commit ed780f6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66436/
Test PASSed.

This is an interesting pull request (very similar to the Parquet approach). Two high level comments:

Can we just convert Hive's vector format into our own column batch, to avoid turning the data into rows? I'd imagine you will get a bigger speedup here.

We need more tests. This patch has almost no tests.

@rxin Thanks for looking at this!

My initial investigation when implementing this PR is looking at this direction too. Because this is waiting for a while, I can only remember that converting Hive vector format to column batch is not trivial. But I will look at this in second try.

Yeah, I will try to add more tests.

@viirya Have you tried the ORC reader in presto? I guess that one is more efficient than the one in Hive. But not sure how hard will be to make it work with Spark SQL (column batch).

Earlier this year I had spent some time trying out Presto's ORC reader with Spark.
In standalone benchmark, Presto's ORC reader is 3x faster than the one in Hive. My experimental setup was to add a CLI around ORC reader code in both hive and presto. It would read a file from local disk and deserialize all the columns.
With that promising result, I tried hooking Presto reader with Spark. Presto has its own notion of "page" to manage memory and "slice" for Unsafe access to data. When I integrated it with Spark, I had to add a shim to convert Spark objects to corresponding Presto objects. This hurts performance.
Then I decided to fork few classes from Presto to make it directly work with Spark's internal data representations (eg. UTF8String). My final numbers (for end to end runs of Spark jobs) were ranging from no gains to 2x improvement. Note that there were things Presto's reader supports (namely vectorization and predicate pushdown) which I had not integrated as that would have demanded more forking. The measurements were over queries which ran over large table, read all the rows and wrote those as-is to another table. Having done all these things, my personal take is that forking classes is bad from a maintenance standpoint. Presto's ORC reader is tightly coupled with the engines' internal constructs and refactoring it to make it generic is non-trivial work. We can think about having a native ORC reader in Spark (just like Presto does) which would be super performant but the downside is that one has to be sync with all upstream changes to the file format as it evolves.
cc @rxin (I recall you were also interested in this at some point)

If we do want to maintain one in Spark, we should probably start by adding Presto's. Data formats don't change super frequently, so I think we would be able to maintain it, especially when a large company like Facebook has resources devoted to making it work :)

I agree that from a maintenance standpoint forking the classes is bad. But if we really want to have the one in Spark, I would like to help too. :)

@tejasapatil Thanks for the review comment! I will update this later.

Test build #68990 has started for PR 13775 at commit 3895a98.

Test build #68990 has finished for PR 13775 at commit 3895a98.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68990/
Test FAILed.

Test build #68995 has started for PR 13775 at commit c24169d.

Test build #68995 has finished for PR 13775 at commit c24169d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68995/
Test PASSed.

Test build #69080 has started for PR 13775 at commit 47bc196.

Can we just convert Hive's vector format into our own column batch, to avoid turning the data into rows?  After converting Hive's column vectors into Spark's column batch, the performance is largely improved. I've posted the newer benchmark in the pr description.

Test build #69080 has finished for PR 13775 at commit 47bc196.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69080/
Test FAILed.

Test build #69082 has started for PR 13775 at commit c297678.

Test build #69082 has finished for PR 13775 at commit c297678.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class OrcColumnVector extends org.apache.spark.sql.execution.vectorized.ColumnVector

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69082/
Test PASSed.

Merged build finished. Test PASSed.

Test build #69124 has started for PR 13775 at commit 8638a0e.

Test build #69124 has finished for PR 13775 at commit 8638a0e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69124/
Test FAILed.

retest this please.

Test build #69131 has started for PR 13775 at commit 8638a0e.

Test build #69133 has started for PR 13775 at commit 55bb19f.

By supporting Spark's ColumnarBatch, the benchmarks show this vectorized Orc reader can boost 2 to 3x improvement.
I will continue to add more tests.

Test build #69131 has finished for PR 13775 at commit 8638a0e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69131/
Test FAILed.

Test build #69133 has finished for PR 13775 at commit 55bb19f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69133/
Test PASSed.

Test build #69147 has started for PR 13775 at commit 160e924.

Test build #69148 has started for PR 13775 at commit 3014834.

Test build #69149 has started for PR 13775 at commit bd15842.

Test build #69147 has finished for PR 13775 at commit 160e924.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69147/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69149/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69148/
Test FAILed.

Test build #69156 has started for PR 13775 at commit 0ac61b7.

@rxin @davies @hvanhovell @yhuai @tejasapatil @zjffdu @dafrista I've address review comments and add tests. Please help review this if you can. Thanks.

Test build #69156 has finished for PR 13775 at commit 0ac61b7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69156/
Test PASSed.

@viirya . If possible, I'd like to benchmark this PR in my laptop.

Otherwise, may I implement this way in my PR as a Viirya's approach?

@dongjoon-hyun Sure. Do you need any help?

@dongjoon-hyun No problem.

Thank you! First, I'll try to rebase and run with my OrcReadBenchmark (which is similar with ParquetReadBenchmark).

Hmm. It seems Merge remote-tracking branch makes rebasing confused. Let me think how to compare this.


parsing a GNUBG ID

Hey, this is impressive work and clearly a lot went into this. Apologies for this question....
I'm interested in parsing GNUBG IDs however I can't quite get my head around the algorithm specified here
I think this is the point in your code where you are doing it? Is that correct?
GNUBG seems to have a format once the ID is decoded that I can read like:
{
  board: {
    o: {
      "6": 5,
      "8": 3,
      "13": 5,
      "24": 2,
    },
    x: {
      "6": 5,
      "8": 3,
      "13": 5,
      "24": 2,
    },
  },
  cubeful: false,
  dice: [3, 1],
  "max-moves": 3,
  player: "x",
  "score-moves": true,
}

I am trying to work out how I can go from the GNUBG ID to a format like this. I can't work out how you are going from the ID to the format that is "human readable" per se and wondered if you might be able to help me extract your algorithm to do so?

The code you are looking for is in the function setDataFromGnuId . Follow it line by line and you will get what you need.
I will close this since it is not an issue. If you have similar questions in the future just email me directly.


Add completion for resource-like declarations

As per https://puppet.com/docs/puppet/latest/lang_classes.html#section-x54-1hk-xhb it is possible to do resource-like declarations for Puppet types e.g.
class {'apache':
  version => '2.2.21',
}

Resource-like declarations look like normal resource declarations, using the class pseudo-resource type. You can provide a value for any class parameter by specifying it as a resource attribute.

The completion provider should provide parameter and property completions for this type of declaration.

PR originally created by @juliosueiras
https://github.com/lingua-pupuli/puppet-editor-services/pull/180


It looks like there are two copies of the source code for log4cpp, and they are identical

https://github.com/huaweicloud/huaweicloud-sdk-c-obs/tree/master/platform/eSDK_LogAPI_V2.1.10/log4cpp/src
vs
https://github.com/huaweicloud/huaweicloud-sdk-c-obs/tree/master/platform/eSDK_LogAPI_V2.1.10/log4cpp
Was that a mistake? :(

您好，可能是在编译时没有做好相应处理，后续版本会排查

ah, thanks for thre response :)    btw, will there be any updates to the repository recently?


EventListener callStart should only fire if we go to network

Per @swankjesse

we might need to move the callStart callback to only occur if you actually use the network

Would it be more useful to keep callStart, callEnd marking the start and end for clients, but expose an event for a cache hit instead?

The issue here is we don't even get that far. One of the user’s interceptors short-circuits before a connection is opened.

No, cache hits will work too. Only thing is broken is if an application interceptor doesn't call proceed().

Fixed!
https://github.com/square/okhttp/pull/4630/files#diff-1bace3b10780b6d78dca6b26ce001321R1171


Resolve gcc problems on Windows

flyby: resolve winsock2.h warnings on windows
flyby: add CI builder for gcc/mingw on Windows

Performance test reportHPX PerformanceComparisonBENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTORFor Each(=)??-InfoPropertyBeforeAfterHPX Datetime2023-05-10T12:07:53+00:002023-05-20T14:48:22+00:00HPX Commitdcb541576898d370113946ba15fb58c20c8325b2c2d99368ceed921770f126fe2307aa991e2d63bbEnvfileClusternamerostamrostamHostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduDatetime2023-05-10T14:50:18.616050-05:002023-05-20T09:59:01.517056-05:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1ComparisonBENCHMARKNO-EXECUTORFuture Overhead - Create Thread Hierarchical - Latch=InfoPropertyBeforeAfterHPX Datetime2023-05-10T12:07:53+00:002023-05-20T14:48:22+00:00HPX Commitdcb541576898d370113946ba15fb58c20c8325b2c2d99368ceed921770f126fe2307aa991e2d63bbEnvfileClusternamerostamrostamHostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduDatetime2023-05-10T14:52:35.047119-05:002023-05-20T10:01:16.378852-05:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1ComparisonBENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATORStream Benchmark - Add(=)(=)=Stream Benchmark - Scale(=)=(=)Stream Benchmark - Triad=(=)(=)Stream Benchmark - Copy(=)(=)(=)InfoPropertyBeforeAfterHPX Datetime2023-05-10T12:07:53+00:002023-05-20T14:48:22+00:00HPX Commitdcb541576898d370113946ba15fb58c20c8325b2c2d99368ceed921770f126fe2307aa991e2d63bbEnvfileClusternamerostamrostamHostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduDatetime2023-05-10T14:52:52.237641-05:002023-05-20T10:01:33.535862-05:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1Explanation of SymbolsSymbolMEANING=No performance change (confidence interval within ±1%)(=)Probably no performance change (confidence interval within ±2%)(+)/(-)Very small performance improvement/degradation (≤1%)+/-Small performance improvement/degradation (≤5%)++/--Large performance improvement/degradation (≤10%)+++/---Very large performance improvement/degradation (>10%)?Probably no change, but quite large uncertainty (confidence interval with ±5%)??Unclear result, very large uncertainty (±10%)???Something unexpected…

Performance test reportHPX PerformanceComparisonBENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTORFor Each(=)??-InfoPropertyBeforeAfterHPX Commitdcb541576898d370113946ba15fb58c20c8325b2cd9e4a4cd4546d71896291d663b853a0b4d09e66HPX Datetime2023-05-10T12:07:53+00:002023-05-20T23:03:51+00:00EnvfileDatetime2023-05-10T14:50:18.616050-05:002023-05-20T18:10:09.504792-05:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduClusternamerostamrostamComparisonBENCHMARKNO-EXECUTORFuture Overhead - Create Thread Hierarchical - Latch=InfoPropertyBeforeAfterHPX Commitdcb541576898d370113946ba15fb58c20c8325b2cd9e4a4cd4546d71896291d663b853a0b4d09e66HPX Datetime2023-05-10T12:07:53+00:002023-05-20T23:03:51+00:00EnvfileDatetime2023-05-10T14:52:35.047119-05:002023-05-20T18:12:26.178943-05:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduClusternamerostamrostamComparisonBENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATORStream Benchmark - Add(=)(=)(=)Stream Benchmark - Scale(=)=(=)Stream Benchmark - Triad(=)=(=)Stream Benchmark - Copy(=)(=)(=)InfoPropertyBeforeAfterHPX Commitdcb541576898d370113946ba15fb58c20c8325b2cd9e4a4cd4546d71896291d663b853a0b4d09e66HPX Datetime2023-05-10T12:07:53+00:002023-05-20T23:03:51+00:00EnvfileDatetime2023-05-10T14:52:52.237641-05:002023-05-20T18:12:43.600655-05:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduClusternamerostamrostamExplanation of SymbolsSymbolMEANING=No performance change (confidence interval within ±1%)(=)Probably no performance change (confidence interval within ±2%)(+)/(-)Very small performance improvement/degradation (≤1%)+/-Small performance improvement/degradation (≤5%)++/--Large performance improvement/degradation (≤10%)+++/---Very large performance improvement/degradation (>10%)?Probably no change, but quite large uncertainty (confidence interval with ±5%)??Unclear result, very large uncertainty (±10%)???Something unexpected…

Performance test reportHPX PerformanceComparisonBENCHMARKFORK_JOIN_EXECUTORPARALLEL_EXECUTORSCHEDULER_EXECUTORFor Each(=)??-InfoPropertyBeforeAfterHPX Commitdcb541576898d370113946ba15fb58c20c8325b2a0e65edd9f6a94d58a99b13ac4d8c4b80289d6dcHPX Datetime2023-05-10T12:07:53+00:002023-05-21T00:08:43+00:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduDatetime2023-05-10T14:50:18.616050-05:002023-05-20T19:14:55.365017-05:00EnvfileClusternamerostamrostamComparisonBENCHMARKNO-EXECUTORFuture Overhead - Create Thread Hierarchical - Latch(=)InfoPropertyBeforeAfterHPX Commitdcb541576898d370113946ba15fb58c20c8325b2a0e65edd9f6a94d58a99b13ac4d8c4b80289d6dcHPX Datetime2023-05-10T12:07:53+00:002023-05-21T00:08:43+00:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduDatetime2023-05-10T14:52:35.047119-05:002023-05-20T19:17:12.660413-05:00EnvfileClusternamerostamrostamComparisonBENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATORStream Benchmark - Add(=)=(=)Stream Benchmark - Scale(=)=(=)Stream Benchmark - Triad=(=)(=)Stream Benchmark - Copy(=)=(=)InfoPropertyBeforeAfterHPX Commitdcb541576898d370113946ba15fb58c20c8325b2a0e65edd9f6a94d58a99b13ac4d8c4b80289d6dcHPX Datetime2023-05-10T12:07:53+00:002023-05-21T00:08:43+00:00Compiler/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1/opt/apps/llvm/13.0.1/bin/clang++ 13.0.1Hostnamemedusa08.rostam.cct.lsu.edumedusa08.rostam.cct.lsu.eduDatetime2023-05-10T14:52:52.237641-05:002023-05-20T19:17:30.028067-05:00EnvfileClusternamerostamrostamExplanation of SymbolsSymbolMEANING=No performance change (confidence interval within ±1%)(=)Probably no performance change (confidence interval within ±2%)(+)/(-)Very small performance improvement/degradation (≤1%)+/-Small performance improvement/degradation (≤5%)++/--Large performance improvement/degradation (≤10%)+++/---Very large performance improvement/degradation (>10%)?Probably no change, but quite large uncertainty (confidence interval with ±5%)??Unclear result, very large uncertainty (±10%)???Something unexpected…

@vhandziuk everything seems to compile now. What's left is some error thrown during startup of the HPX tests (see: https://github.com/STEllAR-GROUP/hpx/actions/runs/5084448959/jobs/9136758961?pr=6250) Please feel free to try this out and report back if you're seeing similar issues when executing an HPX application.

@vhandziuk everything seems to compile now. What's left is some error thrown during startup of the HPX tests (see: https://github.com/STEllAR-GROUP/hpx/actions/runs/5084448959/jobs/9136758961?pr=6250) Please feel free to try this out and report back if you're seeing similar issues when executing an HPX application.

@hkaiser Thanks! I have tried to build the branch with the latest fixes on my machine. Unfortunately, the build still fails. Here is the link to the error output: https://pastebin.com/81tnMNQN

@vhandziuk everything seems to compile now. What's left is some error thrown during startup of the HPX tests (see: https://github.com/STEllAR-GROUP/hpx/actions/runs/5084448959/jobs/9136758961?pr=6250) Please feel free to try this out and report back if you're seeing similar issues when executing an HPX application.

@hkaiser Thanks! I have tried to build the branch with the latest fixes on my machine. Unfortunately, the build still fails. Here is the link to the error output: https://pastebin.com/81tnMNQN

Update: After fetching the very latest changes I can build the project! Thanks for your efforts in fixing this issue. I will try to run the tests tomorrow and let you know about the results.


Support JSON custom fields

Issue type
[x] Feature request
[ ] Bug report
[ ] Documentation
Environment

Python version:  3.6.4
NetBox version:  2.3.4

Description
To integrate Netbox with external systems more reliably, we would like to be able to store JSON structured data against objects with custom fields.
For our use case this is to support marking up our devices, IPs and networks which additional data to support custom tooling and to match metadata which maps to other systems.
To implement this, the proposal would be to add a "structured_data" field to the CustomFieldValue model, and a new CF_TYPE_JSON type for custom fields. CustomField's with JSON data would return the structured_data value rather then a deserialized value, which in the underlying database would be stored as a postgres JSONB type via the Django postgres JSON field.
To support data consistency, two additional fields for the CustomField would be added - json_schema as JSON and validate_json (a bool) to allow enforcing the structure of added JSON and using the django-jsonforms module to generate rich user frontends for custom fields in the UI. django-json-widget can be used to support free editing of schemas in the admin interface.
An early version of this proposal is already implemented here: https://github.com/wrouesnel/netbox/tree/json_custom_fields, the django-jsonforms requires some CSS work to fit nicely into the UI.

#1349

Hi, I've been thinking about migrating to netbox, but the tooling we use relies on JSON custom fields.  I've been following the discussion on #1349 but as far as I could understand it has a different scope. In our case, we'd rather merge the context ourselves. Besides, some fields do not directly relate to configuration information, so we just need a placeholder for the json data and the ability to index and query it efficiently. I'm very new to netbox and I found it amazing to see this early implementation. Btw, what if instead of serialized_value we had a json_value to unify the custom field representation?
netbox=# select json_value,jsonb_typeof(json_value) from extras_customfieldvalue;
                          json_value                           | jsonb_typeof
---------------------------------------------------------------+--------------
 {"employee": {"age": 30, "city": "New York", "name": "John"}} | object
 "my project"                                                  | string
 false                                                         | boolean
 10102                                                         | number

Custom fields are not intended to store large amounts of data. Values are limited to 255 characters. #1349 will introduce config contexts in v2.4, which will support arbitrary data in JSON formats applied to a set of device/VMs (categorized by site, role, etc.). However, the attachment of arbitrary free-form data to individual objects is out of scope for NetBox.


"Large select" example vertically overflows in Forms component documentation

The second example in the Sizing section of the documentation of the Forms component starts with a dropdown which shows by default a "Large select" option. The bottom of this text is not visible in my Google Chrome 60, at default zoom level.
There is no problem with the 2 following selects.

Hi @Chealer, Can you send a screen capture please? I can't reproduce it

That's on windows, right?

Nope, I'm running Debian 9.

Hi @andresgalante, here is a screenshot showing a worst case (each character is affected).

This is taken from Firefox 55.0 on Windows 8 at default zoom level.

@Chealer @houfio I see! that really sucks! I'll start a virtual machine and try to fix it.
Thanks a lot for reporting it and sending the screenshots
@Johann-S I think this should be labled as "bug" and "css"

Yep my bad thanks @andresgalante 👍

@Chealer Are you sure you are running beta?
This is where the height is set: https://github.com/twbs/bootstrap/blob/v4-dev/scss/_forms.scss#L172-L176
And on reboot the line-height is set to inherit
I can't reproduce the issue on browserstack with any browser, sorry 😞
IF you can guide me I can try send a PR, or you can actually send it yourself!

@andresgalante, I am "running" whatever the page I linked to uses, which appears to be a beta indeed, though it is the default documentation the homepage links to.
I do not master CSS, it is hard for me to say why the display is wrong.
I even get overflow if I try with MSIE, though it is less important in this case (only some letters, as shown by houfio).

Sorry @Chealer if since I can't reproduce it I can't fix it. But this is a bug.

No problem @andresgalante, thanks anyway for your diligent effort.
It should not be hard to reproduce though. I also see the issue with Chromium 60 on Debian 9.0. Only some letters are affected there (like houfio again).

Windows 10, Chrome v60.0.3112.113 (64bit)
Display 1680x1050

Edge Browser and Internet Explorer
Large is same as Default

To fix for me need recalucalculate height:

instead of
height: calc(2.3125rem + 2px);

@githubjeka Yeap, the bug is definitely on the height.
If I can help you to create a PR and look for the root of this variable please let me know.

The bug is not only on select element, i get the same result with input text :/

@andresgalante : Sorry @Chealer if since I can't reproduce it I can't fix it. But this is a bug.

I can reproduce this with a fresh install of Windows 10 1703 and Egde. Do you have a 4K display or something? Maybe a DPI other than 100% makes this a non-reproducible error on your side?

I have a mac :)

I suspected it. You said earlier that you created a VM. Could it be possible that the VM was running with a different DPI setting than 100% on that Mac? We have an iMac 5K at work and when you spin up new VMs in Parallels, the DPI setting is adjusted so everything is not only readable by ants. Could be that this is the issue why you can't reproduce this.
Sorry that I can't be of any other help, but debugging CSS stuff is really no my thing. I am glad that I can put Bootstrap to user without any modifications to CSS where I need it :)

There is definitely problem with input and select controls. You can reproduce this problem with free virtual machines from Microsoft: https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/
Download Windows 7 with IE11 and Windows 10 with MS Edge images. Run it in VirtualBox, install latest Chrome and Firefox. Open Bootstrap 4 beta documentation https://getbootstrap.com/docs/4.0/components/forms/ and use Dev tools to change page content. Note that both images are running with 100% DPI (default) settings.  Browserstack is not a reliable method for testing these things, especially when everyone who is using the most common browser on the most common OS is experiencing these issues even though you say you can't reproduce it.

Of course it is. it would be great if anyone sends a PR to fix it. I just can't reproduce it.

You can reproduce it on a Windows or Ubuntu VM.

I could, just not have the time to create a VM and make it run to fix this.

@Julieng50 solution works. However, it even works when only using:
select {
    padding-top: .4rem !important;
    padding-bottom: .4rem !important;
}

Modifying input fields is not necessary.

inputs
Issue occurs because line-height is smaller than input "carret".
Someone can tell that line-height is 1.25 and font size is 1rem. But not for all fonts space needed to render those fonts is the same.

select
The main issue is height on "select" (select.font-control:not() ... selector)
It is almost correct for "Arial" but not necessary for others fonts.

how to fix this.
This is really tricky and for general purpose I don't know how. Using height for "inputs" causes vertical align issues. For "selects" the same situation.
@andresgalante
Try different font-family and I believe that you can reproduce this issue

Found this https://iamvdo.me/en/blog/css-font-metrics-line-height-and-vertical-align
Maybe this help to understand issue more precisely and someone can make PR :)

I believe this bug was actually fixed in #23150 back in July, merged just after the beta was made. If you use bleeding edge v4-dev, the issue is not present. This might be the cause of some confusion of the above comments.

your're right @dracos already fixed with our last dist files from v4-dev see : https://codepen.io/Johann-S/pen/MEpBdR

select is fixed but this bug also mention on inputs

You could easily create a codepen and see that the height fix of #23150 applied to all .form-control-lg including inputs.

@dracos
You are right. Sorry for noise.

Please stop closing issues that aren't fixed properly. The issue still exists for even normal selects in Chrome on Windows. Characters are still being cut off as the text is still not centered vertically properly.

Nothing before in this issue has talked about the default select being cut off.

See my Codepen @dracos which use our last dist file : https://codepen.io/Johann-S/pen/MEpBdR

Yes I agree the text is cut off the bottom of the normal select, I can see that myself in IE. But this ticket and the screenshots above was about the -lg overhang, which was fixed by the PR I gave. It took me long enough to track that down that fix, and having git issues contain many overlapping different bugs does not help matters, witness the many confused tickets in this area.

yep that's right :+1: so close and I won't reopen that issue, track #23347 please

The -lg and normal selects should not be treated as separate issues. They conform to the same element, the only thing different is the sizing. Creating a solution that requires individual changes to each sub-style is not efficient in any way, ESPECIALLY considering we are now using rem. There should be a solution which, when applied to select.form-control, will scale efficiently no matter what sizing changes are made in regards to font/padding.

I agree. The issue of this issue, as it was described and followed up, was that -lg was using 2.3125rem rather than 2.875rem for its height, which was a plain bug due to it using the wrong variable, and that was fixed in https://github.com/twbs/bootstrap/pull/23150. #23347 seems like a more appropriate issue to track the general issue to do with sizing, given that the obvious issue with -lg has been fixed. I hope that explains thing properly, as nothing I've said is contradicting what you are saying.

Regarding issue tracking, it is poor practice to use a report of several issues to track an important issue. Someone looking for a report of this bug will unlikely look at a report with a title as vague as "v4 form input issues".
On the other hand, I did not know the bug I reported here was caused by a bug in Bootstrap itself, so this ticket's title is not optimal to track the Bootstrap bug exposed.
I would recommend to either:

open a new ticket tracking the underlying bug
reopen this ticket and possibly retitle it
retitle #23347 with a more specific name


RLongAdder has a delay bug

Expected behavior
I used the 'org.redisson.api.RedissonClient.getLongAdder()'in the distribution envrionment, when I added in the local node,I hope the other nodes get the last value immediately.
Actual behavior
But, the other node get the value that is a very old value some times.
Steps to reproduce or test case

Run more than two redisson nodes
Updating local adder values for multithreading
Get the adder value on the other redisson nodes

Redis version
5.0.0
Redisson version
3.17.3
Redisson configuration
Config config = new Config();
 String[] nodes = redisProperties.getCluster().getNodes()
                .stream()
                .map(node -> "redis://" + node)
                .toArray(String[]::new);
        config.useClusterServers().addNodeAddress(nodes)
                .setPassword(redisProperties.getPassword());

LongAdder uses pubsub. You need to check your network

LongAdder source

There are no problems with the network.

There are no problems with the network.
I roughly looked the source code of LongAdder, the method of add() has no any pub opration, it's work with method of sum()?

And I will call the sum () method every time after called add (), but some nodes will get the old value after calling add and sum methods. For example, the current adder value is 100, but the value of other nodes may be 80. Is there any caching logic here.

There is no caching all nodes add current value to the sum


wificfg example: Client doesn't get an IP address

I compiled the wificfg example and flashed it onto a Wemos D1 (in fact on several boards of those). Tried to connect to the wifihot spot, but the client never receives (or finishes reception) of an IP address. Tried with an Android phone, an Android  tablet and a an old Blackberry playbook. None succeeds within my time of patience (about a minute). Is there anything to be configured prior to building?
I had poked around a little when I tried to integrate the wificfg library into a project of my own and it seems as if the dhcpserver never gets scheduled...

Quickly tried my own fork on a Wemos D1 and the wificfg example did work, using the two Android devices I tried. Could you please give that a try for a start see https://github.com/ourairquality/esp-open-rtos Could try erasing the D1 flash completely, and perhaps even checking out the source again just to be sure. I have seen somewhat flaky AP connections at times, but not sure of the issues, but it usually works with a few retries.

Hi,
thanks for your quick reply. Completely erasing the flash memory and reflashing the SW did the trick. Apologies for not thinking about flash erasure myself.


Add app.add_state(inital_state) convenience method

What problem does this solve or what need does it fill?
I want to initialize a state to an arbitrary state, not just default.
Right now I am using app.add_state::<MyState>()
What solution would you like?
Add a convenience method app.add_state(MyState::MyArbitraryState) to App
What alternative(s) have you considered?
I read through the motivation for the new state system but it doesn't mention why only the default trait is used for initialization. This seems like a clear win to me so I would be surprised at any objections.

I tried out this idea in my own game and it works great:
use bevy::ecs::schedule::common_conditions::run_once as run_once_condition;
use bevy::ecs::schedule::run_enter_schedule;
use bevy::prelude::*;

pub trait AppExtension {
    fn init_state<S: States>(&mut self, initial_state: S) -> &mut Self;
}

impl AppExtension for App {
    // Remove when [add_state is added](https://github.com/bevyengine/bevy/issues/10731)
    fn init_state<S: States>(&mut self, initial_state: S) -> &mut Self {
        self.insert_resource::<State<S>>(State::new(initial_state))
            .init_resource::<NextState<S>>()
            .add_systems(
                StateTransition,
                (
                    run_enter_schedule::<S>.run_if(run_once_condition()),
                    apply_state_transition::<S>,
                )
                    .chain(),
            );

        // The OnEnter, OnExit, and OnTransition schedules are lazily initialized
        // (i.e. when the first system is added to them), and World::try_run_schedule is used to fail
        // gracefully if they aren't present.
        self
    }
}

If someone gives the greenlight I'll happily create PR 😄

I've done something a little different in my projects before. I'm not sure if its any better, but its another option.
pub struct BasePlugin(GameState);
impl Plugin for BasePlugin {
    fn build(&self, app: &mut App) {
        app.add_state::<GameState>();

        app.world
            .get_resource_mut::<NextState<GameState>>()
            .unwrap()
            .set(self.0);
   }
}

I just copied this from my project, but I don't think it'd be hard to change this to a method on App just like add_state
Is this any better/worse?

How about we split the add_state method into two methods: init_state<T: State>() and insert_state<T: State>(state: T). init_state would work like add_state, and insert_state will initialize the state to the state you pass in.
The names are inspired by the init_resource and insert_resource methods.


Cluster fails to deploy after ZooKeeper connectivity loss

Hi !
I'm having an issue where in case of connectivity loss with the ZooKeeper servers, the Marathon cluster would be inoperable without manual intervention.
Here's some details about the issue:

Restart a ZooKeeper instance
Marathon's leader is automatically killed (and restarted) and another assumes the role of master
I do have access to Marathon's UI but any task operations stays in Waiting/Deploying forever
If I restart the two remaining Marathon services, everything is back to normal, I can deploy and scale my apps

I get that the leader might be killed if ZK connectivity is lost, but I thought the cluster would survive without my having to restart all Marathon instances.
I am running ZooKeeper 3.4.5, Mesos 1.0.0 and Marathon 1.1.2.
Regards,
Antoine.

Apparently, when this happens, forcing a new election through DELETE /v2/leader fixes the issue.
Right now, I have a cronjob that checks /v2/info for the issue and triggers the leader abdication. But somehow, I think Marathon should do it itself.

Unfortunately, this fix doesn't seems to always work, as the cluster can sometimes get into a fully blocked state without leader, making the API answer with a 500 Could not determine the current leader...
So, no workaround :(

I can second that. In that case, only a full rolling-reboot of the master cluster works on my side.

We have seen a very similar issue, both on marathon 1.1.x and now on marathon 1.3.0 with the new curator based leader election. We were hoping 1.3.0 fixed the issue but it has not.
We have seen the current leader (<IP_ADDRESS>) get a message like "ZooKeeper access failed - Committing suicide to avoid invalidating ZooKeeper state",  it just a second later it reelected itself "Defeated (LeaderLatchListener Interface). New leader: <IP_ADDRESS>:8080". After this, it appears to keep leadership but wasn't actually doing anything and manual intervention was needed (stop all marathons, then start them). I will try to reproduce this issue in a sandbox so that I can have some better logs.

Note: This issue has been migrated to https://jira.mesosphere.com/browse/MARATHON-2161. For more information see https://groups.google.com/forum/#!topic/marathon-framework/khtvf-ifnp8.


test gRPC with latest released version

What does this PR do?
Whenever there is a new gRPC version released, we have to update our agent build to ensure it's properly working as expected, this has quite a few drawbacks

does not test latest version
intermediate versions might not be properly tested
requires to update project: PR/review/build while it's a simple and minimal change.

This change will make build modify pom.xml to use the latest grpc dependency on every build.
This implies that doing a release (that requires no local modifications) will fail unless the change has been commited.
Checklist

[x] My code follows the style guidelines of this project
[x] I have rebased my changes on top of the latest master branch
[x] I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
I have updated CHANGELOG.asciidoc
[x] I have updated supported-technologies.asciidoc
Added an API method or config option? Document in which version this will be introduced
Added an instrumentation plugin? How did you make sure that old, non-supported versions are not instrumented by accident?

Author's Checklist

[x] testing locally by simulating a 1.27 -> 1.29 update

Related issues
#1156 : updating to latest version (1.29.0 as of today).

Codecov Report

Merging #1157 into master will increase coverage by 11.68%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master    #1157       +/-   ##
=============================================
+ Coverage     59.39%   71.08%   +11.68%
- Complexity       85     2259     +2174
=============================================
  Files           330      219      -111
  Lines         15010     9855     -5155
  Branches       2086     1279      -807
=============================================
- Hits           8915     7005     -1910
+ Misses         5487     2405     -3082
+ Partials        608      445      -163

Impacted Files
Coverage Δ
Complexity Δ

.../elastic/apm/agent/bci/bytebuddy/MatcherTimer.java
66.66% <0.00%> (-26.67%)
4.00% <0.00%> (+4.00%)
:arrow_down:

...c/apm/agent/bci/MatcherTimerLifecycleListener.java
73.33% <0.00%> (-20.00%)
3.00% <0.00%> (+3.00%)
:arrow_down:

...java/co/elastic/apm/agent/bci/ElasticApmAgent.java
72.69% <0.00%> (-1.07%)
42.00% <0.00%> (+42.00%)
:arrow_down:

...ic/apm/agent/okhttp/OkHttpRequestHeaderSetter.java

...stic/apm/agent/servlet/ServletInstrumentation.java

...lastic/apm/agent/kafka/helper/CallbackWrapper.java

...t/v5_6/ElasticsearchClientSyncInstrumentation.java

.../agent/servlet/AbstractServletInstrumentation.java

...tracing/impl/OpenTracingBridgeInstrumentation.java

...tic/apm/agent/profiler/ProfilingConfiguration.java

... and 108 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7d6d15...6e24cb6. Read the comment docs.