MetaArgs should be renamed to MetaFunctionArgs to match the rest of remix API types

Reproduction
the API Surface in Remix should be guessable.
System Info
N/A

Used Package Manager
pnpm
Expected Behavior
ActionFunction <-> ActionArgs
Loaderfunction <-> LoaderArgs
MetaFunction <-> MetaArgs
or
ActionFunction <-> ActionFunctionArgs
Loaderfunction <-> LoaderFunctionArgs
MetaFunction <-> MetaFunctionArgs
Actual Behavior
MetaArgs is inconsistent with the rest of the remix API

I'm not sure if it was intentional or accidental.
MetaArgs is different from LoaderFunctionArgs in that:

It's an interface and not a type
It's also a generic type (used to type data and matches)

export function meta({
  data,
  params,
  location,
  matches,
}: MetaArgs<
  typeof loader, // current route loader data type
  {
    "root": typeof rootLoader;      // specify types for parent loader matches
    "routes/parent": typeof parentLoader;
  }
>) {
  console.log(data, params, location, matches);
  const m = matches.find((m) => m.id === "routes/parent");
  console.log(m?.data.name);
  return [
    { title: "New Remix App" },
    { name: "description", content: "Welcome to Remix!" },
  ];
}

@kiliman yep, I'm aware, just a suggestion 😅

@kiliman just out of curiosity do you happen to know why its an interface? are there reasons to declaration merging on the interface?

No, I'm not sure of the reasoning behind it.
Anyway, it looks like the naming was an oversight. In the Remix v2.0 release, they renamed LoaderArgs => LoaderFunctionArgs. They might have forgotten to do the same for MetaArgs.
https://github.com/remix-run/remix/blob/main/CHANGELOG.md#:~:text=Renamed LoaderArgs/ActionArgs to LoaderFunctionArgs/ActionFunctionArgs
In fairness, the meta function also underwent a breaking change in v2, since now it returns an array instead of an object with keys.
https://remix.run/docs/en/main/start/v2#route-meta

I think they could just add the MetaFunctionArgs as an alias, deprecate MetaArgs, and drop it in v3.


spt 3.7.0

just reporting that spt is updatet to 3.7.0, ammunition, quests, hits and health feature are faulting.

I'm going to fix that given 3.7.0 was released. Thanks


Having issues upgrading to the latest version from v0

Do you want to request a feature, report a bug or ask a question?
I would like to ask a question about upgrading from the v0.x to the latest version. I have upgraded my existing Icon component to looks like this:
function Icon (props) {
  // support transform in browsers that dont provide support for transforms using CSS
  const {width, height, id, className, style, viewBox, transform} = props

  // using the default value for preserveAspectRatio
  /*
  */
  return (
    <svg
      transform={transform}
      viewBox={viewBox}
      preserveAspectRatio='xMidYMid meet'
      style={style}
      className={className}
      width={width}
      height={height}
    >
      <use xlinkHref={id} />
    </svg>
  )
}

I use it like this within my react-storybook:
import user from '../path/to/icon/user.svg'

.add('Icon', () => {
   const {id, viewBox } = user
   return (<Icon id={id} viewBox={viewBox} width={64} height={64} fillColor='black'  />)
})

What is the current behavior?
I see that the Icon rendered using the Elements in chrome devtools as:
<svg viewBox="0 0 1003 1000" preserveAspectRatio="XMidYMid meet" class="sc-hqyNC gqaKqq sc-bdVaJa edcQzG" width="64" height="64">
  <use xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="user">
     #shadow-root(closed)
  </use>
</svg>

and I see the svg sprite as:
<symbol viewBox="0 0 1003 1000" id="user"><path d="M4 955q-14-43 25-70 119-78 291-116-84-105-119-255-10-55-11-122t23-149 66-138Q363 0 502 0t222 105q43 57 67 138t22 149-11 122q-35 150-119 255 172 38 291 116 39 27 26 71t-61 44H64q-21 0-38-13T4 955zm60-17h875q-109-73-267-108l-79-16q-45 30-91 30t-92-30l-78 16Q172 865 64 938zm193-602q-13 103 16 208t96 186l10 14q56 65 123 65t123-65l9-14q67-82 96-186t16-208q-14-113-76-193T502 63t-168 80-77 193z"></path></symbol>

I dont see the icon in the browser despite having the information available in the browser which was not previously the case.
What is the expected behavior?
I expect to see the Icon rendered correctly,  I am averse to adding a custom runtime to my app because

I pass the result of importing the icon into a javascript/json based configuration

import user from '../path/to/user.svg'

const uiOptions = {
   a: {
     'ui:options': {
         layout: {
            wrapper,
             label
         },
         icons: {
            user: user
         }
    }
  }
}

This info is used to render a tree of components based on a schema.

I want to ensure that Icon is replacable with a component that renders font icons or something else.

I maintain tests that rely on passing a glyph being passed to a component if it is available within the config.

Do I need to add the snippet below to wherever I import the svg icon
import SpriteSymbol from 'svg-sprite-loader/runtime/symbol';
import sprite from 'svg-sprite-loader/runtime/browser-sprite';
import SpriteSymbolComponent from './icon.jsx';

const symbol = new SpriteSymbol({ /* symbol data */ });
sprite.add(symbol);

What is /*symbol data*/ here? Can I/ Should I add a custom runtime without creating a custom React component?
Also, this is the relevant part of the webpack configuration:
{
        test: /\.svg$/,
        loader: 'svg-sprite'
}

UPDATE
I have tried to create a runtime-generator file that looks like this:
const {stringifyRequest} = require('loader-utils')
const {stringifySymbol} = require('svg-sprite-loader/lib/utils')
const pascalCase = require('pascal-case')

module.exports = function runtimeGenerator (data) {
  const {symbol, config, context} = data
  const {spriteModule, symbolModule} = config

  const spriteRequest = stringifyRequest({context}, spriteModule)
  const symbolRequest = stringifyRequest({context}, symbolModule)

  const symbolName = pascalCase(symbol.id)

  return `
  const SpriteSymbol = require(${symbolRequest});
  const sprite = require(${spriteRequest});

  const ${symbolName} = new SpriteSymbol(${stringifySymbol(symbol)});
  sprite.add(${symbolName});

  module.exports = ${symbolName}
  `
}

I changed the config in webpack to look like this:
const config = JSON.stringify({
  runtimeGenerator: require.resolve('../svg-icon-runtime-generator.js')
})

and the loader looks like:

```js
{
  test: /\.svg$/,
  loader: `svg-sprite?${config}`
}

but I still dont see the svg referenced correctly.
I have maintained the same configuration as v0 and the entire webpack config is based on create-react-app. I have added it to a gist
Please tell us about your environment:

Node.js version: 8.0
webpack version: 1.x
svg-sprite-loader version: 3.05
OS type & version: Mac Sierra

Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

I did not need the runtime-config, I removed it in the end. The issue was much simpler:
function Icon (props) {
  // support transform in browsers that dont provide support for transforms using CSS
  const {width, height, id, className, style, viewBox, transform} = props

  // using the default value for preserveAspectRatio
  /*
  */
  return (
    <svg
      transform={transform}
      viewBox={viewBox}
      preserveAspectRatio='xMidYMid meet'
      style={style}
      className={className}
      width={width}
      height={height}
    >
      <use xlinkHref={`#${id}`} />
    </svg>
  )
}

The # needs to be added  because it is a href to the id, the v0 was already providing it, but it has now been removed.


I have some question

And I don't know where I can ask.. It about zsh.
you have set
setopt COMBINING_CHARS in .zshrc file to avoid Thai Language error.
But when I open new terminal this command somehow it didn't load for me, so I need to run exec zsh to activate it.

Can you post your zshrc file for reference? And how do you know it didn't load, what signs are you able to see?


Volunteer Management System(NGO) in Java

Title and Issue number
This is a Java program that allows NGOs (Non-Governmental Organizations) to manage their volunteers. The program provides functionalities to add, view, and delete volunteer records. Volunteers are identified by unique IDs and can be displayed individually or as a list.
Issue number #4226
Close #4226
Checklist:

[X] I have mentioned the issue number in my Pull Request.
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have created a helpful and easy to understand README.md

@MohitGupta121 @TusharKesarwani Sir kindly review the project and merged the pr

@MohitGupta121 @TusharKesarwani @pranjay-poddar Sir kindly review the pr and merged it

@TusharKesarwani @MohitGupta121 Sir please Kindly review it and label

@MohitGupta121 Sir please kindly review it also


Draft: feature/email prepopulate

resolves #20

lol rebase ate all my old commits

@Syncretik can you make sure the Help Grow Glo component looks the same as it currently does on main?
This branch:

vs main:


raise ValueError("matrix contains invalid numeric entries")

Hi. Thanks for your great project.
I have been trying to train Mask2Former on my own dataset but after 11859 iters, i get the following error.
Could it be because of my lr?
Any help is appreciated!
Cheers

[07/30 00:17:57] d2.utils.events INFO:  eta: 5:48:54  iter: 11859  total_loss: 45.91  loss_ce: 1.061  loss_mask: 0.1516  loss_dice: 3.623  loss_ce_0: 0.4758  loss_mask_0: 0.1392  loss_dice_0: 3.208  loss_ce_1: 0.6885  loss_mask_1: 0.186  loss_dice_1: 3.529  loss_ce_2: 0.7608  loss_mask_2: 0.1806  loss_dice_2: 3.595  loss_ce_3: 0.8406  loss_mask_3: 0.137  loss_dice_3: 3.551  loss_ce_4: 1.091  loss_mask_4: 0.1918  loss_dice_4: 3.637  loss_ce_5: 1.09  loss_mask_5: 0.2125  loss_dice_5: 3.718  loss_ce_6: 0.7337  loss_mask_6: 0.1847  loss_dice_6: 3.558  loss_ce_7: 0.9319  loss_mask_7: 0.1548  loss_dice_7: 3.535  loss_ce_8: 0.8802  loss_mask_8: 0.1762  loss_dice_8: 3.604  time: 1.1627  data_time: 0.0892  lr: 0.001  max_mem: 8978M
[07/30 00:18:11] d2.engine.train_loop ERROR: Exception during training:
Traceback (most recent call last):
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/detectron2/engine/train_loop.py", line 149, in train
    self.run_step()
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/detectron2/engine/defaults.py", line 494, in run_step
    self._trainer.run_step()
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/detectron2/engine/train_loop.py", line 395, in run_step
    loss_dict = self.model(data)
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/mnt/data/floorplans/Mask2Former/mask2former/maskformer_model.py", line 209, in forward
    losses = self.criterion(outputs, targets)
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/mnt/data/floorplans/Mask2Former/mask2former/modeling/criterion.py", line 222, in forward
    indices = self.matcher(outputs_without_aux, targets)
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/mnt/data/floorplans/Mask2Former/mask2former/modeling/matcher.py", line 179, in forward
    return self.memory_efficient_forward(outputs, targets)
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/mnt/data/floorplans/Mask2Former/mask2former/modeling/matcher.py", line 151, in memory_efficient_forward
    indices.append(linear_sum_assignment(C))
  File "/home/vvz/anaconda3/envs/mask2former/lib/python3.6/site-packages/scipy/optimize/_lsap.py", line 93, in linear_sum_assignment
    raise ValueError("matrix contains invalid numeric entries")
ValueError: matrix contains invalid numeric entries
[07/30 00:18:11] d2.engine.hooks INFO: Overall training speed: 11869 iterations in 3:50:01 (1.1628 s / it)
[07/30 00:18:11] d2.engine.hooks INFO: Total training time: 3:50:17 (0:00:16 on hooks)
[07/30 00:18:11] d2.utils.events INFO:  eta: 5:48:40  iter: 11871  total_loss: 47.28  loss_ce: 1.052  loss_mask: 0.1669  loss_dice: 3.675  loss_ce_0: 0.5069  loss_mask_0: 0.1637  loss_dice_0: 3.306  loss_ce_1: 0.6921  loss_mask_1: 0.1929  loss_dice_1: 3.571  loss_ce_2: 0.8097  loss_mask_2: 0.1854  loss_dice_2: 3.697  loss_ce_3: 0.8414  loss_mask_3: 0.1516  loss_dice_3: 3.655  loss_ce_4: 1.114  loss_mask_4: 0.2051  loss_dice_4: 3.781  loss_ce_5: 1.079  loss_mask_5: 0.2326  loss_dice_5: 3.772  loss_ce_6: 0.7579  loss_mask_6: 0.189  loss_dice_6: 3.71  loss_ce_7: 0.9967  loss_mask_7: 0.1659  loss_dice_7: 3.699  loss_ce_8: 0.8863  loss_mask_8: 0.1786  loss_dice_8: 3.685  time: 1.1628  data_time: 0.0814  lr: 0.001  max_mem: 8978M

Solved in #58 .
Cheers


Review challenge documentation.

Ensure documentation is linguistically sound.

docs reviewed and pushed to repo. Had Logan do a review so I could merge them.


shell script invocation error - RNtester project

i didnt able to run the RNtester project i got
/Users/xxx/Library/Developer/Xcode/DerivedData/RNTester-dndzugkxoobajrbxsjndfoiucebr/Build/Intermediates.noindex/RNTester.build/Debug-iphonesimulator/RNTester.build/Script-68CD48B71D2BCB2C007E06A9.sh: line 3: /Users/xxx/Desktop/react: No such file or directory
the run script is :
export NODE_BINARY=node
$SRCROOT/../scripts/react-native-xcode.sh RNTester/js/RNTesterApp.ios.js

solve it by removing >> RNTester/js/RNTesterApp.ios.js


support prior return values

Using prior values in options is wonderful. The "when" clause in flows, however, is not so much as it is VERY limited what you can base it off of.
To make things much more powerful with minimal work, I have extended the prior value syntax support into when clause of the flow runner.

@yippie Thanks for the contribution! I was honestly surprised we didn't have this already; it's a good addition. Could you add a new test to validate the change? Let me know if I can provide guidance on our testing strategies.


Impossible to delete remote collection

There is no way to delete a remote collection
I have a remote collection that was created previously.  There appears to be no way to delete it, either from the tool or from the half-baked website.
Desktop (please complete the following information):

OS: Mac OS
Installation Method: [e.g. homebrew]
App Version 2021.4.1

Hi there @gpasq!
Apologies for the confusion, you can delete a remote collection by entering the collection and using the "Delete Collection" button in the Sync menu to the top right:

Note that this requires you to have pulled the collection.
If you run into any further issues with the sync features, please send an email to<EMAIL_ADDRESS>and we'll be able to assist you further.

Got it, thanks.  The problem was that I didn't have the collection locally.


XamlDataDrivenToolTask does not take into account UseCommandProcessor when determining whether to use response files on Windows

Steps to reproduce
Create an XamlTaskFactory definition which uses CommandLineTemplate and will result in a command line over over 8191 characters in length.
Expected behavior
When UseCommandProcessor is set to true, the threshold for whether to use response files will be lowered from 32k to 8k.
Looking closer at XamlTaskFactory, there's no way to determine what the real executable is without guessing, so without major changes the best thing would be for a very clear error message to be logged.
Actual behavior
The 30k limit is still used so command lines over 8k will still attempt to be passed through to cmd. This causes unexpected behaviour - in my case it caused a random letter to be truncated at the 8191 mark.
This behaviour is counter to the comment which says we can ignore the 8K limit as cmd.exe is not used - but it is.
// If the command is too long, it will most likely fail. The command line
// arguments passed into any process cannot exceed 32768 characters, but
// depending on the structure of the command (e.g. if it contains embedded
// environment variables that will be expanded), longer commands might work,
// or shorter commands might fail -- to play it safe, we warn at 32000.
// NOTE: cmd.exe has a buffer limit of 8K, but we're not using cmd.exe here,
// so we can go past 8K easily.

The problem is caused in a combination of these three lines:
https://github.com/microsoft/msbuild/blob/6f41a4fa3c6957d430783b7d31fe16e6a13b6e8e/src/Tasks/XamlTaskFactory/XamlDataDrivenToolTask.cs#L466
https://github.com/microsoft/msbuild/blob/6f41a4fa3c6957d430783b7d31fe16e6a13b6e8e/src/Tasks/XamlTaskFactory/XamlDataDrivenToolTask.cs#L149
https://github.com/microsoft/msbuild/blob/6f41a4fa3c6957d430783b7d31fe16e6a13b6e8e/src/Utilities/ToolTask.cs#L1342
Related links:
https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation

Closing as while this is still a problem that ideally would have an error to detect, I appreciate the error is hard to implement since some people could be using XamlTaskFactory to invoke multiple exes, which would work.


chore(AndroidStudio): Use Android SDK Build Tools 21.1.2

Closes https://github.com/psycholold/TapToUnlock/issues/2

Please check if this is still working for you. I don't want to project configuration as 21.1.1 also works for me.
P.S. You need to ensure 21.1.2 is installed on you machine. Otherwise gradle will complain

Works like a charm.


Why this not working for only login attempts?

For Soft Deleted, param deleted_at; Laravel login attempt for a deleted user; attempt login will return false if user is deleted or user will be logout if in a deleted user.
How make invalidate user session if deactivated_at column is not null, like deleted_at?
Only using query when login attempts?

Hmm, okay i was understand.
This action can be conflict password reset action etc.


Crash on startup (server)

Minecraft 1.18.2
Fabric 0.14.4
Crash on startup (server) - Unknown about clientside
A mod crashed on startup!
net.fabricmc.loader.impl.FormattedException: java.lang.RuntimeException: Could not execute entrypoint stage 'preLaunch' due to errors, provided by 'adventure-platform-fabric'!

Full Logs

duplicate #36

oh, sorry. I thought there was no report since everything was closed

I was under the impression that 0.14.4 is not a beta release anymore? It's now installed by default by the fabric installer.

This announcement says it's still beta https://discord.com/channels/507304429255393322/507304986921664533/970314116956831774
Regardless I'll look at pushing a release when I get the chance

Weird that it installs by default then. But oh well, thanks for the reply.


fix: CVE-2020-26160/CVE-2020-27813/CVE-2018-1099

Codecov Report

Merging #7 (e764181) into main (2aad014) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head e764181 differs from pull request most recent head cb02552. Consider uploading reports for the commit cb02552 to get more accurate results

@@           Coverage Diff           @@
##             main       #7   +/-   ##
=======================================
  Coverage   76.08%   76.08%
=======================================
  Files          29       29
  Lines       11822    11822
=======================================
  Hits         8995     8995
  Misses       2391     2391
  Partials      436      436

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


prometheus meter analyzer produce IllegalArgumentException and NaN value

Please answer these questions before submitting your issue.

Why do you submit this issue?
[ ] Question or discussion
[x] Bug
[ ] Requirement
[ ] Feature or performance improvement

Bug
https://github.com/apache/skywalking/blob/62411964a46aa22d47319ca863feed3a39169548/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/prometheus/PrometheusMetricConverter.java#L87-L100
I think Line 91 should be:
.filter(t -> t != NIL && t._2.samples.length > 0)

because line 98 will throw an IllegalArgumentException if samples.length == 0.

https://github.com/apache/skywalking/blob/62411964a46aa22d47319ca863feed3a39169548/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamily.java#L75-L79
I think line 78 should be:
return new SampleFamily(Arrays.stream(samples).filter(sample -> !Double.isNaN(sample.getValue())).toArray(Sample[]::new), Optional.ofNullable(ctx).orElseGet(RunningContext::instance));

because sample's value may be 'NaN'.

@wankai123 I think you were just asking about this? Please take a look.

@hanahmily Could you confirm this? We faced the same issue in the latest OTEL collector too.

Which plugin did you use, prometheus-fetcher-plugin or otel-receiver-plugin?
AFAIK the otel latest version has the NaN issue.

I agree to filter the NaN value. We didn't take into account that the double values might be NaN.

I got the metrics like this by using otel-collector 0.30.0,  the NaN value from otel-collector, and it seems a duplicate and wrong metric. If we don't know why they send this, I agree to filter the NaN value for defensive.

Which plugin did you use, prometheus-fetcher-plugin or otel-receiver-plugin?
AFAIK the otel-collector latest version has the NaN issue, the version below 0.29.0 words well.

Latest version too.
According to prometheus/client_golang#85, the NaN is reasonable

Which plugin did you use, prometheus-fetcher-plugin or otel-receiver-plugin?
AFAIK the otel-collector latest version has the NaN issue, the version below 0.29.0 words well.

Latest version too.
According to prometheus/client_golang#85, the NaN is reasonable

I agree NaN is meaningful than 0 or any other way. But I think we should take care of it one way or another.

I think we also should add a NaNSampleFamily as a subclass of SampleFamily, what do you think? @hanahmily @Lighfer @wankai123

I think we also should add a NaNSampleFamily as a subclass of SampleFamily, what do you think? @hanahmily @Lighfer @wankai123

I think NaN value should just be handled in more details, aggregation operators or other methods in SampleFamily should take care of it, but I didn't understand the meaning of adding a NaNSampleFamily class, I think NaN value is meaningful when dealing with those samples where it comes from ,but meaningless when just put all NaN value  together？

My point is about, we need to put it works in all operations. But NaN is not a number or double, we need to be very careful about how to process it.
We could add a pull request to fliter it first. But more importantly, we should adopt this, rather than filtering it.


[SPARK-6030][CORE] Don't alignSize the computed shellSize for classes in the getClassInfo method

SizeEstimator gives wrong result for Integer on 64bit JVM with UseCompressedOops on, this pr fixes that. For more details, please refer SPARK-6030
@sryza, I noticed there is a pr to expose SizeEstimator, maybe that should be waited by this pr get merged if we confirm this problem.
And @shivaram would you mind to review this pr since you contribute related code. Also cc to @srowen and @mateiz

Can one of the admins verify this patch?

Jenkins, this is ok to test

Test build #27992 has started for   PR 4783 at commit 8a127dc.

This patch merges cleanly.

@advancedxy -- Thanks for the detailed writeup and the gist. I can see the problem but I am not sure this is the complete fix.  You are right that shellSizes don't need to be aligned to 8 bytes for superclasses and the alignment only needs to happen on object boundaries.  However I am not sure this completely extends all the way -- i.e. when the super class has say a 1 byte field then I don't think we can start counting from 13 (12 for Object + 1 for byte).
While looking around I found this very nice tool called the Java Object Layout [1] which prints out the layout with alignment, gaps given an object name. You can see an example of the alignment gaps I am talking about in [2]  where super-classes with boolean fields end up being aligned to 4 bytes by hotspot (I have no idea what the IBM / Oracle JVMs do).
I'll try to think if there is a more elegant fix and if not we can go with this.
cc @rxin
[1] http://openjdk.java.net/projects/code-tools/jol/
[2] http://hg.openjdk.java.net/code-tools/jol/file/6df1a9b2e5da/jol-samples/src/main/java/org/openjdk/jol/samples/JOLSample_06_Gaps.java

Ok, Thanks for the update. I will look into the jol and the example when I get some spare time.

Test build #27992 has finished for   PR 4783 at commit 8a127dc.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27992/
Test FAILed.

@shivaram -- I followed the JDK-8024912, JDK-8024913 and JDK-7007564, It looks like non-static field size is aligned to wordSize(8 bytes on 64bit JVM without UseCompressedOops, 4bytes otherwise, verified by the Sizeof code I mentioned on the jira). And the hotspot JVM wouldn't change that in short time.
This is the hotspot version, and I have no ideas about other version's of JVM. But I think we can use this setting up for now just like we did with the ALIGN_SIZE?
And this is what I get for now. If we (including others or some JVM experts) can agree on that, I will update the code. So, waiting for others' opinion now

@advancedxy -- Yeah it sounds good to me to use 4-byte shell size alignment without CompressedOops and 8 bytes otherwise (we can wait to see if there are other opinions). If we want to be extra careful we can put this inside a if(isHotspot) check.
BTW it would be great if you can check if this is consistent across Java 1.6, 1.7, 1.8 HotSpot versions.

@shivaram I updated the gist, you can look at the result. gist
Also you can just download the shell script and try it yourself. The script assumes you are on a Mac and installed java 6,7,8. And of course, you should look at first.
curl -o align-test.sh https://gist.githubusercontent.com/advancedxy/2ae7c9cc7629f3aeb679/raw/4f23d55d0f836c2d49f05378398f3c01d7ecdab2/align-test.sh && sh align-test.sh

Test build #28259 has started for   PR 4783 at commit 4b8aa90.

This patch merges cleanly.

Test build #28259 has finished for   PR 4783 at commit 4b8aa90.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28259/
Test FAILed.

Test build #28277 has started for   PR 4783 at commit a55f3e1.

This patch merges cleanly.

@shivaram, I add a bunch of comments in the code. I think it's time for you to review it now. And the previous failures are not related to this code change. Wonder why that would happen.
Also, I think you should review the wording in the comments as I am not a native speaker.

Test build #28277 has finished for   PR 4783 at commit a55f3e1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28277/
Test FAILed.

Jenkins, retest this please

Test build #28351 has started for   PR 4783 at commit a55f3e1.

This patch merges cleanly.

Test build #28351 has finished for   PR 4783 at commit a55f3e1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28351/
Test FAILed.

Test build #28363 has started for   PR 4783 at commit cb0af95.

This patch merges cleanly.

@shivaram I update the comment, please review again. And still unrelated test failure.

Test build #28363 has finished for   PR 4783 at commit cb0af95.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28363/
Test FAILed.

ping @shivaram, @srowen and @mateiz

Test build #28486 has started for   PR 4783 at commit 9f92469.

This patch merges cleanly.

Test build #28486 has finished for   PR 4783 at commit 9f92469.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/28486/
Test FAILed.

@advancedxy Do the tests pass on your local machine ? It looks the ExternalSorter uses a size-tracking hash map which in turn depends on the SizeEstimator. So I am wondering if the test failure is actually related to this PR.

I just run test-only for SizeEstimatorSuite. It do pass the tests. I thought the ExternalSorter was unrelated. I will test the ExternalSorter.

@shivaram ExternalSorterSuite failed on my local machine too. Do you have any thought why ExternalSortedSuite is failing.

I don't know much about the ExternalSorter. You could try to walk through the code and try to see how the size estimator changes are affecting it.
@mateiz @rxin might know how it depends on the SizeEstimator

@shivaram The reason why ExternalSorter failed is that It doesn't spilling files for these two failure tests.
( If we increase the input from 0 until 100000 to 0 until 200000, It will spilling files to disks, which passes the tests) However the input type for sorter is Iterator[(Int, Int)], and the older SizeEstimator gives 32 bytes and the new SizeEstimator gives the same 32 bytes for (Int, Int) (64 bit JVM with UseCompressedOops on assumed). So, It's very wried to see different results.
Seems @mateiz is busy. @jerryshao cloud you take a look at the failed tests, since you wrote some of the tests.
And, I believe there is another bug in the current SizeEstimator, Scala specialized Int, Long, Float, Double for Tuples, So, The size of (Int, Int) should be 24 bytes, rather than 32bytes. Verified by the method introduced by this article http://www.javaworld.com/article/2077496/testing-debugging/java-tip-130--do-you-know-your-data-size-.html.
I will take a look at the size problem.

@shivaram @srowen Tuple2(Int, Int) got specialized to Tuple2$mcII$sp class. But the Tuple2$mcII$sp is a subclass of Tuple2. So in our implementation, the specialized class will get two additional object references. (_1, _2 in superclass Tuple2, in our case). So, for Tuple2(Int, Int), SizeEstimator will give 32 bytes rather than 24 bytes. In theory, the Tuple2(1,2) class filed layout should be something like below.
scala.Tuple2$mcII$sp object internals:
OFFSET SIZE TYPE DESCRIPTION VALUE
0 4 (object header) 01 00 00 00 (0000 0001 0000 0000 0000 0000 0000 0000)
4 4 (object header) 00 00 00 00 (0000 0000 0000 0000 0000 0000 0000 0000)
8 4 (object header) 05 c3 00 f8 (0000 0101 1100 0011 0000 0000 1111 1000)
12 4 Object Tuple2._1 null
16 4 Object Tuple2._2 null
20 4 int Tuple2$mcII$sp._1$mcI$sp 1
24 4 int Tuple2$mcII$sp._2$mcI$sp 2
28 4 (loss due to the next object alignment)
Instance size: 32 bytes (reported by Instrumentation API)
Space losses: 0 bytes internal + 4 bytes external = 4 bytes total

But in practice, the size of Tuple2(1, 2) is 24 bytes. So is there any scala expert we can ping? I really want to know why Tuple2(1, 2) can be 24 bytes when the specialized version is involved.

What do you mean in practice Tuple2 is 24 bytes? How did you measure that? If the object layout tool shows it takes 32 bytes, doesn't it just take 32 bytes?

@rxin as I mentioned in the previous comment, I use the method in this article.
http://www.javaworld.com/article/2077496/testing-debugging/java-tip-130--do-you-know-your-data-size-.html
The related code is listed below. The scala library should be included in the classpath
// ----------------------------------------------------------------------------
/**
 * A simple class to experiment with your JVM's garbage collector
 * and memory sizes for various data types.
 *
 * @author <a<EMAIL_ADDRESS>Roubtsov</a>
 */
import scala.Tuple2;
import scala.Tuple2$mcII$sp;
public class Sizeof
{
    public static class DummyClass1 {}
    public static class DummyClass2 extends DummyClass1 {
        public boolean x;
    }
    public static class DummyClass3 extends DummyClass2 {
        public boolean y;
    }

    public static void main (String [] args) throws Exception
    {
        // "warm up" all classes/methods that we are going to use:
        runGC ();
        usedMemory ();

        // array to keep strong references to allocated objects:
        final int count = 10000; // 10000 or so is enough for small ojects
        Object [] objects = new Object [count];

        long heap1 = 0;

        // allocate count+1 objects, discard the first one:
        for (int i = -1; i < count; ++ i)
        {
            Object object;

            // INSTANTIATE YOUR DATA HERE AND ASSIGN IT TO 'object':

            object = new Tuple2(1, 2); // This gives 24 bytes. 64 bit vm
            // object = new Tuple2$mcII$sp(1, 2); // This gives 32 bytes. 64 bit vm

            // This gives 56 bytes per object.
            // 24B(Tuple2) + 16B(Integer)+ 16B(Integer) = 56B
            // object = new Tuple2(2 * i + 1, 2 * i + 2);

            if (i >= 0)
                objects [i] = object;
            else
            {
                object = null; // discard the "warmup" object
                runGC ();
                heap1 = usedMemory (); // take a "before" heap snapshot
            }
        }

        runGC ();
        long heap2 = usedMemory (); // take an "after" heap snapshot:

        final int size = Math.round (((float)(heap2 - heap1))/count);
        System.out.println ("'before' heap: " + heap1 +
                            ", 'after' heap: " + heap2);
        System.out.println ("heap delta: " + (heap2 - heap1) +
            ", {" + objects [0].getClass () + "} size = " + size + " bytes");
    }

    // a helper method for creating Strings of desired length
    // and avoiding getting tricked by String interning:
    public static String createString (final int length)
    {
        final char [] result = new char [length];
        for (int i = 0; i < length; ++ i) result [i] = (char) i;

        return new String (result);
    }

    // this is our way of requesting garbage collection to be run:
    // [how aggressive it is depends on the JVM to a large degree, but
    // it is almost always better than a single Runtime.gc() call]
    private static void runGC () throws Exception
    {
        // for whatever reason it helps to call Runtime.gc()
        // using several method calls:
        for (int r = 0; r < 4; ++ r) _runGC ();
    }

    private static void _runGC () throws Exception
    {
        long usedMem1 = usedMemory (), usedMem2 = Long.MAX_VALUE;

        for (int i = 0; (usedMem1 < usedMem2) && (i < 1000); ++ i)
        {
            s_runtime.runFinalization ();
            s_runtime.gc ();
            Thread.currentThread ().yield ();

            usedMem2 = usedMem1;
            usedMem1 = usedMemory ();
        }
    }

    private static long usedMemory ()
    {
        return s_runtime.totalMemory () - s_runtime.freeMemory ();
    }

    private static final Runtime s_runtime = Runtime.getRuntime ();

} // end of class
// ----------------------------------------------------------------------------

I took another look at the result. I think I got fooled by the result of Tuple2(1,2);  It gives 24 bytes as result, but the javac don't pick the specialized version class. The Tuple2$mcII$sp(1, 2) does give 32 bytes as result. So, The (1, 2) in Scala on 64 bit JVM do takes 32 bytes size. We get that right
The last thing left is to figure out why the ExternalSort keeping failing tests.

Ping @shivaram, @rxin. I finally got some time to look at the code.
The ExternalSorterSuite kept failing because there was no spilling when we just insert 100000 elements.
I don't know how 100000 was calculated, maybe you guys can give some related info. Before this pr, The size of  an integer instance is 24 bytes on a 64-bit JVM with UseCompressedOops on. Now, It's a correct 16 bytes. The ExternalSorterSuit inserts integers into buffers, so a reduced size of integer class results a smaller total estimated size, hence no spilling.
Another Note: When I was digging the ExternalSorterSuite and ExternalSorter, I noticed something wrong. The ExternalSorter uses the SizeTrackingPairBuffer, SizeTrackingPairBuffer[(Int, Int), Int] to be specifically in the test suite. The SizeTrackingPairBuffer[(Int, Int), Int] increases only 40 bytes per insertion(estimated by SizeEstimator). The 40 bytes  consisted of 24 bytes of Tuple2 and 16 bytes Integer. I changes the insertion number based on this fact (24 + 24 per insertion to 24 + 16 per insertion, 48 * 100000 = 40 * 120000). However 24 bytes of Tuple2 is not right, It should be the specialized version of Tuple2[Int, Int] which is 32 bytes. It's not SizeEstimator's fault as It can only get the Tuple2 class rather than scala.Tuple2$mcII$sp. I doubt it may be something related to the scala compiler, I will look into this problem or someone more familiar with scala compiler should.
As for this pr, I think it's good to go. Hope we can get it merged before 1.4

Jenkins, test this please

Can one of the admins verify this patch?

Jenkins, ok to test

Test build #31061 has started for   PR 4783 at commit db1e948.

Test build #31061 has finished for   PR 4783 at commit db1e948.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.
This patch does not change any dependencies.

Thanks @advancedxy for fixing the tests. This change LGTM. @rxin @srowen Could you also take a final look at this ?

ping @rxin or @srowen .

Needs a rebase.
I know it's not changed in this PR, but why are all those fields var? they can be initialized once if the initialize() method is removed.

Merged build triggered.

Merged build started.

Test build #31548 has started for   PR 4783 at commit 3f80640.

initialize() is invoked multiple times in the test suite. Maybe it's a hook for tests....

OK that all looks good. Let's see what Jenkins says.

A closer look at the ResetSystemProperties and SizeEstimatorSuite, the beforeEach in ResetSystemProperties is overriden by SizeEstimatorSuite. Should call super.beforeEach() in SizeEatimatorSuite to be stackable.
@JoshRosen, you added the ResetSystemProperties trait, should we add the  super.beforeEach() in the beforeEach() method in the SizeEstimatorSuite?

Test build #31548 has finished for   PR 4783 at commit 3f80640.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  case class ExecutorUIData(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31548/
Test PASSed.

@advancedxy I can't answer the ResetSystemPrroperties question off the top of my head and am a bit busy with other stuff right now, so why not try writing a simple standalone example with some printlns or something to figure out the proper override order, etc?

@JoshRosen @srowen, I already tested in the test suite. Just want to make sure if I don't miss anything obvious.
Here is the standalone reproduce sample.  https://gist.github.com/advancedxy/08a76fb618a3f804e108
If that's the case, I believe there is the BlockManagerSuite to be changed too. @JoshRosen when would you have time to look at that? I can send a pr for that if you are busy.

Merged build triggered.

Merged build started.

Test build #31624 has started for   PR 4783 at commit c4dcb41.

Test build #31624 has finished for   PR 4783 at commit c4dcb41.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31624/
Test FAILed.

Jenkins, retest this please.

Merged build triggered.

Merged build started.

Test build #31663 has started for   PR 4783 at commit c4dcb41.

Test build #31663 has finished for   PR 4783 at commit c4dcb41.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31663/
Test PASSed.

LGTM


MauveMultiThreadLoadTest_0 and others FAILED in Test_openjdk14_hs_sanity.system_aarch64_linux

Test Info
Test Name: MauveMultiThreadLoadTest_0
Test Duration: 7 sec
Machine: test-aws-rhel76-armv8-3
TRSS link for the test output: https://trss.adoptopenjdk.net/output/test?id=5f0edf597eaab3443ca8494c
Build Info
Build Name: Test_openjdk14_hs_sanity.system_aarch64_linux
Jenkins Build start time: Jul 15 2020, 05:39 am
Jenkins Build URL: https://ci.adoptopenjdk.net/job/Test_openjdk14_hs_sanity.system_aarch64_linux/140/
TRSS link for the build: https://trss.adoptopenjdk.net/allTestsInfo?buildId=5f0eded87eaab3443ca84781
Java Version
openjdk version "14.0.2" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.2+12)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.2+12, mixed mode, sharing)
This test has been failed 19 times since Jun 25 2020, 04:45 pm
Java Version when the issue first seen
[0.025s][error][cds] Unable to map CDS archive -- os::vm_allocation_granularity() expected: 65536 actual: 4096
openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment AdoptOpenJDK (build 14.0.1+7-202006252018)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 14.0.1+7-202006252018, mixed mode)
Jenkins Build URL: https://ci.adoptopenjdk.net/job/Test_openjdk14_hs_sanity.system_aarch64_linux/122/
The test failed on machine test-aws-rhel76-armv8-3 5 times
The test failed on machine test-aws-rhel76-armv8-6 1 times
The test failed on machine test-aws-rhel76-armv8-5 3 times
The test failed on machine test-packet-ubuntu1604-armv8-2 1 times
The test failed on machine test-aws-rhel76-armv8-2 1 times
The test failed on machine test-aws-rhel76-armv8-4 1 times
The test failed on machine test-aws-ubuntu1804-armv8-1 3 times
The test failed on machine test-aws-rhel76-armv8-1 3 times
The test failed on machine test-packet-ubuntu1604-armv8-1 1 times
2 other Mauve tests are also failing consistently for a long time:
MauveSingleInvocationLoadTest_HotSpot_0 (deep history)
and
MauveSingleThreadLoadTest_HotSpot_0 (deep history)

Console output:
[2020-07-15T10:35:42.981Z] LT  10:35:42.632 - First failure detected by thread: load-0. Not creating dumps as no dump generation is requested for this load test
[2020-07-15T10:35:43.401Z] LT  10:35:43.152 - Uncaught exception:
[2020-07-15T10:35:43.401Z] LT  java.lang.StackOverflowError: null
[2020-07-15T10:35:43.401Z] LT  	at java.lang.Throwable.getLocalizedMessage(Throwable.java:398) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at java.lang.Throwable.toString(Throwable.java:497) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at java.lang.String.valueOf(String.java:3388) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at java.io.PrintStream.println(PrintStream.java:1047) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:764) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at java.lang.Throwable.printStackTrace(Throwable.java:671) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at java.lang.Throwable.printStackTrace(Throwable.java:660) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at java.lang.Throwable.printStackTrace(Throwable.java:651) ~[?:?]
[2020-07-15T10:35:43.401Z] LT  	at net.adoptopenjdk.loadTest.reporting.OutputFilter.write(OutputFilter.java:60) ~[bin/:?]
...
[2020-07-15T10:35:45.826Z] LT  10:35:45.682 -   Ran     : 2743
[2020-07-15T10:35:45.826Z] LT  10:35:45.682 -   Passed  : 2740
[2020-07-15T10:35:45.826Z] LT  10:35:45.683 -   Failed  : 0
[2020-07-15T10:35:45.826Z] LT  10:35:45.683 -   Uncaught: 3
[2020-07-15T10:35:45.826Z] LT  10:35:45.683 -   Result  : FAILED
[2020-07-15T10:35:45.826Z] STF 10:35:45.740 - **FAILED** Process LT  ended with exit code (1) and not the expected exit code/s (0)
[2020-07-15T10:35:45.826Z] STF 10:35:45.740 - Monitoring Report Summary:
[2020-07-15T10:35:45.826Z] STF 10:35:45.741 -   o Process LT  ended with exit code (1) and not the expected exit code/s (0)
[2020-07-15T10:35:45.826Z] STF 10:35:45.741 - Killing processes: LT
[2020-07-15T10:35:45.826Z] STF 10:35:45.741 -   o Process LT  is not running
[2020-07-15T10:35:45.826Z] **FAILED** at step 1 (Run Mauve load test). Expected return value=0 Actual=1 at /home/jenkins/workspace/Test_openjdk14_hs_sanity.system_aarch64_linux/openjdk-tests/TKG/../TKG/test_output_1594806171798/MauveMultiThreadLoadTest_0/20200715-103538-MauveMultiThreadLoadTest/execute.pl line 94.
[2020-07-15T10:35:45.826Z] STF 10:35:45.775 - **FAILED** execute script failed. Expected return value=0 Actual=1

related: https://github.com/AdoptOpenJDK/stf/pull/83

Given https://github.com/AdoptOpenJDK/stf/pull/83 has been merged, should validate this issue has been fixed & close.
FYI @lumpfish

The mauve test targets were renamed (MauveMultiThrdLoad), but are still failing (not with StackOverflowError, but with a test case failure, console output from recent jdk15 run):
LT  20:49:18.732 - Test failed
LT    Failure num.  = 1
LT    Test number   = 1899
LT    Test details  = 'Mauve[gnu.testlet.java.lang.Math.strictfp_modifier.cos]'
LT    Suite number  = 0
LT    Thread number = 2
LT  >>> Captured test output >>>
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 0)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 1)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 2)
LT  FAIL: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 3)
LT  got -4616189728938546314 but expected -4616189728938546315

The mauve test targets were renamed (MauveMultiThrdLoad), but are still failing (not with StackOverflowError, but with a test case failure, console output from recent jdk15 run):
LT  20:49:18.732 - Test failed
LT    Failure num.  = 1
LT    Test number   = 1899
LT    Test details  = 'Mauve[gnu.testlet.java.lang.Math.strictfp_modifier.cos]'
LT    Suite number  = 0
LT    Thread number = 2
LT  >>> Captured test output >>>
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 0)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 1)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 2)
LT  FAIL: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 3)
LT  got -4616189728938546314 but expected -4616189728938546315

It looks like this is a hotspot aarch64 (> jdk8) specific issue where Math.cos inside a static function with a strictfp modifier is not returning the expected value.
It may be that the strictfp modifier is not being honored, or that StrictMath.cos itself returns the wrong value. I'll try top create a test case to narrow it down.

It looks like this is a hotspot aarch64 (> jdk8) specific issue where Math.cos inside a static function with a strictfp modifier is not returning the expected value.
It may be that the strictfp modifier is not being honored, or that StrictMath.cos itself returns the wrong value. I'll try top create a test case to narrow it down.

I have created a standalone test case (
StrictfpTest.java.txt
) from the failing mauve test case which if I understand what the strictfp class modifier is meant to do correctly (to guarantee strictfp operations for all methods within the class with the strictfp modifier), appears to show
that the strictfp modifier simply does not work.
The test:

Declares a class with the strictfp modifier
Executes StrictMath.cos, and Math.cos inside a static method, a non-static method and a method which is also defined with the strictfp modifer
Executes StrictMath.exp, and Math.exp inside a static method, a non-static method and a method which is also defined with the strictfp modifer

Where the Math operations result in different values to their StrictMath counterparts (which is permissible according to the specification) those values are not overridden by the StrictMath result.
Here are two examples from running the test (jdk15 aarch64 and jdk8 x86-64_linux) :
java -cp .  StrictfpTest
Running on:
os.arch: amd64
os.name: Linux
java.runtime.version: 1.8.0_275-b01
java.vm.name: OpenJDK 64-Bit Server VM
java.vm.version: 25.275-b01
To recalibrate the reference values:
Set outputValueCos variable to -0.9999876894265599
Set outputValueExp variable to 0.2942869403572507
strictfp Math.cos: got -4616189728938546315 , expected -4616189728938546315
Pass
non static Math.cos: got -4616189728938546315 , expected -4616189728938546315
Pass
static Math.cos: got -4616189728938546315 , expected -4616189728938546315
Pass
strictMath.cos: got -4616189728938546315 , expected -4616189728938546315
Pass
strictfp Math.exp: got<PHONE_NUMBER>137637591 , expected<PHONE_NUMBER>137637590
Fail
non static Math.exp: got<PHONE_NUMBER>137637591 , expected<PHONE_NUMBER>137637590
Fail
static Math.exp: got<PHONE_NUMBER>137637591 , expected<PHONE_NUMBER>137637590
Fail
strictMath.exp: got<PHONE_NUMBER>137637590 , expected<PHONE_NUMBER>137637590
Pass

I have created a standalone test case (
StrictfpTest.java.txt
) from the failing mauve test case which if I understand what the strictfp class modifier is meant to do correctly (to guarantee strictfp operations for all methods within the class with the strictfp modifier), appears to show
that the strictfp modifier simply does not work.
The test:

Declares a class with the strictfp modifier
Executes StrictMath.cos, and Math.cos inside a static method, a non-static method and a method which is also defined with the strictfp modifer
Executes StrictMath.exp, and Math.exp inside a static method, a non-static method and a method which is also defined with the strictfp modifer

Where the Math operations result in different values to their StrictMath counterparts (which is permissible according to the specification) those values are not overridden by the StrictMath result.
Here are two examples from running the test (jdk15 aarch64 and jdk8 x86-64_linux) :
java -cp .
I am curious what if any openjdk test material exercises strictfp modifier (not that a straight-forward test case would catch this issue, as you say, it does read like you need a specific case where they return different to then verify the strictfp behaviour).

That is an interesting/subtle discovery @lumpfish ...
I am curious what if any openjdk test material exercises strictfp modifier (not that a straight-forward test case would catch this issue, as you say, it does read like you need a specific case where they return different to then verify the strictfp behaviour).

Browsing the openjdk code has confused me further - in https://github.com/AdoptOpenJDK/openjdk-jdk15u/blob/61c35054bd4b320479c1764a790cfb2c82909f3e/src/java.base/share/classes/java/lang/Math.java the implementations of Math.cos and Math.exp look like they just delegate to StrictMath.cos and StrictMath.exp anyway - so how can they give different results?
The output is the same if I run with -Xint (and -XX:+PrintCompilation to make sure nothing is being compiled).

Browsing the openjdk code has confused me further - in https://github.com/AdoptOpenJDK/openjdk-jdk15u/blob/61c35054bd4b320479c1764a790cfb2c82909f3e/src/java.base/share/classes/java/lang/Math.java the implementations of Math.cos and Math.exp look like they just delegate to StrictMath.cos and StrictMath.exp anyway - so how can they give different results?
The output is the same if I run with -Xint (and -XX:+PrintCompilation to make sure nothing is being compiled).

The following disables the Math.exp() intrinsic on jdk-17, and makes your testcase pass:
 java -XX:+UnlockDiagnosticVMOptions -XX:DisableIntrinsic=_dexp StrictfpTest

With Intrinsics disabled, Math.exp() uses this function: https://github.com/openjdk/jdk/blob/05a764f4ffb8030d6b768f2d362c388e5aabd92d/src/hotspot/share/runtime/sharedRuntimeTrans.cpp#L238
With Intrinsics on it uses this: https://github.com/openjdk/jdk/blob/b78cd633f6b885a6a3059042df8dd38bee43c21e/src/hotspot/cpu/x86/stubGenerator_x86_64.cpp#L6310
StrictMath.exp() uses native C library:
FdLibm.Exp.compute(a);
My reading of the spec and this article lead me to believe that a user method specifying strictfp will not make its use of java.lang.Math.exp() strictfp:
https://en.wikipedia.org/wiki/Strictfp

The following disables the Math.exp() intrinsic on jdk-17, and makes your testcase pass:
 java -XX:+UnlockDiagnosticVMOptions -XX:DisableIntrinsic=_dexp StrictfpTest

With Intrinsics disabled, Math.exp() uses this function: https://github.com/openjdk/jdk/blob/05a764f4ffb8030d6b768f2d362c388e5aabd92d/src/hotspot/share/runtime/sharedRuntimeTrans.cpp#L238
With Intrinsics on it uses this: https://github.com/openjdk/jdk/blob/b78cd633f6b885a6a3059042df8dd38bee43c21e/src/hotspot/cpu/x86/stubGenerator_x86_64.cpp#L6310
StrictMath.exp() uses native C library:
FdLibm.Exp.compute(a);
My reading of the spec and this article lead me to believe that a user method specifying strictfp will not make its use of java.lang.Math.exp() strictfp:
https://en.wikipedia.org/wiki/Strictfp

exp is buggy on 32 bit x86 if that's the question. I dug into it a while ago and the resolution is either to fix the assembler 🤢, ask Intel for a fix or remove the intrinsic. Never came around doing any of those. If it's not about that, sorry for the noise.

exp is buggy on 32 bit x86 if that's the question. I dug into it a while ago and the resolution is either to fix the assembler 🤢, ask Intel for a fix or remove the intrinsic. Never came around doing any of those. If it's not about that, sorry for the noise.

The java Math.exp() specification states:
The computed result must be within 1 ulp of the exact result. Results must be semi-monotonic.

So in the above example: https://github.com/AdoptOpenJDK/openjdk-tests/issues/1891#issuecomment-756192451
Math.exp(-1.2232D) exact to 20dp: 0.29428694035725071819
So Java Math.exp() with 16-bit double 1ulp accuracy is:
0.2942869403572507 or
0.2942869403572508

The java Math.exp() specification states:
The computed result must be within 1 ulp of the exact result. Results must be semi-monotonic.

So in the above example: https://github.com/AdoptOpenJDK/openjdk-tests/issues/1891#issuecomment-756192451
Math.exp(-1.2232D) exact to 20dp: 0.29428694035725071819
So Java Math.exp() with 16-bit double 1ulp accuracy is:
0.2942869403572507 or
0.2942869403572508

@aahlenst thanks that is useful, it quite possibly is.

@aahlenst thanks that is useful, it quite possibly is.

See also https://github.com/AdoptOpenJDK/openjdk-support/issues/182.

See also https://github.com/AdoptOpenJDK/openjdk-support/issues/182.

The mauve test targets were renamed (MauveMultiThrdLoad), but are still failing (not with StackOverflowError, but with a test case failure, console output from recent jdk15 run):
LT  20:49:18.732 - Test failed
LT    Failure num.  = 1
LT    Test number   = 1899
LT    Test details  = 'Mauve[gnu.testlet.java.lang.Math.strictfp_modifier.cos]'
LT    Suite number  = 0
LT    Thread number = 2
LT  >>> Captured test output >>>
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 0)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 1)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 2)
LT  FAIL: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 3)
LT  got -4616189728938546314 but expected -4616189728938546315

I'm suspecting those Mauve testcases are not valid, it's implying you can alter the ulp accuracy of a called functional libary, which I don't think you can reading the Java strictfp spec.
Where is the Mauve source for these tests?

The mauve test targets were renamed (MauveMultiThrdLoad), but are still failing (not with StackOverflowError, but with a test case failure, console output from recent jdk15 run):
LT  20:49:18.732 - Test failed
LT    Failure num.  = 1
LT    Test number   = 1899
LT    Test details  = 'Mauve[gnu.testlet.java.lang.Math.strictfp_modifier.cos]'
LT    Suite number  = 0
LT    Thread number = 2
LT  >>> Captured test output >>>
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 0)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 1)
LT  PASS: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 2)
LT  FAIL: gnu.testlet.java.lang.Math.strictfp_modifier.cos (number 3)
LT  got -4616189728938546314 but expected -4616189728938546315

I'm suspecting those Mauve testcases are not valid, it's implying you can alter the ulp accuracy of a called functional libary, which I don't think you can reading the Java strictfp spec.
Where is the Mauve source for these tests?

Confirmed from OpenJDK: https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-January/073458.html

Confirmed from OpenJDK: https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-January/073458.html

@andrew-m-leonard - thank you for getting to the bottom of this.  From what you have established the test cases are invalid (though I can see how the spec might be misinterpreted here: https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.4).
The mauve test cases are downloaded and built as part of the openjdk-systemtest repository build (see https://www.sourceware.org/mauve/cvs.html for the cvs repository location of the files) , and then inventory files used in the load tests specify exactly which tests within the test suite to run.  I'll create a PR to exclude these tests from the inventory(-ies).

@andrew-m-leonard - thank you for getting to the bottom of this.  From what you have established the test cases are invalid (though I can see how the spec might be misinterpreted here: https://docs.oracle.com/javase/specs/jls/se7/html/jls-15.html#jls-15.4).
The mauve test cases are downloaded and built as part of the openjdk-systemtest repository build (see https://www.sourceware.org/mauve/cvs.html for the cvs repository location of the files) , and then inventory files used in the load tests specify exactly which tests within the test suite to run.  I'll create a PR to exclude these tests from the inventory(-ies).

In fact the non-strictfp equivalent test fails in the same way (not surprising since that is effectively what the strictfp test actually is).  That test will also need to be excluded because its assumption is that the Math.cos result will be the same as that hard coded into the test from a reference implementation whereas in fact the result may vary by 1 ulp ( https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html#cos-double- ).
There are also additional subtests which fail as described in https://github.com/AdoptOpenJDK/openjdk-tests/issues/2120 where NaN is not returned unchanged on aarch64.  Excluding the test will also exclude those subtests, which at this point I assume are failing because of a real jdk issue,

In fact the non-strictfp equivalent test fails in the same way (not surprising since that is effectively what the strictfp test actually is).  That test will also need to be excluded because its assumption is that the Math.cos result will be the same as that hard coded into the test from a reference implementation whereas in fact the result may vary by 1 ulp ( https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html#cos-double- ).
There are also additional subtests which fail as described in https://github.com/AdoptOpenJDK/openjdk-tests/issues/2120 where NaN is not returned unchanged on aarch64.  Excluding the test will also exclude those subtests, which at this point I assume are failing because of a real jdk issue,

This is fixed (tests excluded) by https://github.com/AdoptOpenJDK/openjdk-systemtest/pull/400, closing.


feat!: revamp action

Update to latest actions/typescript-action
Complete refactor and improve code

❤


Retweet talk announcements automatically

The VSF twitter account should retweet everything mentioning it and a speaker's corner hashtag.

Quick update: I just applied for a developer account using the VSF twitter. We will be able to create twitter apps/bots after approval. We will not be able to gather tweet statistics etc yet, that will require an 'upgrade' of the account afterwards.

Quick update: I just applied for a developer account using the VSF twitter. We will be able to create twitter apps/bots after approval. We will not be able to gather tweet statistics etc yet, that will require an 'upgrade' of the account afterwards.

Do we want to write our own bot (run it as a separate cloudflare worker on an hourly cron job?), or would we rather use what seems to be the most common google script (see e.g. https://www.labnol.org/twitter-bots-tutorial-4796 by that script's creator)?

Do we want to write our own bot (run it as a separate cloudflare worker on an hourly cron job?), or would we rather use what seems to be the most common google script (see e.g. https://www.labnol.org/twitter-bots-tutorial-4796 by that script's creator)?

We decided to go for writing our own bot at the Feb 2nd 2021 Org meeting


Pass access token to the downstream without modification

Expected Behavior / New Feature
To have an option where in downstream a service handles authentication and authorization. The gateway should just forward the access token to the downstream service. What would be nice is to be able to configure Ocelot to allow downstream services to handle authentication and authorization.
Actual Behavior / Motivation for New Feature
It appears Ocelot assumes it should do central authentication and would send back "Unauthorized" to a request that has an access token and when authentication is configured in Ocelot, it will block all requests with or without token recognized by Ocelot.
Steps to Reproduce the Problem

Get an access token from some authority and it to a non-authentication configured Ocelot.
Configure Authentication for Ocelot and send a request with or without an access token from some authority other than Ocelot.

Specifications

Version: 16.0.1
Platform: Windows
Subsystem:

Have you tried disabling authentication on Ocelot?
Ocelot should just pass headers to downstream and what you want should work.

Have you tried disabling authentication on Ocelot?
Ocelot should just pass headers to downstream and what you want should work.

Have you tried disabling authentication on Ocelot?
Ocelot should just pass headers to downstream and what you want should work.
If that doesn't work. You can make token validation always pass.

Yes, I didn't configure authentication in the gateway before but when I send a request with an access token to the ocelot gateway it returns back with unauthorized even when the controller in the downstream is decorated with [Authorize].
How can I make token validation always pass?

Have you tried disabling authentication on Ocelot?
Ocelot should just pass headers to downstream and what you want should work.
If that doesn't work. You can make token validation always pass.

Yes, I didn't configure authentication in the gateway before but when I send a request with an access token to the ocelot gateway it returns back with unauthorized even when the controller in the downstream is decorated with [Authorize].
How can I make token validation always pass?

This this out https://stackoverflow.com/questions/49932596/ignore-jwt-bearer-token-signature-i-e-dont-validate-token

This this out https://stackoverflow.com/questions/49932596/ignore-jwt-bearer-token-signature-i-e-dont-validate-token

This this out https://stackoverflow.com/questions/49932596/ignore-jwt-bearer-token-signature-i-e-dont-validate-token

Thanks for the link you posted. I have tried to follow the link carefully, assuming if I enable authentication and disable all validation checks at the gateway, it will forward my request to the backend service unmolested and then hopefully it will work. But alas, it doesn't . The gateway sees it as its responsibility to evaluate security authentication. It assumes, whether the backend configures security or not , that an endpoint is secured or unsecured if it's authentication configuration is enabled.
When I send my access tokens in either case , they get rejected by the gateway . This is strange behavior. What is happening here?

This this out https://stackoverflow.com/questions/49932596/ignore-jwt-bearer-token-signature-i-e-dont-validate-token

Thanks for the link you posted. I have tried to follow the link carefully, assuming if I enable authentication and disable all validation checks at the gateway, it will forward my request to the backend service unmolested and then hopefully it will work. But alas, it doesn't . The gateway sees it as its responsibility to evaluate security authentication. It assumes, whether the backend configures security or not , that an endpoint is secured or unsecured if it's authentication configuration is enabled.
When I send my access tokens in either case , they get rejected by the gateway . This is strange behavior. What is happening here?

I am using Ocelot like that and it works. Please open a question on stackoverflow if you still have the issue.

@B6692

To have an option where in downstream a service handles authentication and authorization. The gateway should just forward the access token to the downstream service. What would be nice is to be able to configure Ocelot to allow downstream services to handle authentication and authorization.

To handle auth stage by downstream service, do the following:

Switch off auth feature for the route in Ocelot config at all!
If auth-token is a header, just forward the header value by Headers Transformation feature.
If auth-token is a query parameter, just keep it in query string of URL, and define appropriate Ocelot everything-route or CatchAll-route by Routing feature. Your token will be forwarded to downstream service for evaluation.

Am I right, @wast ? 😃

Header is propagated to downstream regardless Headers Transformation feature afaik.


feat/ update _partition_pdf_or_image_with_ocr to optionally use PaddleOCR

Currently, _partition_pdf_or_image_with_ocr uses pytesseract to detect text in images. The goal of this issue is to enable ocr partitioning to optionally use PaddleOCR as an alternative text detection model. In addition to updating _partition_pdf_or_image_with_ocr , add_pytesseract_bbox_to_elements will also need to be updated as it expects the bounding box format from pytesseract.
https://pypi.org/project/paddleocr/
In paddle, ocr.ocr(img) returns similar data to pytesseract.image_to_boxes(img)

https://github.com/yuanjie-ai/ChatLLM/blob/master/chatllm/llmchain/document_loaders/image.py
    from chatllm.llmchain.document_loaders import UnstructuredImageLoader

    loader = UnstructuredImageLoader(
        "/Users/betterme/PycharmProjects/AI/MeUtils/meutils/ai_cv/invoice.jpg",
        strategy='ocr_only'
    )
    data = loader.load()

https://github.com/yuanjie-ai/ChatLLM/blob/master/chatllm/llmchain/document_loaders/image.py
    from chatllm.llmchain.document_loaders import UnstructuredImageLoader

    loader = UnstructuredImageLoader(
        "/Users/betterme/PycharmProjects/AI/MeUtils/meutils/ai_cv/invoice.jpg",
        strategy='ocr_only'
    )
    data = loader.load()

add UnstructuredImageLoader-PaddleOCR

This PR https://github.com/Unstructured-IO/unstructured/pull/1811 is doing a ocr_only workflow refactor. And after the PR is merged, we might just need to add tests to double check paddle is working in ocr_only for this issue.


puppet gem/standalone, "feature aws is missing"

Basically the same issue as #230, except that I have no ability to use the guidance suggested there for changing where I do the gem install.
I'm using docker to have a quick workspace/sandbox to play in, see https://gist.github.com/bdruth/7aeb0641dee7f5b42e10 for how ruby/gems/puppet are being installed.
Running
docker build -t aws-tsm . && docker run -it --rm --env AWS_ACCESS_KEY_ID=$my_aws_id --env AWS_SECRET_ACCESS_KEY=$my_aws_secret aws-tsm

Yields
Sending build context to Docker daemon 3.072 kB
Step 0 : FROM cgswong/aws:latest
 ---> dc85ea86a7cb
Step 1 : RUN apk update && apk add ruby
 ---> Using cache
 ---> 580085b5c971
Step 2 : RUN gem install --no-rdoc --no-ri aws-sdk-core retries puppet
 ---> Using cache
 ---> 5e62b6b03eff
Step 3 : RUN puppet module install puppetlabs-aws
 ---> Using cache
 ---> 8c8f99a8a78d
Step 4 : RUN addgroup -S puppet && adduser -G puppet -D -S -s /bin/bash puppet
 ---> Using cache
 ---> eb6c46c530cf
Step 5 : USER puppet
 ---> Using cache
 ---> b705b460b026
Step 6 : ADD /ec2_instance.pp /home/puppet/ec2_instance.pp
 ---> Using cache
 ---> 5c771350c8cd
Step 7 : CMD puppet apply --modulepath=/etc/puppetlabs/code/modules -t --debug /home/puppet/ec2_instance.pp
 ---> Running in 71ec39268c6f
 ---> 2f69842d1fd3
Removing intermediate container 71ec39268c6f
Successfully built 2f69842d1fd3
Debug: Runtime environment: puppet_version=4.2.2, ruby_version=2.2.2, run_mode=user, default_encoding=UTF-8
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Loading external facts from /home/puppet/.puppetlabs/opt/puppet/cache/facts.d
Debug: Facter: Found no suitable resolves of 1 for ec2_metadata
Debug: Facter: value for ec2_metadata is still nil
Debug: Facter: value for agent_specified_environment is still nil
Debug: Facter: Found no suitable resolves of 1 for lsbminordistrelease
Debug: Facter: value for lsbminordistrelease is still nil
Debug: Facter: Found no suitable resolves of 1 for selinux_enforced
Debug: Facter: value for selinux_enforced is still nil
Debug: Facter: Found no suitable resolves of 1 for selinux_policyversion
Debug: Facter: value for selinux_policyversion is still nil
Debug: Facter: Found no suitable resolves of 1 for selinux_current_mode
Debug: Facter: value for selinux_current_mode is still nil
Debug: Facter: Found no suitable resolves of 1 for selinux_config_mode
Debug: Facter: value for selinux_config_mode is still nil
Debug: Facter: Found no suitable resolves of 1 for selinux_config_policy
Debug: Facter: value for selinux_config_policy is still nil
Debug: Facter: value for is_rsc is still nil
Debug: Facter: value for is_rsc is still nil
Debug: Facter: Found no suitable resolves of 1 for rsc_region
Debug: Facter: value for rsc_region is still nil
Debug: Facter: value for is_rsc is still nil
Debug: Facter: Found no suitable resolves of 1 for rsc_instance_id
Debug: Facter: value for rsc_instance_id is still nil
Debug: Facter: Found no suitable resolves of 1 for dhcp_servers
Debug: Facter: value for dhcp_servers is still nil
Debug: Facter: Found no suitable resolves of 1 for system32
Debug: Facter: value for system32 is still nil
Debug: Facter: ruby-augeas not available
Debug: Facter: value for augeasversion is still nil
Debug: Facter: Found no suitable resolves of 1 for xendomains
Debug: Facter: value for xendomains is still nil
Debug: Facter: Found no suitable resolves of 1 for lsbdistdescription
Debug: Facter: value for lsbdistdescription is still nil
Debug: Facter: Found no suitable resolves of 2 for iphostnumber
Debug: Facter: value for iphostnumber is still nil
Debug: Facter: Found no suitable resolves of 2 for swapencrypted
Debug: Facter: value for swapencrypted is still nil
Debug: Facter: value for zfs_version is still nil
Debug: Facter: value for sshdsakey is still nil
Debug: Facter: value for sshdsakey is still nil
Debug: Facter: value for sshfp_dsa is still nil
Debug: Facter: value for sshrsakey is still nil
Debug: Facter: value for sshrsakey is still nil
Debug: Facter: value for sshfp_rsa is still nil
Debug: Facter: value for sshecdsakey is still nil
Debug: Facter: value for sshecdsakey is still nil
Debug: Facter: value for sshfp_ecdsa is still nil
Debug: Facter: value for sshed25519key is still nil
Debug: Facter: value for sshed25519key is still nil
Debug: Facter: value for sshfp_ed25519 is still nil
Debug: Facter: Found no suitable resolves of 1 for lsbdistrelease
Debug: Facter: value for lsbdistrelease is still nil
Debug: Facter: Found no suitable resolves of 1 for lsbdistcodename
Debug: Facter: value for lsbdistcodename is still nil
Debug: Facter: Found no suitable resolves of 1 for lsbmajdistrelease
Debug: Facter: value for lsbmajdistrelease is still nil
Debug: Facter: Found no suitable resolves of 1 for lsbrelease
Debug: Facter: value for lsbrelease is still nil
Debug: Facter: value for ipaddress6 is still nil
Debug: Facter: value for cfkey is still nil
Debug: Facter: value for zpool_version is still nil
Debug: Facter: value for vlans is still nil
Debug: Facter: Found no suitable resolves of 1 for ec2_metadata
Debug: Facter: value for ec2_metadata is still nil
Debug: Facter: Found no suitable resolves of 1 for ec2_userdata
Debug: Facter: value for ec2_userdata is still nil
Debug: Facter: value for ipaddress6_eth0 is still nil
Debug: Facter: value for ipaddress6_lo is still nil
Debug: Facter: value for macaddress_lo is still nil
Debug: Facter: Found no suitable resolves of 1 for gce
Debug: Facter: value for gce is still nil
Debug: Facter: Found no suitable resolves of 1 for processor
Debug: Facter: value for processor is still nil
Debug: Facter: Found no suitable resolves of 1 for lsbdistid
Debug: Facter: value for lsbdistid is still nil
Debug: Facter: Found no suitable resolves of 1 for zonename
Debug: Facter: value for zonename is still nil
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Notice: Compiled catalog for f5fc92dfbce1.tds in environment production in 0.34 seconds
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Puppet::Type::Ec2_instance::ProviderV2: feature aws is missing
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Puppet::Type::Ec2_instance::ProviderV2: feature aws is missing
Debug: Creating default schedules
Info: Applying configuration version '1444828193'
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Puppet::Type::Ec2_instance::ProviderV2: feature aws is missing
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Puppet::Type::Ec2_instance::ProviderV2: feature aws is missing
Debug: Class[Main]: Resource is being skipped, unscheduling all events
Debug: Stage[main]: Resource is being skipped, unscheduling all events
Error: Could not find a suitable provider for ec2_instance
Debug: Finishing transaction<PHONE_NUMBER>3320
Debug: Storing state
Info: Creating state file /home/puppet/.puppetlabs/opt/puppet/cache/state/state.yaml
Debug: Stored state in 0.01 seconds
Notice: Applied catalog in 0.08 seconds
Debug: Applying settings catalog for sections reporting, metrics
Debug: Finishing transaction<PHONE_NUMBER>5760
Debug: Received report to process from f5fc92dfbce1.tds
Debug: Evicting cache entry for environment 'production'
Debug: Caching environment 'production' (ttl = 0 sec)
Debug: Processing report from f5fc92dfbce1.tds with processor Puppet::Reports::Store

Using terraform instead. Much easier.


update ticket confirmation sustainoss

Coverage remained the same at 81.034% when pulling 1da201f6c556381a64f288de4a2ca53896e79931 on piamancini-patch-1 into 7d03bd549467a2016144e3ac826aa8ba136a026e on master.

Will need to make other code changes to load template and then call it for sustain group.

Added this to my PR with support for timezone: https://github.com/OpenCollective/opencollective-api/pull/842


Logo

I thought of a huge graph which has the shape of a flower. I need to write an algorithm probably to make it. So I'll do this when I have a lot of time. Surely not now.

The LOD cloud is nice :-)

Probably there is RDF data available to print it. What I need is an algorithm which makes the shape of a flower. Probably fractal or something like that. I tried once with d3 to write graph rendering algorithm, but it was a complete failure... :-) I guess I have to try again...
I though of something like this:

I thought about it. I have to use a complex force layout. I can use any image as template. The attracting forces have most strength by the darkest pixels, while the repelling forces have their most strength by the brightest pixels. The nodes are slightly repelling each other. I can use 6 color of nodes: red, green, blue. cyan, magenta, yellow, key(black). The attraction can be calculated by the color levels of the image, e.g.
RBG -> CMYK
111 -> 0000 white
000 -> 0001 black
110 -> 0010 yellow
101 -> 0100 magenta
011 -> 1000 cyan
001 -> 1100 blue
010 -> 1010 green
100 -> 0110 red

CMYK works completely differently...
I think I'll stay by the RGB with my algorithm, or should I use both?
check it out...
white repells:
111/0000
black attracts
000/***1|111*
own color attracts
110 -> 0010 yellow
101 -> 0100 magenta
011 -> 1000 cyan
001 -> 1100 blue
010 -> 1010 green
100 -> 0110 red

Now what happens for example with a red (100,0110) and a cyan (011, 1000) node? (Should I have black nodes as well?)
red node (100, 0110)

111/0000 // strongest repelling
000/***1|111* // strongest attraction
110 -> 0010 // med attraction yellow
101 -> 0100 magenta // stronger attraction than yellow
011 -> 1000 cyan // weak repell
001 -> 1100 blue // stronger repell than cyan
010 -> 1010 green // weaker repell than cyan
100 -> 0110 red // second strongest attraction

so to order the color by attraction on red:
000/***1|111* // strongest attraction
100 -> 0110 red // second strongest attraction
101 -> 0100 magenta // stronger attraction than yellow
110 -> 0010 // med attraction yellow
010 -> 1010 green // weaker repell than cyan
011 -> 1000 cyan // weak repell
001 -> 1100 blue // stronger repell than cyan
111/0000 // strongest repelling

let's check cyan: 011 -> 1000
000/***1|111* // strongest attraction
011 -> 1000 cyan //2nd str
001 -> 1100 blue //3rd str
010 -> 1010 green //4th
101 -> 0100 magenta //5th
100 -> 0110 red //6th
110 -> 0010 yellow //7th
111/0000 // strongest repelling

according to the color circle:
red 100 -> 0110
	black 000 -> ***1|111*
	red 100 -> 0110
	magenta/darkorange 101 -> 0100 /
	purple/orange
	indigo/yellow (75'0'130 -> 0.423'1'0'0.490) / 110 -> 0010
	blue/yellowgreen 001 -> 1100 /
	cyan/green 011 -> 1000 / 010 -> 1010
	oceangreen (46'139'87 -> 0.669'0'0.374'0.455)
	white 111 -> 0000

cyan 011 -> 1000
	black 000 -> ***1|111*
	cyan 011 -> 1000
	blue/oceangreen 001 -> 1100 / (46'139'87 -> 0.669'0'0.374'0.455)
	indigo/green (75'0'130 -> 0.423'1'0'0.490) / 010 -> 1010
	purple/yellowgreen
	magenta/yellow 101 -> 0100 / 110 -> 0010
	red/orange 100 -> 0110 /
	darkorange
	white 111 -> 0000

I think I'll need ANOVA to find out whether the forces correlate to the color codes and how they do that...

Sadly I am not an expert of statistics, but I always wanted to learn how I can use ANOVA, so this is the best time I think...

According to this: http://stackoverflow.com/questions/22534203/does-libreoffice-calc-support-json-file-importing-sorting I can export a calc/excel table as CSV and convert it CSV->JSON with an online converter. So I can use calc to record my data.

what I need a matrix of colors and forces between them

1 for attract -1 for repell 0 for neutral?
hmm
000/***1|111* // +1
011 -> 1000 cyan //+2/3
001 -> 1100 blue //+1/3
010 -> 1010 green //0
101 -> 0100 magenta //0
100 -> 0110 red //-1/3
110 -> 0010 yellow //-2/3
111/0000 // -1

Okay the signed values are good enough...

![qqq](https://camo.githubusercontent.com/e3a32b7eab11884175bb4abc5416ca790b2e6bd2/687474703a2f2f746865677261706869637366616972792e636f6d2f77702d636f6e74656e742f75706c6f6164732f626c6f676765722f2d5643384652314f49746c382f545f69414c39584b3863492f41414141414141415372492f686a5841473644446e73452f73313630302f73756e666c6f7765722d56696e746167652d4772617068696373466169727932622e6a7067 =250x)

I'll draw it with svg and with some mathematical algorithms: https://github.com/inf3rno/colorforce/issues/3
So I won't create a force layout now, but I'll keep that project alive. I always wanted to learn more about ANOVA and data mining, so it will be a good practice to check whether there is correlation between the color codes and the forces I recorded based on the color wheel/circle.

I close this issue, I solve it in the other project. Btw. it is not necessary to have a logo, so it can wait. It will be a good hobby project.

https://www.jasondavies.com/sunflower-phyllotaxis/
here is a simple d3 script, which generates the inside of the sunflower. all I need is adding the leaves and the stem.

Other logo ideas:
https://thumbs.dreamstime.com/z/binary-flower-composition-which-depicts-computer-abstraction-form-cyberspace-44482931.jpg
https://cdn1.vox-cdn.com/thumbor/0aJP-GRwyF7ooPsh0_MoHrYYiKE=/0x32:648x397/1600x900/cdn0.vox-cdn.com/assets/3246009/binary-flower.png
https://thumbs.dreamstime.com/z/binary-rose-7068163.jpg
Maybe the last one is the best.

Ofc after resizing probably we won't see anything of this. :S It should be in SVG. The d3 script appears to be an appropriate solution. Another possible solution to use some kind of fractal to build the parts of the image. That is computable either.


[Feature][Module Name] jar模式支持告警和savepoint管理

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
希望jar模式支持告警和savepoint管理，flink还是有一些场景不太适合用flinksql来实现的，着一些任务目前没有任务失败告警，比较麻烦
Use case
No response
Related issues
No response
Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

告警和savepoint管理不区分jar还是sql任务

但是jar任务部署页面没有告警配置，所以导致jar任务失败，不会告警啊


Issue of Proceed button of Wyre and Ramp account.

I pressed this once. It went to Ramp. With Ramp open in browser I came back to the wallet and pressed it again. No response.

Same issue with this button. It went to ramp once. I came back. The button is not responding.

-Still this proceed button is not responding if I came back to the wallet and pressed it again.

@cakesoft-swati this is not an issue. There is already a browser window open, so the user shouldn't leave that and come back to the app. just like you switched the app, you can switch back to the browser window.
If you close the modal and click on buy button again then it will open the browser again but not if you have just done that.

Oh got it.
-Verified tis issue o dev app v1.5.0(245)


j2lint fails with python 3.10

j2lint templates/eos/
Traceback (most recent call last):
  File "/Users/tamas/.pyenv/versions/3.10.2/bin/j2lint", line 33, in <module>
    sys.exit(load_entry_point('j2lint==0.1', 'console_scripts', 'j2lint')())
  File "/Users/tamas/.pyenv/versions/3.10.2/lib/python3.10/site-packages/j2lint/cli.py", line 150, in run
    collection.extend(RulesCollection.create_from_directory(
  File "/Users/tamas/.pyenv/versions/3.10.2/lib/python3.10/site-packages/j2lint/linter/collection.py", line 86, in create_from_directory
    result.rules = load_plugins(os.path.expanduser(rules_dir))
  File "/Users/tamas/.pyenv/versions/3.10.2/lib/python3.10/site-packages/j2lint/utils.py", line 32, in load_plugins
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/tamas/.pyenv/versions/3.10.2/lib/python3.10/site-packages/j2lint/rules/JinjaTemplateIndentationRule.py", line 7, in <module>
    from j2lint.linter.indenter.node import Node
  File "/Users/tamas/.pyenv/versions/3.10.2/lib/python3.10/site-packages/j2lint/linter/indenter/node.py", line 11, in <module>
    MIDDLE_TAGS = list(flatten([[i[1:-1] for i in JINJA_STATEMENT_TAG_NAMES]]))
  File "/Users/tamas/.pyenv/versions/3.10.2/lib/python3.10/site-packages/j2lint/utils.py", line 106, in flatten
    if (isinstance(el, collections.Iterable) and
AttributeError: module 'collections' has no attribute 'Iterable'

This issue is resolved as part of https://github.com/aristanetworks/j2lint/pull/25


Avaandmete päev 2020?

Kas? Kes? Kus?
Nüüd on hea aeg end üles anda.
Siit saab veebi: https://github.com/okfn/opendataday/tree/master/content

Here is a localized version of the web:

https://github.com/okestonia/opendataday (http://opendata.ee/opendataday/et/)

Vitaly who presented at Open Data Day in Narva last year showed interest in doing something this year too.

Is there something going to be happening in Tartu aswell?

There seems to be an event in Tartu:

Estonias School of Data hosts an environmental data meetup on March 3rd in Tartu. We aim to bring together both environmental data creators and holders as well as data journalists, advocates and others bedazzled by environmental data, and take a collective look on currently existings enironmental data in Estonia as well as access to it. Thanks to Open Knowledge Estonias partnership with Ministery of Economy, the data meetup will offer an unique possibility to actually open up new important datasets by direct nudging of national open data work group. The environmental data meetup also celebrates World Open Data Day with events held worldwide.
Specific location to be announced.

https://www.facebook.com/events/948883462173909/

Anybody wants to work on FOI portal during Open Data Day weekend? I have personal interest to get it running and can meet up for mini-hackathon. There is ISOC meeting in LIFT99 on 7th from 1 pm to 3 pm and this might be used for Open Data Day discussions or FOI portal hacking, if anybody's interested. Also discussing historical FOI requests related to Zonefile published last year might be a relevant topic.


Kdenlive 23 Download error  403 Forbidden.

Brief description of your issue
The Url for repro is invalid  and it returning  403 Forbidden.
for working download like is https://download.kde.org/stable/kdenlive/23.08/windows/kdenlive-23.08.3.exe

winget install -e --id KDE.Kdenlive
Found Kdenlive [KDE.Kdenlive] Version 23.08.3
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://binary-factory.kde.org/view/Windows MingW/job/Kdenlive_Stable_mingw64/1289/artifact/kdenlive-23.08.3-1289-windows-gcc-x86_64.exe
An unexpected error occurred while executing the command:
Download request status is not success.
0x80190193 : unknown error
Steps to reproduce
run: winget install -e --id KDE.Kdenlive
Expected behavior
Download and install kdenlive
Actual behavior
not downloading ending in error
Environment
winget --info
Windows Package Manager v1.6.3133
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22631.2715
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.21.3133.0

Winget Directories
-----------------------------------------------------------------------------------------------------------------------
Logs                               %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\Diag…
User Settings                      %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\sett…
Portable Links Directory (User)    %LOCALAPPDATA%\Microsoft\WinGet\Links
Portable Links Directory (Machine) C:\Program Files\WinGet\Links
Portable Package Root (User)       %LOCALAPPDATA%\Microsoft\WinGet\Packages
Portable Package Root              C:\Program Files\WinGet\Packages
Portable Package Root (x86)        C:\Program Files (x86)\WinGet\Packages
Installer Downloads                %USERPROFILE%\Downloads

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Admin Setting                             State
--------------------------------------------------
LocalManifestFiles                        Disabled
BypassCertificatePinningForMicrosoftStore Disabled
InstallerHashOverride                     Disabled
LocalArchiveMalwareScanOverride           Disabled

Hi,
I'm getting the same error as below:

winget install --id=KDE.Kdenlive  -e
Found Kdenlive [KDE.Kdenlive] Version 23.08.4
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://binary-factory.kde.org/view/Windows MingW/job/Kdenlive_Stable_mingw64/1311/artifact/kdenlive-23.08.4-1311-windows-gcc-x86_64.exe
An unexpected error occurred while executing the command:
Download request status is not success.
0x80190193 : Forbidden (403).

https://github.com/vedantmgoyal2009/vedantmgoyal2009/issues/1532


Application with no Component Shows as Successful

If an Application is deployed with no Component and no Custom Action is set, it will execute and show a "nothing to do" message in the log, but it will show as Successful in the Timeline tab, and send a Successful email if it exists for the Application.

Tried taking out Credential and got msg "No Credentials for server, Unable to deploy" but again sent Successful email.

I think the 1st case is as designed. It shouldn't necessarily fail just because there's no activities to deploy (you may legitimately set it up to send an email for example and nothing else!) The 2nd example may be more problematic. I'll take a look.


acorn update does not pick up changes done to computeclass .

acorn version v0.6.0-rc1-6-gb57d7f88+b57d7f88
Steps to reproduce the problem:

Create a computeclass (say c1)
Deploy an app (test1) using this computeclass c1 (acorn run --update -n test1 --compute-class c1 . )
In my case , the app did not get scheduled to any node because of incorrect scheduling rules .
Make the required changes to the scheduling rules in the computeclass c1.
Update app created in step2  using the same compute class (acorn run --update -n test1 --compute-class c1 . )

This does not result in the changes in compute class to be applied to the apps.
Containers dont get restarted as part of this update.
Note -- If there is any change made to the app ( in changed some args in Acornfile) , then update picks up the changes made to th compute class and containers get redeployed successfully.
Expected Behavior:
Changes done  to computeclass should get picked up on an acorn update .

Talked briefly with @thedadams offline and the solution here is to likely add a --force flag to acorn update which would update the generation flag even if nothing changes. Then trying to get this desired behavior becomes running acorn update --force.

@tylerslaton did you confirm this works? If so, this issue can be closed

@cjellick the --force flag does not currently exist. It is something we would need to add.


new conversation service created each time

the current logic does the following: create a conversation service, upload the workspace.
there is no checking if a conversation service exists, we should add to an existing service, instead... and run the risk of too many workspaces instead of too many services

closed with #9


feat(fixer): implement fixer

Ref #124
Base on ESLint's fixer.

Fix texts per rule

sequentially processed
fix texts by A -> fix texts by B

Could revert fixed result

SourceCodeFixer#revertFixes

Next: We publish beta release.

[ ] rename .text to .output

Current rule's fixer Interface
export default {
    linter(context){}
    fixer(context){
        return {
            [Context.Str](node){
               report(node, {
                   fix: Context.fixer.insertText("test");
               }
            }
        }
    }
}

Current, I try to implement some rule and get the feel of fixer interface.

One report has only one fix.(not need multiple fix.)

report(node, {
                    message: "error",
                    column: indexOfSymbol,
                    fix: fixer.replaceTextRange([node.range[0] + indexOfSymbol, node.range[0] + indexOfSymbol + 1], symbol)
                });

fix should handle position as relative from node.
To be
report(node, {
                    message: "error",
                    column: indexOfSymbol,
                    fix: fixer.replaceTextRange([ndexOfSymbol, indexOfSymbol + 1], symbol)
                });

[ ] update d.ts

export default {
    linter: reporter,
    fixer: reporter
}

Pros

simple

vs.
export default class CustomRule {
    linter: reporter
    fixer: reporter
}

Pros

extensible
available share state between linter and fixer

[ ] We need to support length ?

Does We renam fixer to fixCommand.
It is clearly that fixer does command pattern.

Merge! as --experimental


Don't use group size as group identifier

At the moment we are only considering one group of each size, but there may someday be cause to want to use a different 4096 bit group. Our group lookup keys shouldn't just be those integers representing the size.

Never mind. In this repo the map keys are strings.

It's also consistent with how OpenSSL does it via SRP_get_default_gN, no?


Reworked the READMEs and the way we show the XKCD example.

Pull Request Test Coverage Report for Build 22

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 21:
0.0%

Covered Lines:
282

Relevant Lines:
282

💛  - Coveralls


Unsuccessful authentication without errors/warnings

At first the authentication worked fine and I could see the API requests in the dashboard. But at some point the requests were not showing up anymore in the dashboard API overview.
I created a service account and stored the json file locally. Then I set the environment variable GCLOUD_PROJECT to the project ID and GOOGLE_APPLICATION_CREDENTIALS to the path of the json file.
from google.cloud import speech
client = speech.Client()
print(client._credentials.service_account_email)

Prints the correct service account email.
import io
with io.open(audio_file, 'rb') as f:
    audio = client.sample(f.read(), source_uri=None, sample_rate=48000, encoding=speech.encoding.Encoding.FLAC)
alternatives = audio.sync_recognize(language_code='de-DE')

The resulting transcript is correct, but nothing happens on the dashboard at
https://console.cloud.google.com/apis/api/speech.googleapis.com/overview?project=[project-ID]
I also tried  to initialize the client with the function speech.Client.from_service_account_json and  to authenticate by installing the Google Cloud SDK and gcloud auth application-default login, but without success.
The error count (Fehler) is increasing here but I don't know how to get additional information.

Windows 7
conda environment
python 3.5.2
google-cloud-speech 0.23.0

@elcombato thanks for reporting!

The resulting transcript is correct, but nothing happens on the dashboard...

That is pretty strange, because if you're getting results and a 200 response then it should be counting the requests on the dashboard.
Does the request count on the dashboard go up a few hours later?

@daspecster No it won't show up. The last activity is more than a week old and I used the API daily in the last week.
How can I evaluate that the client is correctly authenticated, to eliminate the possibility of an user error by me?

You can check the actual authenticated account and project via client._credentials.__dict__ and client._project

@dhermes With my code from above the object client has no attribute _project.
In client._credentials.__dict__ everything looks fine. Is it expected behavior, that client._credentials.expiry is None after the initialization of speech.Client() and after running client.sample() and sync_recognize() the variable client._credentials.expiry is set to the time of execution?

It's None at first because the credentials have yet to be refreshed. It should be the time of execution plus one hour after because it refreshed during the execution.

There is no plus one hour:
from google.cloud import speech
from datetime import datetime

client = speech.Client()

with open("speech.flac", 'rb') as f:
    audio = f.read()
sample = client.sample(audio, source_uri=None, sample_rate=48000, encoding='FLAC')
a = sample.sync_recognize()

print(client._credentials.expiry)
print(client._credentials.expired)
print(datetime.now())

# 2017-03-13 17:24:59.845493
# False
# 2017-03-13 17:25:40.128734

@elcombato what's datetime.utcnow()?

ok yea, that is 2017-03-13 16:32:29.200383. So expiry is in utc.
Any thoughts on the missing _project attribute?

Not sure, you can try executing credentials, project = google.auth.default() and see what it says. You can also explicitly pass the project into the Client constructor.

Both the credentials and project are correct.

@daspecster It seems there is no project used in the Speech API. Can you confirm? @elcombato Since that's the case, it seems the relevant project dashboard would be the project that owns the service account credentials.

@dhermes looking now.

Yeah, it's using google.cloud.core.client.Client instead of ClientWithProject.

The Speech authentication docs mention projects but kind of in a general way. Also, it used to be an API key based API so maybe that has something to do with it?
@lukesneeringer is there someone we can ask about how Speech supports project IDs?

Maybe @gguuss knows?

re: project IDs on Speech API projects, I have a few contacts from the team but I'm not sure they're on GH.

@gguuss Any update on this?

The project ID is only used when the content is a private GCS file.

Thanks @gguuss. That puts us more or less back where we started; project is not the culprit.
To be honest, I have no idea why the dashboard does not reflect anything, and that is a backend issue... :-/

The API calls are now showing up on my dashboard after updating to 0.24

How incredibly bizarre.
Closing this issue. Feel free to reopen if the problem pops back up.