Helvetica.afm file missing in Next.js

Bug Report
No Helvetica.afm file is being created when I use Next.js locally.
Description of the problem
I have to manually create a) the data folder and b) the Helvetica.afm file in .next/server/vendor-chunks every time I start the project.
Code sample
import { NextResponse } from 'next/server';
import PDFDocument from 'pdfkit';

export async function GET() {
  // Create a new PDF document
  const doc = new PDFDocument();

  // Create a buffer to store the PDF data
  const pdfBuffer: Buffer = await new Promise((resolve, reject) => {
    const buffers: Uint8Array[] = [];
    doc.on('data', buffers.push.bind(buffers));
    doc.on('end', () => {
      const pdfData = Buffer.concat(buffers);
      resolve(pdfData);
    });

    // Add content to the PDF document
    doc.fontSize(25).text('Hello, world!', 100, 100);
    doc.fontSize(12).text('This PDF was generated server-side with pdfkit in Next.js.', 100, 150);

    // Finalize the PDF
    doc.end();
  });

  // Return the PDF as a response
  return new NextResponse(pdfBuffer, {
    headers: {
      'Content-Type': 'application/pdf',
      'Content-Length': pdfBuffer.length.toString(),
      'Content-Disposition': 'inline; filename="output.pdf"', // or 'attachment;' for download
    },
  });
}

Your environment

pdfkit version: 0.15.0
Node version: 20.12.2
Next version: 14.2.9
React version: 18.3.1

did you find a solution? I also have the same error

The correct solution for the current version of next is declaring pdfkit as a serverComponentExternalPackage in the nextjsconfig:
const nextConfig = {
  experimental: {
    serverComponentsExternalPackages: ["@node-rs/argon2", "pdfkit"],
  },
};

The correct solution for the current version of next is declaring pdfkit as a serverComponentExternalPackage in the nextjsconfig:
const nextConfig = {
  experimental: {
    serverComponentsExternalPackages: ["pdfkit"],
  },
};

This "fix" worked for me on Next.js 14.2.16.
Thank you!

The correct solution for the current version of next is declaring pdfkit as a serverComponentExternalPackage in the nextjsconfig:
const nextConfig = {
  experimental: {
    serverComponentsExternalPackages: ["pdfkit"],
  },
};

This "fix" worked for me on Next.js 14.2.16.
Thank you!

Note that in Next.js 15 serverComponentsExternalPackages seemed to become a production feature as tstikvoort mentioned in #1467.
So you use:
const nextConfig = {
  serverExternalPackages: ["pdfkit"],
}


Unable to set schema in a GraphQL Config file and operations in parserOptions

Issue workflow progress

Progress of the issue based on the
Contributor Workflow

[x] 1. The issue provides a reproduction available on GitHub, Stackblitz or CodeSandbox

For example, you can start off by editng the
'basic' example on Stackblitz.

Please make sure the graphql-eslint version under package.json matches yours.

[ ] 2. A failing test has been provided

[ ] 3. A local solution has been provided

[ ] 4. A pull request is pending review

Describe the bug

Currently, if you have a project GraphQL config file, the ESLint plugin only looks at it and stops looking at the parserOptions ESLint config. This is a problem when the GraphQL config file only contains config for schema, but you need to set parserOptions.operations to a glob that selects JavaScript or TypeScript modules because your operations and fragments only exist as template strings with leading magic /* GraphQL */ comments. It doesn't make sense to move that glob into the GraphQL config file, because tools other than the ESLint plugin that attempt to use the GraphQL config won't understand how to process GraphQL from JavaScript or TypeScript.
To Reproduce Steps to reproduce the behavior:

https://stackblitz.com/edit/github-jpjt8b?file=eslint.config.js&view=editor
If you run npm run lint, there is an ESLint error:
Oops! Something went wrong! :(

ESLint: 8.43.0

Error: Rule `known-fragment-names` requires `parserOptions.operations` to be set and loaded. See https://bit.ly/graphql-eslint-operations for more info
    at requireSiblingsOperations (<anonymous>)
    at handleMissingFragments (<anonymous>)
    at Document (<anonymous>)
    at ruleErrorHandler (<anonymous>)
    at emit (<anonymous>)
    at applySelector (<anonymous>)
    at applySelectors (<anonymous>)
    at enterNode (<anonymous>)
    at enterNode (<anonymous>)
    at <anonymous> (<anonymous>)
    at runRules (<anonymous>)
    at _verifyWithFlatConfigArrayAndWithoutProcessors (<anonymous>)
    at _verifyWithFlatConfigArrayAndProcessor (<anonymous>)
    at verify (<anonymous>)
    at verifyAndFix (<anonymous>)
    at verifyText (<anonymous>)
    at <anonymous> (<anonymous>)

If you delete the .graphqlrc.json file and try again, there is no error.
Expected behavior

The ESLint plugin should load from a project GraphQL config file as a base, then merge over the top any config that is in parserOptions; e.g. you may define schema in GraphQL config but then only define operations in parserOptions.
Environment:

OS: macOS
@graphql-eslint/eslint-plugin: v3.19.1
Node.js: v20.3.1

Additional context
In our project, we want to use:
operations: "(components|helpers|hooks|pages)/**/*.{mts,ts,tsx}"

The ESLint plugin should load from a project GraphQL config file as a base, then merge over the top any config that is in parserOptions; e.g. you may define schema in GraphQL config but then only define operations in parserOptions.

GraphQL-ESLint doesn't merge graphql-config with parserOptions (and there are no plans to merge it).
GraphQL-ESLint uses either physical config AND if config missing parserOptions setting. You can disable search physical config by specifying skipGraphQLConfig: true
if you want to use physical graphql-config, then you should specify your operations as documents key in graphql-config


⚠Public pre-test of Silero-VAD v5

Dear members of the community,
Finally, we are nearing the release of the v5 version of the VAD.
Can you please send your audio edge cases in this ticket so that we could stress test the new release of the VAD in advance.
Ideally we need something like this https://github.com/snakers4/silero-vad/issues/369 (which we incorporated into validation when choosing the new models), but any systematic cases where the VAD underperforms will be good as well.
Many thanks!

I find the v4 for chinese single word 【bye】，is not good.
and the cantonese single word 【喺啊】and 【喺】 is not good.

I do not have any edge cases but it would be nice if you could change your benchmark methodology. There are a lot models out there by now. Adopting some new datasets like dihard3 etc. and comparing them against other sota models like pyannote would be dope.

Systematic cases would be:
The false-positives on ~silence. (introduced in v4)
Inaccurate end of segments, trailing usually includes up to ~1000 ms of "padding". (introduced in v4)
Maybe not systematic  but often the start of segment is ~200-100ms too late.

Hi, It's me again 😄
I've done some experiments on what I called "model expectation" w.r.t. the LSTM states' reset frequency.
Recall from the previous issue that my interest is mainly in  always-on scenarios, which consist of a VAD listening all the time to whatever is going on in the environment and triggering only when there's speech, which we'll assume to be a rare event. As such, the model would be expected to trigger only a few times (a day, say) w.r.t. the infinite audio stream that it keeps receiving over time.
The experiment consist in feeding long-ish stream of non-speech data to the model and check how often it hallucinates --- i.e., how often it sees speech when there is none. For that, we used Cafe, Home and Car environments from QUT-NOISE dataset, which contains 30-50 minute-long noise-only audio recordings.
In theory, we presume that one is advised to reset the model states only after it has seen speech, but we took the liberty to reset at regular time intervals irrespective of whether speech detection has been triggered.
The following plots show Scikit learn error rate (1-acc, which goes up to 100% == 1.00), therefore formulating the VAD as a frame-wise binary classification problem. X-axis show the frequency of model state resetting. Finally, v3 and v4 models are shown in blue and red colors, respectively.

pic 1
pic 2
pic 3

I'll formulate my conclusions later when I have time, just wanted to provide a heads-up asap since it's been a while since this issue has been opened.

A possible takeaway could be that this whole speech-expectation thingy reflects the training scheme, since the model has probably not seen (or it has, but very rarely) instances of non-speech-only data after the LSTM states have been initialized. IOW, if the datasets used for training the VAD are the same ones used to train ASR systems, all data contains speech, and that's what the model expects to see at the end of the day.

We focused on this scenario when training the new VAD since we had some datasets and our own issues when running noise only / "speechless" audios through the VAD.
The new VAD version was released just now - https://github.com/snakers4/silero-vad/issues/2#issuecomment-2195433115.
We changed the way it handles context now - we pass a part of the previous chunk as well as the current chunk and we made the LSTM component 2x smaller but improved the feature pyramid pooling (we has an improper pooling layer).
So in theory and in our practice the new VAD should work better with this edge case.
Can you please re-run some of your tests, and if the issue persists - please open a new issue referencing this one as context.
Many thanks!


Bump version to 2.0.2 for first AsciiDoc version.

Just bumping the version to 2.0.2 to differentiate between the final LaTeX based document and the first AsciiDoc based document.

Hi Bill - can you also add a row to the revision history table in Header.adoc?

Thanks Bill


current-rating event isn't fired on mouse leave

event is not fired when mouse leaves component, however I see from dev-tools that the current rating is reset to 0 ..

I just tested this in JSFiddle and it works correctly. Could you present an example of the code you are having issues with?

As there's been no response to this and we are unable to replicate it, I'll assume that the problem lies elsewhere, so I'll close this for now. Please let me know if this issue re-occurs and I'll look in to it.

Experiencing the same problem here: there is no event fired when the mouse cursor leaves the component that tells you what the currently selected rating is.


Make sure serverless operator handles reconciliation of PVC with expanded volume

Description
Introduce custom logic in the serverless operator that allows applying pvc manifests also in case of expanded volumes.
Normally it is not allowed as:
'kubeClient failed to update PersistentVolumeClaim ''serverless-docker-registry'' (namespace: kyma-system)  with strategy ''PATCH'': cannot patch "serverless-docker-registry" with kind PersistentVolumeClaim: PersistentVolumeClaim "serverless-docker-registry" is invalid: spec.resources.requests.storage: Forbidden: field can not be less than previous value',

Reasons
Serverless reconciliation should not fail in case of of PVC volume expansion.
Attachments
https://github.com/kyma-incubator/reconciler/issues/1379


Feature request: calling Prolog functions from JavaScript

Currently, Tau-Prolog requires Prolog queries to be passed as strings to the interpreter:
session.query("nonvar(1)");
But instead of passing a string, I'd prefer to call Prolog functions directly from JavaScript:
var passToProlog = 1;
session.call.nonvar(passToProlog);

I found one possible solution, but it requires a proxy to convert a function call to a query string:
var handler = {
    get: function(target, name) {
        return function(){
          return name+"("+Array.prototype.slice.call(arguments).map(JSON.stringify).join(",")+")";
        };
    }
};
var proxy = new Proxy({}, handler);

var passToProlog = 1;
var query_string = proxy.nonvar(passToProlog);
console.log(query_string); // output: nonvar(1);

Is it possible to query the interpreter without passing the query as a string?

I found one possible solution, but it requires a proxy to convert a JavaScript function call to a Prolog query:
var handler = {
    get: function(target, name) {
        return function(){
          return name+"("+Array.prototype.slice.call(arguments).map(JSON.stringify).join(",")+")";
        };
    }
};
var proxy = new Proxy({}, handler);

alert(proxy.nonvar(1)); // output: Hello, world

IMO a better interface design would be tagged template literals. That way you can interpolate values and Tau would know how to convert them into internal representations:
let num = someCalculation();
session.query`nonvar(${num})`;

@jarble using tagged template literals as proposed by @gilbert works for you? Does it address all the use case scenarios?

@nicoabie Tagged template literals might also be useful, though I haven't used them before. Would it be possible to use these literals with JavaScript expressions instead of variables, like this?
session.query`nonvar(${someCalculation()})`;

@jarble yeah, you can do that as well!
Take a look at the documentation https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals


[wip] Force static link to openblas on ci + add lapacke option for lapack

This pr forces static link to libopenblas.a on all pipelines to avoid the name clashing issue (except for windows where visual studio does not support static link to openblas according to this https://github.com/xianyi/OpenBLAS/wiki/How-to-use-OpenBLAS-in-Microsoft-Visual-Studio see bottom). On ubuntu and centos7 we changed from apt get/ yum openblas and lapack to building openblas(blas+lapack) from source. Also we add a flag to support LAPACKE api for non-mkl builds. The original description is below:
In mxnet we wrap lapack functions in c_lapack_api.h so that we hide the differences in the underlying blas/lapack libraries such as mkl, openblas, atlas, and accelerate.

For mkl we use/wrap the LAPACKE (https://www.netlib.org/lapack/lapacke.html) c interfaces which are the cleanest. For the rest of the libraries we wrap the old CLAPACK interfaces.

Openblas is by default built with lapack support and the binary will contain a full set of LAPACKE functions. Thus this poc adds a build option with which we can use the LAPACKE apis even for openblas.

This change will make ilp64 blas/lapack support easier as now we have the same wrapping logic or both mkl and openblas with USE_LAPACKE_INTERFACE = ON. Support for ilp64 mkl #19067 will automatically mean support for ilp64 openblas.

Known issue: linux distros ship openblas binaries WITHOUT lapack so this option can only work if we build openblas from source. This is not a problem for our mxnet binary distributions as we always use our own openblas build there. However, For users building mxnet from source they must be advised against using this option unless they also build openblas from source

@mxnet-bot run ci [centos-cpu]

@mxnet-bot run ci [centos-cpu]

@mxnet-bot run ci [macosx-x86_64]

@mxnet-bot run ci [unix-gpu]

@mxnet-bot run ci [unix-gpu]

@mxnet-bot run ci [unix-cpu]

So at commit 5f6e3f2d210b98f3139346e731f13b9ef4516ec2 we were doing a omp=1 openblas build. And I only added LINK-GOMP to the builds that had -DUSE_OPENMP=OFF. For other builds that have -DUSE_OPENMP=ON static link to openblas actually worked off the bat (without setting LINK-GOMP).
Docker Image:
https://github.com/apache/incubator-mxnet/blob/5f6e3f2d210b98f3139346e731f13b9ef4516ec2/ci/docker/Dockerfile.build.ubuntu#L98
Build Function:
https://github.com/apache/incubator-mxnet/blob/5f6e3f2d210b98f3139346e731f13b9ef4516ec2/ci/docker/runtime_functions.sh#L317
CI:
https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-19174/24/pipeline/44

@mxnet-bot run ci [all]

@mxnet-bot run ci [all]

asking cmake about how to find gfortran
https://gitlab.kitware.com/cmake/cmake/-/issues/21488

@mxnet-bot run ci [sanity]

@leezu would you review thanks!

@mxnet-bot run ci [unix-cpu]

@mxnet-bot run ci [windows-gpu]


NullPointerException running DynamoDBLocalFixture

I'm running it inside Eclipse having successfully done mvn compile and mvn eclipse:eclipse.
Exception in thread "main" java.lang.NullPointerException
at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess.initializeMetadataTables(SQLiteDBAccess.java:388)
at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess.(SQLiteDBAccess.java:224)
at com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccess.(SQLiteDBAccess.java:193)
at com.amazonaws.services.dynamodbv2.local.embedded.DynamoDBEmbedded.create(DynamoDBEmbedded.java:40)
at com.amazonaws.services.dynamodbv2.local.embedded.DynamoDBEmbedded.create(DynamoDBEmbedded.java:30)
at com.amazonaws.services.dynamodbv2.DynamoDBLocalFixture.main(DynamoDBLocalFixture.java:33)

The workaround described here didn't work for me:
https://forums.aws.amazon.com/thread.jspa?threadID=212169

It works for me. I copied libsqlite4java-linux-amd64-1.0.392.so (using linux) file into lib directory located in my root project directory. Then only:
System.getProperties().setProperty('sqlite4java.library.path', 'lib')
def localDynamo = DynamoDBEmbedded.create().amazonDynamoDB()

I no longer work on DynamoDB so I can't verify this. I'm closing the ticket - I hope someone else can reopen it if they still need help on it.


an identity matrix/operator missing

please check the last equation ( "For example, if we wish to apply the
X gate to the first qubit and then perform a CNOT between the second and third qubits we may express this transformation as"). The dimensions of the two sides of the equation are out of sync.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6516e6bf-4cd0-2d9b-4d18-565d7dbc38b6
Version Independent ID: a5b54d5e-8b9a-f7d5-2dac-ecd515478ba5
Content: Multiple qubits
Content Source: articles/concepts/multiple-qubits.md
Product: quantum
GitHub Login: @QuantumWriter
Microsoft Alias<EMAIL_ADDRESS>

Indeed the defined unitary acts on 6 qubits but the state is a product of 7 qubits. Good catch, I opened PR to fix it. Thank you!

Super, thx:)
Pozdrawiam/Best Regards
Piotr Sienkiewicz
Sent from my mobile thingy
On 14 Jun 2019, at 00:02, Mariia Mykhailova <notifications<EMAIL_ADDRESS>wrote:
Closed #446https://github.com/MicrosoftDocs/quantum-docs-pr/issues/446.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/quantum-docs-pr/issues/446?email_source=notifications&email_token=AGW4OITQBSE5ZQC54ALPGEDP2K7XBA5CNFSM4HXJ5RW2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOR7CM5PY#event-2412039871, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGW4OIWCCIVVDUFKKUZ6DP3P2K7XBANCNFSM4HXJ5RWQ.

Thx!
Pozdrawiam/Best Regards
Piotr Sienkiewicz
Sent from my mobile thingy
On 14 Jun 2019, at 00:02, Mariia Mykhailova <notifications<EMAIL_ADDRESS>wrote:
Closed #446https://github.com/MicrosoftDocs/quantum-docs-pr/issues/446.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/quantum-docs-pr/issues/446?email_source=notifications&email_token=AGW4OITQBSE5ZQC54ALPGEDP2K7XBA5CNFSM4HXJ5RW2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOR7CM5PY#event-2412039871, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGW4OIWCCIVVDUFKKUZ6DP3P2K7XBANCNFSM4HXJ5RWQ.


Add support for PodIdentityAssociation.eks.aws.upbound.io/v1beta1 resource

Description of your changes
This PR adds support for PodIdentityAssociation.eks.aws.upbound.io/v1beta1 resource.
Fixes https://github.com/upbound/provider-aws/issues/985
I have:

[ ] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

/test-examples="examples/eks/v1beta1/podidentityassociation.yaml"


BUG: EntityReference UpdateReguest bad URL translation

Doing an update request against an entity that uses an lookup alternate key results in a bad URL:
Case: Contact Entity has a lookup field to related Entity SocialSecurityNo on field ka_cpr.
The patch reguest generated when adding a alternate key to the EntityReference:
https://demolap.api.crm4.dynamics.com/api/data/v9.1/contacts(_ka_cpr_value='(00000000-0000-0000-0000-000000000000)')
It uses the empty Id of the entity reference and not the keycollection.
However i have not figured out how the URL should be manual either yet.
Somthing like?
https://demolap.api.crm4.dynamics.com/api/data/v9.1/contacts(_ka_cpr_value='socialsecuritynos(dca_socialsecurityno='xxxxxxxxxx')')

In each of theses, you can revert to old behavior by using executorganizationrequest and passing useWebAPI as false,
That will fall back to the older protocol while we resolve theses issues.
MattB

Thanks @MattB-msft - where do i set useWebAPI = cant seem to find it?

It's a parameter on that method call,  it has to be done per call.

@pksorensen
Can you include the snippet of the CdsServiceClient code you are using?
I'm not clear on your intent.

The method is called ExecuteCdsOrganizationRequest.

Its because many of use works wiht IOrganizationService, and did not notice theses. If instead the class is used, all these methods are there.
Also, just discovered that ExecuteCdsOrganizationRequest do not throw on error, it just returns null and you have to check yourself for the last error. Just imported 1.500.000 rows thinking everything vent fine :)

Sorry about the wrong message name :) working off my phone and memory.
Regarding the null return process,  yes that is the older pattern that the crmserviceclient uses. We wrap it with an exception throw patten when you use the iorganizationservice interfaces.
It's something we changed in the cdsserviceclient for iorganizationservice interfaces.
The older pattern is the lastError pattern,  thus on a null return, check last error for the actual error that were generated.

Everything makes sense now, also found some othre nice stuff from using CdsServiceClient directly.
On that note with LastError Pattern, i just vent over the code - dont the implemented way cause a potential thread issue?
I noticed how the locking happens inside the ExecuteCdsOrganizationRequest call and the lock is released before it reaches the
If(lastError) throw LastError ? so if doing a lot of parallel requests, then the last error can be reset before it reaches the throw step.
To make sure that do not happen for me, i moved the lock further out.
  private object lockobject = new object();
        public T Execute<T>( OrganizationRequest reg, bool userApi = false)

            where T : OrganizationResponse
        {
            lock (lockobject)
            {
                var os = service as CdsServiceClient;
                var a = os.ExecuteCdsOrganizationRequest(reg, useWebAPI: userApi) as T1;
                if (a == null)
                    throw os.LastCdsException;
                return a;
            }

        }

The tracer creates a stack of errors, and traces are specific to the particular connection.  the Lock behavior is there to prevent one calling thread from interrupting a connection via WCF that was initiated by a different calling thread,  LastError will be active at the moment of the call.
Its also why we guide folks to use the .clone process as that bypasses this whole issue and prevents one connection waiting on another connection problems.
The issue is the same with the HTTP Client , however it provides some built into call back mechanic's to help deal with it so the implementation there is a bit different.

@pksorensen  we looked more in depth at this.
can you provide a snip of how your making this call?  We have code currently in place that should auto route this back to the organization web service for Upsert requests


Demo script: 'bike_token_id' is not defined

bigchaindb-driver version: v0.5.0
bigchaindb server version (bigchaindb -v): 2.0.0b1
Python version: 3.5.2
Operating System: Ubuntu

Description
Demo from the driver's landing page fails with an error about 'bike_token_id' being undefined.
What I Did
ran the demo script
nameError: name 'bike_token_id' is not defined

Hey @giddyhup
That's a mistake. I'll fix it later.
For now you have to use bike_token instead.
Or if you want you can make a PR too and fix it :)

@codegeschrei thanks for the quick reaction. I thought so
Yet, it looks like there are additional errors after that:
/bigchaindb_driver/common/transaction.py", line 717, in transfer
    raise TypeError('`asset_id` must be a string')
TypeError: `asset_id` must be a string

I checked, it's a dict.

Hey @giddyhup
I'm sorry, I gave you the wrong information
you need to replace
this line
transfer_asset = {'id': bike_token_id}

with
transfer_asset = {'id': fulfilled_token_tx['id']}


Move away from using ICubicWorld as replacement of World in cubic chunks code

Instead, use World, and cast to ICubicWorld when needed.
It wasn't really a good decision to do it that way. It provides little simplification while making it hard to use methods in World that aren't already there in ICubicWorld, and makes updates harder. Same should go for IColumn

Do you want it to be cast on every use or are stored casted references okay?

It really depends on what and where is used more frequently. If a class nearly exclusively uses ICubicWorld methods - storing ICubicWorld is ok. If a method uses ICubicWorld methods a lot - storing it i  lical variable would be ok.


Create and implement websocket with Player API

This is essentially a proposal for adding functionality to the websocket portion of the project. It is incomplete but shows how it could be done.

Refactors PlayerAPI.java to separate data collection from request handling, allowing the data collection to be reused from the websocket portion
Renames WebsocketHandler to ConsoleWebsocketHandler
Creates WebsocketHandler to handle the v1/ws endpoint, with inputs following mostly the ones in the v1/player api. Outputs are JSON strings including either an error or data, along with type, the type of data being sent.

Here's a Node.js sample client:
import { client } from "websocket";
import { createInterface } from "readline";

const ws = new client();
const rl = createInterface({
    input: process.stdin,
    output: process.stdout,
});

ws.on("connect", (conn) => {
    console.log("Connection established");
    rl.on("line", (line) => conn.send(line));
    conn.on("message", (message) => {
        console.log(JSON.parse(message.utf8Data));
    });
});

ws.on("connectFailed", console.log);

ws.connect("ws://localhost:4567/v1/ws", undefined, undefined, {
    Cookie: "x-servertap-key=change_me",
});

Feedback appreciated, and if this format makes sense I will continue with the other apis.

@7dev7urandom Thank you for your PR! Can you talk a little bit about the use-case for this change? i.e. why a client would want to send getPlayers via the ws and get the same results back as if they had done a GET /players? I'm not against the idea, just curious what the use-cases are.

The primary use case is for a client that wants to keep a connection open rather than reopen a new connection for each query. In my scenario, the client is an android app, where creating new requests is a larger drain on the battery than keeping a single connection open. In addition, using a ws allows the client to subscribe to events instead of continuously querying. For example, the client could subscribe to playerJoin and receive messages when a player joins. Since this use case requires a websocket, it seems unhelpful to require separate http requests when a connection already exists.
That's a couple reasons anyway. Does that make sense at all?

Closing per convo in the Discord: https://discord.com/channels/919982507271802890/920026454207643688/1007085603365593178


Added Prompt Injection Payload For Jailbreak

Added a custom payload to the prompt injection section that utilizes the use of hexadecimal. A custom jailbreak prompt is used, in which a value (in this case, writing a script to create a reverse shell on a windows PC) is also declared in hexadecimal form to the variable "x". This tends to bypass restrictions on writing malicious code and results in the decoding of the hexadecimal and whatever task is requested being completed. The following is an attached screenshot showcasing the  proof of concept:

@swisskyrepo I have also added a badusb section and added 2 badusb scripts utilizing the flipper zero, one acts as a vulnerability scanner and the other harvests all credentials from chrome, Firefox, and edge, saving it to a secondary USB. More information on these scripts is at the top of the respective script commented out via a REM section. I hope you like this :)

If you would like any changes, please let me know. I am pretty proud of this :)

Hello, may I ask if there is something wrong with this?

The Bad USB section is not appropriate for this repository as I'm trying to stick to web vulnerability.
For the LLM part, I don't want a list of jailbreak that can be patched in a later version of the model, but more of strategies and tactics that can be used to bypass the restriction of the prompt.
Finally I prefer to avoid any curse words in the repository :) 😁


Add scrolling speed and rename to bounceSpeed

The current scrollingSpeed is confusingly the bounce speed. And the actual scrolling speed is a fixed value of 150.
I renamed the scrollingSpeed to bounceSpeed, and moved the fixed 150 value to a new prop called scrollingSpeed.
I do realise that this is a BC break, but I think it's for the better. If both my PRs end up getting merged I'll update the types of the other PR to include the new prop in this PR as well.

This change makes sense but I'm keen to avoid breaking changes, especially since we're still using the same variable name for a different purpose. We could also perhaps change the scrollingSpeed name entirely so there's no confusion between old versions and new versions.
It doesn't quite feel like a big enough change to warrant a major version bump.
What are your thoughts on this transition?

New code that keeps both the bounceSpeed and scrollingSpeed props pointing to the bounce speed (so it doesn't break existing functionality). Users can use either oft hem but there will be a console.warn for anyone still using  the scrollingSpeed prop, telling them to move over to the bounceSpeed prop as scrollingSpeed will be deprecated.
After a period of time we push out another release, removing the scrollingSpeed prop entirely. People now have to use bounceSpeed if they want the same functionality as they had. Replace console.warn with a console.error
We introduce a new prop for scrollingSpeed but called something different to avoid confusion with previous versions.

My thought is this change haha.
Semver dictates a major version bump. It's not a huge deal to have a major version bump as long as you have a changelog. If you want it deprecated first that's of course fine, too, but I recommend not doing that because it's such a small change. It makes it easy to update for people using it.

Fair enough, you're the one that done the work on it. 🙂 Happy to merge this and major bump the version as long as scrollingSpeed name isn't reused.
Some quick ideas, marqueeSpeed, rotationSpeed, animationSpeed

I don't like those suggestions. Using marqueeSpeed is too non-specific, same goes for rotation and animation. Which animation? Which marquee? Which rotation?
But this argument goes both ways and made me realise that I chose the wrong name to begin with. I'll rename it to scrollSpeed That way it's consistent. Bounce has a bounceSpeed, and scroll has a scrollSpeed.
All done.


Dependency fails to resolve

The dependency - implementation 'com.dropbox.mobile.store4:4.0.0-SNAPSHOT' failing to resolve. it is included in build.gradle(Module: app)
I need help fixing it.
@yigit  @digitalbuddha

Do you have maven snapshot repo included?

Do you have maven snapshot repo included?

I'm sorry, i don't know how to do that. please could you point me to the steps?

In your root build.gradle:

allprojects {
    repositories {
        // Your other repositories...
        // Add maven snapshot repository
        maven {
            url "https://oss.sonatype.org/content/repositories/snapshots/"
        }
    }
}

One more thing to add:
make sure you add the repository inside allprojects and not buildscript
allprojects {
    repositories {
        maven { url "https://oss.sonatype.org/content/repositories/snapshots" }**// this works**
    }
}

and not:
buildscript {
    repositories {
        maven { url "https://oss.sonatype.org/content/repositories/snapshots" } **// this doesn't work**
    }
}


May I ask why the source vocab and target vocab are separated?

It seems a bit late but I just read your paper and tried to run the demo. Amazing work, thank you for sharing the repo.

I take a look at the model structure and find that encoder and decoder have their own embedding layers of different sizes, meaning the source and target vocabularies are separated. It makes sense if it's the default setting in OpenNMT since the source and target are in two languages in translation tasks. However, here we are dealing with the QG task where both of them are English sentences, similar to the summarization task.
In Abi's original implementation we can see that the embeddings are shared by encoder and decoder (https://github.com/abisee/pointer-generator/blob/master/model.py @ line 209).
I'm wondering how the copy mechanism works exactly here. If it's the case in the paper that the generation vocab for ith question is the union of the question vocabulary and the vocabulary of the ith input context, does that mean the purpose of using pointer network is not to deal with OOVs? because in the "extended generation vocab" for each question, you are adding words that are in the encoder's vocab but are not in the decoder's vocab, then they are not really OOVs for the entire model.
Second, in this setting, whenever there is a copy, that means the word is not in the question vocabulary, so there will be a mismatch between the reference question and generated question. To me, it seems that the loss will increase every time the copy mechanism works.
I'm not sure if I have a misunderstanding about your work. Please correct me if you find any. I will appreciate your reply.

Hi, thanks for your interest in the work!

why source and target has different vocab: our assumption is that questions use a more limited set of vocabulary than the input source context. This constrains the vocabulary that the generator can use to generate a question, which I think makes sense in the context of question generation.

the role of pointer network (or copy mechanism): this is to twofold. First, it deals with OOV words. This is natural because the source and target vocab are different; in our context the target vocab will be much smaller than the source vocab so I think it is natural to copy words from the source. Second, copying words from the source also makes the question more relevant to the input, i.e., it is a way to make sure the question is not irrelevant to the input.

copy mechanism and OOV: When copy happens, it does not necessarily mean a word not in the question vocab appears in the generated question. For a word that appears in both the question sentence and input sentence(s), copy mechanism simply increases the probability of this word. Of course, during training, there will be no OOV words in the generation because the model has full knowledge of what word will appear in the question. For testing, however, copy mechanism does have the capability to generate a word that appears in the input but not in the question sentence.

I hope the above answers some of your questions. Please let me know if I can provide further clarifications. Thanks!

Hi, thanks for your interest in the work!

why source and target has different vocab: our assumption is that questions usually contains a much more limited set of vocabulary than the input source context. This constrains the vocabulary that the generator can use to generate a question, which I think makes sense in the context of question generation.
the role of pointer network (or copy mechanism): this is to twofold. First, it deals with OOV words. This is natural because the source and target vocab are different; in our context the target vocab will be much smaller than the source vocab so I think it is natural to copy words from the source. Second, copying words from the source also makes the question more relevant to the input, i.e., it is a way to make sure the question is not irrelevant to the input.
copy mechanism and OOV: When copy happens, it does not necessarily mean a word not in the question vocab appears in the generated question. For a word that appears in both the question sentence and input sentence(s), copy mechanism simply increases the probability of this word. Of course, during training, there will be no OOV words in the generation because the model has full knowledge of what word will appear in the question. For testing, however, copy mechanism does have the capability to generate a word that appears in the input but not in the question sentence.

I hope the above answers some of your questions. Please let me know if I can provide further clarifications. Thanks!

Hi, thanks for the quick response!
I just dig into the implementation a bit. Could you provide more explanation of "For a word that appears in both the question sentence and input sentence(s), copy mechanism simply increases the probability of this word."? It seems that all words in the ith source are added to the generation vocabulary ( _dynamic_dict() in TextDataset & src_map) for ith question.
I notice the CopyGenerator returns torch.cat([output_prob, copy_prob]). I don't check the sizes of these two probabilities, but I assume that output_prob is of size (question_vocab) and copy_prob is of size (ith_input_vocab). So, will the words that are in both input and question appear twice in this concatenation?

Hi ! I think what happens (on a high level) is that a new vocab is created in the process. Here is a simple example:
input: new york is a city in usa .
question: where is new york located ?
The original vocab for this question: ["where", "is", "new", "york", "located"]
let's say we are generating the third word. Ideally, we want to generate the word "new".
Without copy mechanism, the following could be the softmax output (bad example but illustrative):
["where": 0.2,
"is":0.2,
"new":0.2,
"york":0.2,
"located":0.2]
With copy mechanism, we do three things here:

get the attention weights for the words in the input:
["new": 0.3,
"york": 0.1,
"is": 0.1,
"a": 0.1,
"city": 0.1,
"in": 0.1,
"usa": 0.2]
compute the copy probability, say it is 0.8, which means we prefer to copy a word from the input to the generated question
add the two lists together, weighted by the copy probability. So for example, the word "where" gets a probability of 0.2 * (1-0.8) + 0 * 0.8 = 0.04 (weighted by the not-copy probability which is 1-0.8=0.2, and add 00.2 because it does not appear in the input). Similarly, the word "new" gets a probability of 0.2(1-0.8) + 0.3*0.8 = 0.28. The other word probabilities can be computed similarly.

Carrying the above process for all words in the input and question, you'll see that the word "new" has the highest probability among the augmented vocabulary (vocab of question + vocab of input) and thus we select this word.
It has been some time since I update the repo but I can take a look at the implementation and provide more information on how this is actually implemented. Nevertheless, I hope the above example gives you a high level idea on how the copy mechanism is computing the word probabilities during generation.

Hi ! I think what happens (on a high level) is that a new vocab is created in the process. Here is a simple example:
input: new york is a city in usa .
question: where is new york located ?
The original vocab for this question: ["where", "is", "new", "york", "located"]
let's say we are generating the third word. Ideally, we want to generate the word "new".
Without copy mechanism, the following could be the softmax output (bad example but illustrative):
["where": 0.2,
"is":0.2,
"new":0.2,
"york":0.2,
"located":0.2]
With copy mechanism, we do three things here:

get the attention weights for the words in the input:
["new": 0.3,
"york": 0.1,
"is": 0.1,
"a": 0.1,
"city": 0.1,
"in": 0.1,
"usa": 0.2]
compute the copy probability, say it is 0.8, which means we prefer to copy a word from the input to the generated question
add the two lists together, weighted by the copy probability. So for example, the word "where" gets a probability of 0.2 * (1-0.8) + 0 * 0.8 = 0.04 (weighted by the not-copy probability which is 1-0.8=0.2, and add 0 * 0.2 because it does not appear in the input). Similarly, the word "new" gets a probability of 0.2 * (1-0.8) + 0.3 * 0.8 = 0.28. The other word probabilities can be computed similarly.

Carrying the above process for all words in the input and question, you'll see that the word "new" has the highest probability among the augmented vocabulary (vocab of question + vocab of input) and thus we select this word.
It has been some time since I update the repo but I can take a look at the implementation and provide more information on how this is actually implemented. Nevertheless, I hope the above example gives you a high level idea on how the copy mechanism is computing the word probabilities during generation.

Thanks for the enlightening example！
Yes，I agree that on the high level these two probabilities should be “merged” rather than “stitched together” with potential vocab overlap.  I guess I just need to spend more time  figuring out how exactly it is implemented.
I really appreciate your detailed replies. Thank you again for that. :)

Glad you find the above example useful ! let me know if I can help clarify other things.


cache test fix

fix for https://github.com/nodezoo/nodezoo-npm/issues/91

Coverage remained the same at 76.667% when pulling 3d3b4de8bbfa622fa73df1472fdbcdb9ac5e9a30 on testfix into b2f6e863ec85c62241c44f582ad8de7fec52b37b on master.

👍

🐑 it


dockerTools.buildLayeredImage creates skewed sized layers

Issue description
Recently dockerTools.buildLayeredImage ( https://grahamc.com/blog/nix-and-layered-docker-images ) functionality was merged into master.
It creates docker images with layers of a size with a distribution that is very skewed. See below for more information.
Steps to reproduce
let
  old_pkgs = import <nixpkgs> {};
  pkgs  = <snip (assumes that it points at master> ;
 inherit (pkgs) stdenv;

in pkgs.dockerTools.buildLayeredImage {
   contents =  [pkgs.bashInteractive old_pkgs.curl old_pkgs.jq old_pkgs.emacs old_pkgs.git
                old_pkgs.cacert old_pkgs.openssh old_pkgs.gnugrep
                old_pkgs.coreutils
                old_pkgs.which
                old_pkgs.nix
                old_pkgs.gnused old_pkgs.file old_pkgs.gawk];

  name = "foo/bar";
  tag = "latest";
  config.Cmd = [ "/bin/bash" ];
  maxLayers = 25;
}

The layers looks like:
b6cdec945d1b: Loading layer [==================================================>]    477MB/477MB
86e60f3ed08f: Loading layer [==================================================>]  174.1kB/174.1kB
79d24adb2c31: Loading layer [==================================================>]  348.2kB/348.2kB
9354bd19facd: Loading layer [==================================================>]  12.19MB/12.19MB
611dae553d72: Loading layer [==================================================>]  1.178MB/1.178MB
cb916eb69cd1: Loading layer [==================================================>]  71.68kB/71.68kB
ee38b4411beb: Loading layer [==================================================>]    1.7MB/1.7MB
05bad8e60f92: Loading layer [==================================================>]    809kB/809kB
d30df3eb574c: Loading layer [==================================================>]    768kB/768kB
bb3ba7943090: Loading layer [==================================================>]  40.96kB/40.96kB
e7e6d2f94b2e: Loading layer [==================================================>]  5.612MB/5.612MB
15048416f85e: Loading layer [==================================================>]  1.526MB/1.526MB
b00ca1c01fd7: Loading layer [==================================================>]    512kB/512kB
0518754facbc: Loading layer [==================================================>]  92.16kB/92.16kB
13859dfd2c8b: Loading layer [==================================================>]  266.2kB/266.2kB
e696a9248983: Loading layer [==================================================>]    256kB/256kB
560c99588b76: Loading layer [==================================================>]  81.92kB/81.92kB
dcf8aeee86d0: Loading layer [==================================================>]   51.2kB/51.2kB
9b58d0959ca7: Loading layer [==================================================>]  2.898MB/2.898MB
9caed6f91ab9: Loading layer [==================================================>]  112.6kB/112.6kB
e7379e4a6dcb: Loading layer [==================================================>]  1.352MB/1.352MB
4a5ae92f18dc: Loading layer [==================================================>]  40.96kB/40.96kB
d91816088e26: Loading layer [==================================================>]  40.96kB/40.96kB
7ddbb507422c: Loading layer [==================================================>]  21.79MB/21.79MB

As you can see the size is unreasonably distributed (a 477MB layer with a bunch of tiny ones).
My question is: is this the intended behavior? If not, can we have tests that make sure the distribution of the size of the layers follows a particular (statistical) distribution.
Relevant people
@graham-at-target
Technical details

system: "x86_64-linux"
host os: Linux 4.14.74, NixOS, 18.09.git.08005ed (Jellyfish)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.1.1
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

This is the expected behavior. The layers, big or small, are prioritized by likelihood they are going to be cache hits.  I recommend changing your maxlayers to something closer to 100, and will be updating the documentation about this.

Thank you.
What is the equivalent of RUN useradd -r -u 1001 -g appuser appuser when using such images? I would like the root user to exist in /etc/passwd such that I can set config.User="root";

You can add a passwd file, like this:
        userSetup = self.runCommand "user-setup" { } ''
          mkdir -p $out/etc/

          echo "${toString uid}:x:${toString uid}:${toString gid}:anonymous uid:${homeDir}:/bin/false" >> $out/etc/passwd
        '';

and then add userSetup to the contents list.

That works great. Only issue I seem to have now is that /usr/bin/env is not part of the image. /sbin/env is.

You probably want to add
echo "root:x:0:" >> $out/etc/group

too

You might be able to make this work via:
env-shim = pkgs.runCommand "env-shim" ''
  mkdir -p $out/usr/bin
  ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env
'';

and putting env-shim in contents

Thanks, you missed a {}, but otherwise this helps a lot.

Not sure if I am pushing it if I also ask this question here:
When trying to build nixops inside a docker container (nix build -f release.nix build.x86_64-linux):
warning: the group 'nixbld' specified in 'build-users-group' does not exist

pushing it

A little bit :)
You might want to check out https://hub.docker.com/r/lnl7/nix/ / https://github.com/lnl7/nix-docker which is for running nix inside the container.
The little bit I do know, though, is you probably need to run nix not as root so it doesn't try to use build users or features which require the ability to fiddle with the kernel.

I was already using that image, but not running as root might be an idea.

@graham-at-target Is it possible to build on top of https://github.com/LnL7/nix-docker while still using your approach too?

Yo, I recommend closing this issue and opening new ones for new issues. :)

I've got a WIP: that allows one to hint at how many layers to put where: https://github.com/tomberek/nixpkgs/blob/tom/docker_hints/pkgs/build-support/docker/default.nix#L299
Usage would be something like:
let
repeat = n : a : accum : if n == 0 then accum else repeat (n - 1) a (accum ++ [a]);
r = n : a : repeat n a [];
in
buildLayeredImage {
hints = (r 5 10 ) ++ ( r  100 1)
...
}

... meaning to create 5 layers of 10 paths each, then up to the next 100 layers having 1 path each. The order is the same as how the current system works. This just groups them differently.

I propose that the buildLayeredImage tool have one or two extra parameters that control grouping of several packages at once per layer. Thus, in a 178-package image, instead of having 98 layers each with one package, then one huge layer with 79 packages, we'd have 80 layers with 2 packages each, then 18 layers with one package each. Ideally the "last / top" packages explicitly requested by the programmer, that are presumably most subject to change, would be among the single-package-per-layer ones, or fewer-packages-per-layer ones.

See the discussion here between Fare and gchristensen: https://logs.nix.samueldr.com/nixos/2020-07-24#3773968
Another feature that could help alleviate the pain would be if buildLayeredImage accepted a buildImage as initial input and/or let me specify some of the exact layers I want, with e.g. "just this package" or "all these packages", then filled the blanks.

@fare : proof of concept https://github.com/tomberek/nixpkgs/blob/tom/docker_hints/pkgs/build-support/docker/default.nix#L299 that i've successfully used in the past to give me control over the grouping of the layers. I called it "hints" and normally would do sets of 20 packages. I found it helpful to also give very large packages their own layer to help even out the sizes (improving parallel download, processing speed).
This is probably bit rot now that dockerTools has been updated, but the concept may survive.
For discussion:

layerGrouping = [ 10 10 10 10 ]; Make up to 4 layers with 10 packages each, then individuals packages.
layerGroupMaxMb = 1024: Group layers together unless it would exceed 1Gb.

If both options are given, both would apply? So you can end up with something like:
[ 8 2     10     10     3 1 6     1 1 1 1 1 1 1 1 1 1 1 1 1 1]

If the first grew to be above 1G and last set of 10 had a single package that was almost 1G.

For a new function, another possible approach is to let the user determine the grouping using an algorithm that takes

a starting set (the entire closure)
splitPaths: a small set of user input store paths that is known to vary between successive builds. This will probably reference the main store path.

Emphasis is on successive, because this does not try to deduplicate much between dissimilar image with some matching store paths.
It could then fold over splitPaths: fold f [wholeClosure] splitPaths, iterating on the layers starting with wholeClosure.
f will split each layer into potentially three layers.
First, we check whether splitPath occurs in the layer. If not, we return the layer as is.
Then, we compute the intersection between the splitPath closure and the closure of the layer after removing splitPath itself. This represents the commonalities.
With this intersection, we can return the layers

the intersection, containing commonalities
splitPath closure the commonalities removed
the remainder of the layer, with splitPath and the commonalities removed

This looks like it has O(3^#splitPaths) worst case complexity, but fear not: the complexity is also bounded by O(#items(wholeClosure) * #splitPaths) because no store path is duplicated. A clever implementation could bring the average down further, but I don't think we have a performance issue here.
Comparing to buildLayeredImage
Because this solution operates entirely on set operations of closures specified by humans, instead of low-level store path based heuristics, it will only split layers across fairly high-level boundaries. Another way to see this is if you'd refactor a derivation by splitting out a dependency, you won't cause new layer, unless it happens to introduce a new commonality with another splitPath or the whole closure, which is rare.
This approach will not guarantee reuse between distinct images (ie not versions of the same image), although it may still cause some happy collisions.
Comparing to buildImage
buildImage itself creates a single layer, so we should be comparing to a chain of buildImage, which behaves like a Dockerfile.
Besides the downside of having to copy files over and over when low layers change, it can only find commonalities with entire previous layers. It's not always feasible to find these commonalities, unless you want to explicitly specify layers for low-level store paths like for your libc, or some set of libraries that should change as your image evolves but will most likely bitrot.
You lose a little bit of control using this solution, but it seems unlikely that it will cause the number of layers to increase by too much. If it does, you can reduce the number of splitPaths.
Further optimization: split "configuration"
This is somewhat independent of the algorithm I'm proposing and can probably be applied to the other approaches as well.
One problem that the algorithm may suffer from is having too few splitPaths, causing large "remainder" layers which change too often. Although the prior can only be solved by deferring to another clever algorithm like @tomberek's or the existing one; we can improve the latter by detecting which files are "configuration". Small files near the top of the closure are typically configuration files like /etc contents, startup scripts and such that don't benefit from multiple layers, yet change more often than the big "remainder" layer. By taking those out of that layer, we reduce number of versions of that big layer.
This is similar to the "customization layer" but also applies to store paths and it's automatic. Maybe it's not similar, except for the name.
Erroneous splitPaths
Despite their small number, splitPaths still have to be maintained. Depending on how the expression is design, by accident, one of the splitPaths could start referring to a path that's not in the actual image closure. This should be reported as an error, both to avoid accidental inclusion of irrelevant paths, and to provide an opportunity to fix the problem.
Call to action
This idea has been sitting in the back of my mind for a while, but I don't suffer from this issue. So as much as I'd like to see an implementation, and I hate to say it, I can't justify implementing it as things stand.
So if anyone is looking for an interesting project, or wants to solve the problem really bad, please give it a shot, ask me if anything is unclear and I'd be happy to review.

For a new function, another possible approach is to let the user determine the grouping using an algorithm that takes

a starting set (the entire closure)
splitPaths: a small set of user input store paths that is known to vary between successive builds. This will probably reference the main store path.

Emphasis is on successive, because this does not try to deduplicate much between dissimilar image with some matching store paths.
It could then fold over splitPaths: fold f [wholeClosure] splitPaths, iterating on the layers starting with wholeClosure.
f will split each layer into potentially three layers.
First, we check whether splitPath occurs in the layer. If not, we return the layer as is.
Then, we compute the intersection between the splitPath closure and the closure of the layer after removing splitPath itself. This represents the commonalities.
With this intersection, we can return the layers

the intersection, containing commonalities
splitPath closure the commonalities removed
the remainder of the layer, with splitPath and the commonalities removed

This looks like it has O(3^#splitPaths) worst case complexity, but fear not: the complexity is also bounded by O(#items(wholeClosure) * #splitPaths) because no store path is duplicated. A clever implementation could bring the average down further, but I don't think we have a performance issue here.
Comparing to buildLayeredImage
Because this solution operates entirely on set operations of closures specified by humans, instead of low-level store path based heuristics, it will only split layers across fairly high-level boundaries. Another way to see this is if you'd refactor a derivation by splitting out a dependency, you won't cause new layer, unless it happens to introduce a new commonality with another splitPath or the whole closure, which is rare.
This approach will not guarantee reuse between distinct images (ie not versions of the same image), although it may still cause some happy collisions.
Comparing to buildImage
buildImage itself creates a single layer, so we should be comparing to a chain of buildImage, which behaves like a Dockerfile.
Besides the downside of having to copy files over and over when low layers change, it can only find commonalities with entire previous layers. It's not always feasible to find these commonalities, unless you want to explicitly specify layers for low-level store paths like for your libc, or some set of libraries that should change as your image evolves but will most likely bitrot.
You lose a little bit of control using this solution, but it seems unlikely that it will cause the number of layers to increase by too much. If it does, you can reduce the number of splitPaths.
Further optimization: split "configuration"
This is somewhat independent of the algorithm I'm proposing and can probably be applied to the other approaches as well.
One problem that the algorithm may suffer from is having too few splitPaths, causing large "remainder" layers which change too often. Although the prior can only be solved by deferring to another clever algorithm like @tomberek's or the existing one; we can improve the latter by detecting which files are "configuration". Small files near the top of the closure are typically configuration files like /etc contents, startup scripts and such that don't benefit from multiple layers, yet change more often than the big "remainder" layer. By taking those out of that layer, we reduce number of versions of that big layer.
This is similar to the "customization layer" but also applies to store paths and it's automatic. Maybe it's not similar, except for the name.
Erroneous splitPaths
Despite their small number, splitPaths still have to be maintained. Depending on how the expression is design, by accident, one of the splitPaths could start referring to a path that's not in the actual image closure. This should be reported as an error, both to avoid accidental inclusion of irrelevant paths, and to provide an opportunity to fix the problem.
Call to action
This idea has been sitting in the back of my mind for a while, but I don't suffer from this issue. So as much as I'd like to see an implementation, and I hate to say it, I can't justify implementing it as things stand.
So if anyone is looking for an interesting project, or wants to solve the problem really bad, please give it a shot, ask me if anything is unclear and I'd be happy to review.

Apologies @adrian-gierakowski if you've been hacking away on this locally, but we've actually implemented a solution to this issue a week ago (without seeing this issue, somehow).
We had the same problem as the OP: an enormous final layer with essentially all the dependencies, and a tiny change to the application has to re-ship that big layer.  We see three potential heuristics when the number of packages exceeds the layers:

Prefer the lowest dependencies (current behavior).  This maximizes sharing of packages like glibc across different applications.
Prefer the highest dependencies.  This minimizes the diffs to upload/download between iterative releases of the same application.
Use some domain specific knowledge to group more intelligently.  (*)

We have implemented (1) and (2) as selectable out-of-the-box algorithms in buildLayeredImage, and provided a hook for users to implement (3) relatively easily.  Let us know what you think!
In our case, switching from (1) to (2) reduced cycle iteration times in kubernetes by more than a full order of magnitude.

(*) Another thing we tried, mentioned above, was to split evenly. For example, with 500 pkgs and 100 layers, put 5 pkgs in each layer, but upon further consideration this does not have the cross-app sharing properties of (1), and is just less efficient than (2) for one app, so we scrapped it.

@bhpdt no worries, I only spent a couple of hours last week on this. Anyway the work I'm about to do is mostly orthogonal to what you've done (although it's trying to address similar problem).

In my team we've had a similar issue and I had an idea for a solution to a different but closely related problem: when you know the set of images that need to be built and therefore have a full view of all the packages that are going to make it into Docker images.
With that assumption, it's possible to (attempt to) search for a globally optimal layering that solves many issues at once, rather than merely hoping the layering scheme that was used for one image is going to be of help in another. Maybe this can be yet another layering strategy. I'm not sure it would fit into @adrian-gierakowski's PR however, because it likely could not be expressed within the DSL. It's also possible this is overly complex and that @roberth's and @bhpdt's very pragmatic ideas yield better result.
Since my team is slowly moving away from having many Docker images I have not dedicated time to writing code for this and I can only contribute an idea at this point:
The fundamental idea is to create layers such that each layer comprises packages required by the same images and only by those images. So there's this idea of minimalism, and of a "canonical" layering. Then, if the resulting layering breaks the maximum number of layers allowed for a given image, it's possible to perform merge operations (described further below) on those layers.
Formally, suppose you want to build a known, finite set I of images. Let P be the union of the closures of the contents of the images in I (i.e. the set of all packages to be distributed among layers), and Gp the dependency graph of the packages in P. Let R be the relation over P such that pRq if and only if the set of images that require package p is equal to the set of images that require package q. R is an equivalence relation. Then, the candidate layers are the nodes of the (directed) quotient graph Gp/R. The layers to be used to build a final image are those whose contents are required by that image.
Before going any further, here's a practical example: suppose you want to make images 1, 2 and 3 with the following contents:

1: [a, b]
2: [c]
3: [f]

where the packages in P have the following dependency graph (with arrows pointing down so dependents are on top and dependencies are towards the bottom):
a   b   c
 \ / \ / \
  d   e   f
 / \ / \ /
g   h   i

There are several ways to compute the quotient graph. An intuitive but sub-optimal one is to first compute the mapping of packages to sets of images that require them:

Package
Images that need it

a
{1}

b
{1}

c
{2}

d
{1}

e
{1, 2}

f
{2, 3}

g
{1}

h
{1, 2}

i
{1, 2, 3}

and then to invert that mapping and regroup the packages corresponding to a given set of images:

Set of images
Packages that are required only by those images

{1}
[a, b, d, g]

{2}
[c]

{1, 2}
[e, h]

{2, 3}
[f]

{1, 2, 3}
[i]

The sets of packages in that second column are the (contents of the) candidate layers. The quotient graph that they form can be built from this table because it is induced by the subset relation over the sets of images in the first column (more on that later).
Here's the quotient graph, with each row of the above table reproduced as a node (arrows still pointing down, this graph is also directed):
{1} [a, b, d, g]       {2} [c]
         \              /  \
          \            /    \
           \          /      \
          {1, 2} [e, h]     {2, 3} [f]
                 \          /
                  \        /
                   \      /
               {1, 2, 3} [i]

Say you want to build image 2. You would take the layers corresponding to the "image sets" containing 2: {2} [c], {1, 2} [e, h], {2, 3} [f] and {1, 2, 3} [i].
Note: the algorithm we presented above to compute the quotient graph is sub-optimal. The optimal algorithm (I mean for this particular problem, not to compute quotient graphs in general) is to traverse the full dependency graph Gp in topological order starting at the roots defined by the image contents and, as you go, to mark the packages with the images that need them.
We can see that as we go down this graph of layers, the layers become required by more images. A node's image set is always a strict superset of any of its parents' image set.
Indeed, let (S, Q) be a node of the quotient graph of layers where S is the layer's image set and Q is the set of packages in the layer, and let (S', Q') be a parent node of (S, Q). By definition of the original graph Gp as a dependency graph, at least one package in Q is a dependency of some package in Q'. Since the packages in Q' are required by the images in S', it follows that those packages in Q are transitively required by the images in S'. Since Q is an equivalence class for relation R (reminder: pRq if and only if packages p and q have the exact same "requiring images"), it follows that all packages in Q are required by the images in S'. Since S is the exact set of images that require the packages in Q, and S ≠ S' (otherwise they wouldn't form two distinct nodes), we conclude that S' ⊊ S.
Once we have the layers that form an image, the order in which they're stacked to produce the image is arbitrary but it seems best if it follows a topological ordering of the graph of layers, i.e. the deepest layer in the Docker image is needed by the most images so it should also be the lowest layer in the graph of layers, and vice versa for the top layer which is the most image-specific.
It is possible that the number of layers generated by this method exceeds some allowed maximum. Maybe the graph of layers produces a set of layers small enough for one image but too large for another. In that case, we need to merge some layers that are required by that image that needs fewer layers. Those merges will make our solution less optimal so as to satisfy the constraint. There are two possible approaches: either we push packages down from image- and application-specific layers into layers where they're not needed by all images, or we bubble packages from lower, shared layers, up towards the more image- and application-specific layers, but they will get duplicated across layers. I think it's best not to add to an image packages that were not requested by the user. So that leaves the second option:
Consider a node (S,Q) in the graph of layers and call (s1, q1), ..., (sn, ..., qn) its parent nodes. By definition of the graph of layers, all the packages in Q are needed by the images in the s_i, associated to its parent nodes (the nodes above). So we can safely remove that layer from the graph and merge its package set Q "upwards" into the package sets qi of the parent nodes. In fact, it is sufficient to merge Q into any subset of its parents { (s'1, q'1), ..., (s'm, q'm) } such that S = s'1 U ... U s'm, because what matters is that no image loses the packages it needs. (I'm not sure what happens if the s'i overlap though. Two layers required by the image would have the same nix store paths, not sure what happens then.)
Note: it's possible to prove that if an upwards merge is at all possible, then you need look no further than into parent nodes to find a viable set of merge "targets", i.e. that you don't have to merge into farther ancestor nodes to guarantee that all images will get the packages they need, but that proof is easy and left to the reader.
In the example, we can merge {1, 2} [e, h] into {1} [a, b, d, g] and {2} [c], which gives us an updated graph:
{1} [a, b, d, g, e, h]   {2} [c, e, h]
             \              |   \
              \             |    \
               \            |     \
                \           |   {2, 3} [f]
                 \          |     /
                  \         |    /
                   \        |   /
                   {1, 2, 3} [i]

Images 1 and 2 still have all the packages they need and nothing has changed for image 3.

Here's the result of merging {1, 2, 3} [i] into {1} [a, b, d, g] and {2, 3} [f]:
{1} [a, b, d, g, i]    {2} [c]
         \              /  \
          \            /    \
           \          /      \
          {1, 2} [e, h]     {2, 3} [f, i]

Because we merged into a grandparent, now package e which depends on i appears above it, but the layers will still work to produce the desired images.
At this point one can wonder when such opportunities for upwards merges are present in a graph of layers, and the answer is "in most cases that are of interest to us". Let's imagine that for a given layer node (S, Q), there are no (s1, q1), ..., (sn, qn) among its parents such that S = s1 U ... U sn. In other words, some images in S are nowhere to be found in the si.
In our example, trivially, a layer node such as {2} [c, e, h] satisfies this condition since it is specific to one image. It cannot be merged upwards since there's no upper layer to merge it into. But this condition of "unmergeability" is also satisfied by {2, 3} [f] because 3 does not occur in its parent nodes' image sets. This occurs because there is no package required by image 3 that is not also required by another image.
I think it is rare enough for an image to have no package specific to it that we can assume upwards merges will almost always be possible. But this can happen if you make a final, tagged image containing, say, Chrome and Firefox, and one containing only Firefox.
With that being said, how do you choose among the many possible upwards merges? A (naive?) measure of how good an upwards merge is is how many extra bytes it introduces in the layers as a whole. If packages p1, ..., pn in layer Q get duplicated across d parent layers and then Q gets removed then the net weight gain of the operation is (d - 1) * weight(Q) = (d - 1) * (weight(p1) + ... + weight(pn)).
Minimizing the d-factor for a given layer node (S, Q) is easy: just search among (S, Q)'s parent nodes (s1, q1), ..., (sm, qm). Since their image sets are the largest strict subsets of S to be found in the graph, the smallest union of si yielding S will most easily come from them.
So to search for the best candidate upwards merges, walk the graph of layers topologically (actually, only the subgraph that concerns images that have too many layers) and at each node, if it can be used for an upwards merge, calculate its d factor and its potential net weight gain.
Let's assume that a set Q of packages has been merged into q1, ..., qm. Then, each qi's weight has increased by weight(Q) so the net weight gain of upwards-merging qi itself has increased by (di - 1) * weight(Q) where di is the d-factor of (si, qi).
As you traverse the graph and compute the candidate upwards merges, store them in a min-priority queue. Then, get popping and merging until you've reached the desired number of layers. As you remove elements from the queue and perform the corresponding merges, also update the priorities of the candidate merges of the fattened parent qi layers (some kind of "decrease key" operation on the priority queue, or even "k-decrease key" maybe?).
This weight gain minimization approach may not be the best though. Maybe it's best to merge more often closer the bottom of the graph because those packages are shared the most and change the least, whereas up top any large layer created by merges would have to be rebuilt if a package changes which happens more often there, close to applications. I suspect the lowest-level packages tend to be small though, so these two goals could converge.
Another interesting question to ask is how the layering produced by this technique evolves as images are added or removed, and as image contents and packages evolve. We would like to be able to simply re-run the algorithm and be able to reuse most of the previously-built layers, especially if they're deep or big. And then there's the question of how layer merges interact with those evolutions, but hopefully from one evolution to the next the layers selected for merging and therefore the resulting merged layers would remain the same. I haven't gone very far thinking about those problems.
A new image x could be added. Any packages that it introduces that are not already needed by other images end up in a layer specific to x. Those packages could depend on packages already somewhere in the graph, so edges would be introduced and layers might get split as some of their packages would now belong in a different R-equivalence class. The new image x will reuse layers that already contain only packages it needs, but if a package that x requires is only ever present in layers together with packages x doesn't require, then at least one of those layers needs to be split.
An image x could be removed. The opposite of the above would happen. The layer specific to x, if it exists, would disappear. If there exists a pair of layers with image sets S and S U {x} (where S doesn't contain x) then removing x would allow us to merge those layers.
Then there are the cases of adding packages to an image that are already in P (therefore somwhere in the graph), adding entirely new packages (which may introduce new edges in Gp), updating the dependencies of a package, removing a package, etc...
For completeness' sake, updating a package in any way, even when its dependencies don't change, causes the layers it belongs to and their ancestors in the graph to need a rebuild.
That's all from me, please tell me what you think, or if I've made errors or omitted something. Thanks for reading!


Navbar too broad on smaller screen

The logo should be samller accordingly and height lesser.
Kindly assign this issue to me under hactoberfest, gssoc-ext and relevant tags.

ok...modify the changes accordingly and check for responsiveness

It works perfectly, the flex option is there for this reason. Closing this issue now.


[Feature Request] New Project Item :  User Credentials

Can you please add a new Project Item type just to handle personal credentials for different services used by a project? These are details we usually store on a random text file or use the browser's default password manager.
Use cases

Credentials for services like AWS, Heroku, etc.
Git access keys (HTTPS)

yes, I have needed this myself, and for that I'm currently using notes. we can discuss whether there could be a better approach.
What you suggest sounds like server->extra user. I felt it was a little inflexible for this purpose here -- but I could be wrong. What I'm doing is using project notes with the notes password support.
Within notes if you click the lock icon in the toolbar on top of the text, you can insert passwords: [passmy password]. Once passwords are entered in notes like that, they are not revealed when displaying the notes, and when clicking on them, you can copy them to the clipboard or reveal them.
You can also paste keys in notes, and if you paste them in code blocks, you get a button to copy them to the clipboard.
Note that we're talking about project-wide settings. For aws/heroku and even possibly git I might add those under a server entry (depending). I've hit this case for VPN credentials, where I found the ability to enter extra information (eg other VPN settings) useful.
If you feel your case doesnt fit for notes, can you explain why another approach would be better than notes and why the server credentials or server extra users are not a fit?

I've just discovered the project by chance when looking upon rust-based projects. So I wasn't aware of the neat little pass feature. I'm certain the password notes can be used as-is for personal use.
However, when I'm exporting my data and sharing it with the rest of the team or friends, it will be much easier if I do not have to worry about forgetting to remove my personal credentials.
It can be used as a standard text field with global values or unique to each env.  Additionally, we can use these fields as reference/FK to be used in a Project note like footnotes/bibligraphy are used in markdown. This will can be used to signal to the app that pass fields are to be ignored when exporting while maintaining referential integrity. The new users can just insert their passwords on the empty fields.
One way to go about it is to add a Add new password on the Add new Project Item tab which open a dialog for a single credential with env options

This is mostly because I use Notes more like documentation for the project with only minor adaptations between the env based versions.

Yes, I was interested in linking to items from notes, but it gets complicated. They couldn't be inline in the note, but would have to be some sort of note meta-data so that the SQLite foreign keys really check that the link target wasn't removed or something of the sort. But yes, you made me think that it could be doable as notes meta-data, which could be referenced from within the note. It gets a little complicated though.
In terms of hiding credentials on export, it's not done. Currently the export will export all credentials. But if we add an option to mask credentials in password-specific fields, we can also mask them in "password" fields in notes.
I'll think about a new project-level "password"/"credentials" item type. It feels like a narrow need, between server credentials & project notes. But maybe I do still add it.


Find The Minimum Total Travel Expense

Description
N of us are going on a trip, by train or taxi.
The train will cost each of us AA yen (the currency of Japan).
The taxi will cost us a total of BB yen.
How much is our minimum total travel expense?
The user will input in the format: N A B.
Print the minimum total travel expense.
For example:  if the inputs are: 4 2 9
then the train will cost us 4×2=8 yen, and the taxi will cost us 9 yen, so the minimum total travel expense is 8 yen.
Domain
CPP
Type of Contribution
No response
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of LearnCPP.

/assign

/assign


Duplicates of usings in metadata

Hi, I just installed SharpPad Extension for the first time on VS Code. Went by reference on type "int" (F12) as was written in a book ("C# 8.0 and .NET Core 3.0 – Modern Cross-Platform Development Fourth Edition published by Packt"). And got using duplicates:

Is it the problem of extension or just on my side (OS/VS Code/etc.)?
Project type: Console App (.NET Core v3.0)
Info about my VS Code:
VS Code Version: 1.40.2 (user setup)
Commit: f359dd69833dd8800b54d458f6d37ab7c78df520
Date: 2019-11-25T14:54:45.096Z
Electron: 6.1.5
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Windows_NT x64 10.0.18363

Info about my Windows:
Edition: Windows 10 Pro
Version: 1909
Installed on: 30-Sep-19
OS Build: 18363.476

I'm not the maintainer but is SharpPad the only extension you have installed, or do you have the C# Extension installed as well?
If the latter, have you tried uninstalling the SharpPad extension and seeing if you can reproduce the issue?

Yeah, I have C# Extension. I tried to disable/enable and reinstall some extensions. and it was fixed. I think the problem was in incorrect plugins load/install order or some type of it.


Mark CVE-2016-6485 for magento2 as fixed in 2.2.5

Ref: https://github.com/FriendsOfPHP/security-advisories/pull/161#issuecomment-402142322

Replaced by #306


Android build crashes when using C# threads

Godot version: Godot Engine v3.2.3.stable.mono.official
OS/device including version: Linux Mint 19.3
C# version: .NET Core 3.1
Issue description: Any scene crashes if it contains a C# script that uses the Threads library.
Steps to reproduce:
Make a script that contains an at least await and an async function
Export for Android
Run the .apk on any debugging platform
It crashes immediatly (even before the await or the async are reached)
You can find here a minimal Godot project

Can you still reproduce this in 3.5.2 and 3.6-beta2 or later?
If so, could you provide a new reproduction project? The previous link expired. You can upload a zip directly to GitHub with a drag and drop.

Closing due to lack of response if you provide the requested MRP and details this can be reopened


Create club page for IEEE

Hey @AakarshShreshth I want to contribute into this issue can you please assign me to this issue #19 under hactoberfest2023 guidelines...Thank You!!!

Hey @ietebitmesra I am working on this issue can you please assign me to this issue #19 under hacktoberfest2023 guidelines.....Thank You!!!

Please assign me this issue