Generated SSH keypair not working

Struggled a little bit setting FrameOS up.
For reference, I have a Pi Zero set up with the ~5 inch Pimoroni inky screen. I've deployed the controller on a separate server and trying to set the connection up.
I do not have passwords enabled for SSH, so I was happy to see that SSH keypairs are supported in settings. Generating a keypair and adding the public key to .ssh/authorized_keys on the Pi Zero fails to connect because the key doesn't seem to be a valid RSA key
2024-01-03 17:22:09 Deploying frame <name> with build id mkjdapkjexeb
2024-01-03 17:22:09 Connecting via SSH to<EMAIL_ADDRESS>(keypair)
2024-01-03 17:22:09 not a valid RSA private key file

I tried creating my own SSH keys (using ed25519) and saving that directly in settings, which gives me a different error
2024-01-03 17:21:02 Deploying frame <name> with build id tqfnsxmojkfk
2024-01-03 17:21:02 Connecting via SSH to<EMAIL_ADDRESS>(keypair)
2024-01-03 17:21:02 unpack requires a buffer of 4 bytes

I then finally created a RSA keypair with ssh-keygen -t rsa and was able to get that working. If the generated SSH keypairs work on your end, then I'm happy to provide any further information for troubleshooting.

Hey @ikornaselur , thanks for trying out FrameOS and for reporting back!
Indeed, currently only RSA keys are supported. It doesn't seem too hard to add ed25519 though. I'll see what can't be done, but can't promise exactly when I'll get to it.

No worries! I think it's fair that only RSA keys are supported. But the main issue is still that the generated keypair from the UI, with this button:

produces the "not a valid RSA private key file" error, so seemingly the key being generated by FrameOS is invalid. I was only able to get a manually generated keypair working.
Does a generated keypair from the UI work on your end? I'm happy to investigate further on my end and see if I can identify the issue.
Oh, and for further reference, I am deploying the controller with docker and the :latest tag

Same here. Also docker with latest tag.

This should work now!
I tried changing the default key from RSA to ed25519, but ran into some weird issue. The generated key seemed much too short, even for ed25519, and ChatGPT seems to think it's better to defer to ssh-keygen than use python with ed25519. So for another day.

Strange, I thought this was fixed by #27  but it recurred with me again, running ac2e102795db3b6b07704681b36308d18d8195b5.
Working around for now since I want to (1) test #30  again and (2) try out the dithering on a red/black/white InkyWhat.  (I had code lying around locally that tried to dither with Pillow but never quite worked)


revert common-utils to lockfileVersion 1

Not sure if we need to publish another pre-release version with the proper lockfile or if the one with lockfileVersion 2 is ok...

@curtisman AFAIK this lockfile is in the prerelease version  we're depending on now.  Do we need to roll that version to restore the right lockfile version in there or is it ok?

package.lock files are not published and has no impact on package that get installed.


x/sys/windows/svc/mgr: Allow clients to request read-only access to the Service Control Manager and to individual services

Mgr.Connect always requests SC_MANAGER_ALL_ACCESS rights, and Mgr.OpenService always requests SERVICE_ALL_ACCESS rights. Since both of those methods are requesting write access, they require the current user to run at a higher permission level than may otherwise be required by the caller.
It would be nice if additional methods were exposed that would allow either for finer-grained control over the access requested, or at least a variant that requests read-only (ie, GENERIC_READ) access.
Of course, this is easily worked around by directly calling the necessary Win32 APIs, but that isn't the cleanest.
What version of Go are you using (go version)?

$ go version
go version go1.17.1 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/user/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/user/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.1"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/user/src/tailscale/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2930209240=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Compile this program for Windows
On a Windows machine, run it as a normal, non-elevated user.

What did you expect to see?
The program returns without error.
What did you see instead?
The program fails with access denied errors.

cc @golang/runtime @alexbrainman

@dblohm7 thank you for creating the issue.
I am not sure what you are trying to do, but I tried running your program with these changes in golang.org/sys/windows/svc/mgr:
diff --git a/windows/svc/mgr/mgr.go b/windows/svc/mgr/mgr.go
index de75f4a4..e17b1e84 100644
--- a/windows/svc/mgr/mgr.go
+++ b/windows/svc/mgr/mgr.go
@@ -39,7 +39,7 @@ func ConnectRemote(host string) (*Mgr, error) {
        if host != "" {
                s = syscall.StringToUTF16Ptr(host)
        }
-       h, err := windows.OpenSCManager(s, nil, windows.SC_MANAGER_ALL_ACCESS)
+       h, err := windows.OpenSCManager(s, nil, windows.GENERIC_READ)
        if err != nil {
                return nil, err
        }
@@ -162,7 +162,7 @@ func (m *Mgr) CreateService(name, exepath string, c Config, args ...string) (*Se
 // OpenService retrieves access to service name, so it can
 // be interrogated and controlled.
 func (m *Mgr) OpenService(name string) (*Service, error) {
-       h, err := windows.OpenService(m.Handle, syscall.StringToUTF16Ptr(name), windows.SERVICE_ALL_ACCESS)
+       h, err := windows.OpenService(m.Handle, syscall.StringToUTF16Ptr(name), windows.GENERIC_READ)
        if err != nil {
                return nil, err
        }

and your program still fails with
Opening service rpcss: Access is denied.

What you are trying to do exactly?
I made decision to just use windows.SC_MANAGER_ALL_ACCESS and windows.SERVICE_ALL_ACCESS when I developed this package for my own use case. And no one complained about running this code as non-elevated user, until now.
I obviously tried to keep mgr.Connect and mgr.OpenService parameter list short. Otherwise every user would have to pass at least windows.SC_MANAGER_ALL_ACCESS and windows.SERVICE_ALL_ACCESS to these functions.
Then I discovered that I needed access to the service manager on another Windows pc. That is how mgr.ConnectRemote happened.
I suppose we can add new methods similar to mgr.ConnectRemote and mgr.OpenService that also allows passing access rights. I am not sure what to call these. Happy with your suggestions.
Perhaps you have other ideas.
Thank you.
Alex

What you are trying to do exactly?

I'm trying to query a service - find out if a service by a specific name exists or not, and if it does query its configuration.

I made decision to just use windows.SC_MANAGER_ALL_ACCESS and windows.SERVICE_ALL_ACCESS when I developed this package for my own use case. And no one complained about running this code as non-elevated user, until now.

I appreciate that, and if it were a personal project I wouldn't be complaining, but since this package is sanctioned and sponsored by Google/Golang we would like to see it working better. Working when run as non-admin/not elevated (i.e. non-root in Linux equivalent terminology) is not a high bar to expect.

I obviously tried to keep mgr.Connect and mgr.OpenService parameter list short. Otherwise every user would have to pass at least windows.SC_MANAGER_ALL_ACCESS and windows.SERVICE_ALL_ACCESS to these functions.

More likely that the actual required permissions would have been passed rather than these two.  It's just one more parameter, not dozens. And if you want to assume required permissions, like os.Open in contract with os.OpenFile does, why assume the most extreme?

@alexbrainman, how's the investigation going? What are the chances that this gets fixed?

Ping?

@alexbrainman I have been working on some code to return a list of services on a Windows machine, and I needed to list the services without elevated privileges. Instead of passing windows.GENERIC_READ to windows.OpenSCManager, I found I needed to pass the specific access right to the function, SC_MANAGER_ENUMERATE_SERVICE in this case.
For opening a service, I similarly had to pass the specific access right I wanted to windows.OpenService. Since all I wanted was the configuration and status of the service, I passed in windows.SERVICE_QUERY_CONFIG|windows.SERVICE_QUERY_STATUS.
So the calls should look like this:
h, err := windows.OpenSCManager(s, nil, windows.SC_MANAGER_ENUMERATE_SERVICE)

and
h, err := windows.OpenService(m.Handle, syscall.StringToUTF16Ptr(name), windows.SERVICE_QUERY_CONFIG|windows.SERVICE_QUERY_STATUS)

I think making two new functions that accept permissions makes sense. Perhaps we can call them mgr.ConnectWithPermissions and mgr.OpenServiceWithPermissions. I know those are a bit long, but I am a verbose person :sweat_smile: .

@RafBishopFox

I think making two new functions that accept permissions makes sense. Perhaps we can call them mgr.ConnectWithPermissions and mgr.OpenServiceWithPermissions.

Sounds good to me. But I suspect we need to write a proposal now to add these functions.
If you want to write a proposal, here
https://github.com/golang/proposal?tab=readme-ov-file
are some info. You can probably make your proposal short, because it is not complicated.
And others might suggest better names for these functions.
Alex

I've been wanting exactly this as well.
I have locally developed changes that create a "ConnectPerms" method and an "OpenServicePerms" method (I prefer the suggested "ConnectWithPermissionss", "OpenServiceWithPermissions" though). The changes are simple enough, the existing methods just delegate to the new ones with the appropriate permissions. I've used them and they work just fine.
My use case is that I want to be able to create a user with only permissions to start and stop certain permissions, so I need to control the permissions I request when I open the service control manager and then when I open the service.
I was going to work up a proper PR, but it sounds like that would be premature at this point until a proposal is agreed?

Proposal raised at #66694 since I couldn't find an existing one.
The changes are simple enough, but it would take me a little while to get into the position to submit the PR according to requirements etc.


Garden Rocks

Add middle size rocks

Improved version of Rocks assets. The top image shows how it was looking before and the bottom image how it looks now.


Document your project

Document your project in your README.md so that you and other people can understand what the project does.

Abstract
Installation Guide
License
Thank you note maybe
You can refer this for the documentation purpose Document

Completed


Fixed filters

There was a bug that prevented the use of filters in the listAll() method

Accidentally created the pull request on the master of my fork, so I'll close this one.

Feel free to add it to the master repo as well :-)


Added support for optimistic locking on delete

Please ignore the empty line changes, my editor is just stripping out trailing whitespaces. I did not make any changes to the destroy_all method since that does batch writes and batch writes do not support conditional checks.

I have also noticed that using integers for lock version is not ideal since you can bypass the lock version check if you put in a decremented value. For example the following spec fails
it 'prevents concurrent writes even when lock version is dirty (modified)' do
    address.save!
    a1 = address
    a2 = Address.find(address.id)

    a1.lock_version = 0
    a1.city = 'Seattle'
    a2.city = 'San Francisco'

    a1.save!
    expect { a2.save! }.to raise_exception(Dynamoid::Errors::StaleObjectError)
end

It might be better to use SecureRandom.hex to randomly generate a lock_version string instead of using integers and incrementing them

Thanks for the PR! Looks great overall. I just added some small stuff. I'll think about what you said regarding using a random string.


Chisel tester support

…subclass which allows tester to have clean up and other construction code executed after a user code executed during constructor of that subclass.
Created a spec file TesterDriverSpec that shows finish method being called

retest this please

The problem appears to be due to changes in firrtl.  I was running an older version, specifically:
commit f3c4c604549db6bbe824c29649bb05bba7470d8a
Author: jackkoenig<EMAIL_ADDRESS>Date:   Fri Jan 29 11:08:14 2016 -0800

    Add project/build.properties with requirement of sbt 0.13.6, this version is required for sbt assembly to work

When I pulled the latest changes to take me to:
commit 404b5a18483e86088e29b333cc61d6628ae50d15
Author: azidar<EMAIL_ADDRESS>Date:   Tue Feb 9 19:15:25 2016 -0800

    Updated Readme

When I run
sbt
> test-only chiselTests.TblSpec

I see errors like:
Starting High Form Check
Fatal Uncaught Exception.
NoFileInfo: [module Tbl]  Width cannot be negative or zero.
NoFileInfo: [module Tbl]  Width cannot be negative or zero.
<EMAIL_ADDRESS>[module Tbl]  Width cannot be negative or zero.
   at<EMAIL_ADDRESS>   at<EMAIL_ADDRESS>

retest this please

Can this get merged now.  Adam's firrtl fix, fixed the errors

One last grammar issue to fix.

It's been changed from "it's" has been changed to "its"

lgtm


Firebase-admin 10.0.0 modular imports: Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

[READ] Step 1: Are you in the right place?
[REQUIRED] Step 2: Describe your environment

Operating System version: windows 10
Firebase SDK version: firebase: 9.1.3 ; firebase-admin: 10.0.0 ; firebase functions: 3.16.0
Firebase Product: app
Node.js version: 16
NPM version: 7.5.6

[REQUIRED] Step 3: Describe the problem
After the migration to the new firebase-admin 10.0.0 and the new modular sdk imports, deploying the functions and running them works great.
However, when unit testing them, jest doesn't find the module packages and gives this as output:
FAIL  spec/tests/app/functions/util-helpers.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  spec/tests/app/functions/util-helpers.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

PASS  spec/tests/app/import-service/validators/inbetween-integer.validator.spec.ts
FAIL  spec/tests/app/scoring/team-ranking.service.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  functions/src/utils/AcroSpecificRankingServiceBackend.ts
  spec/tests/app/scoring/team-ranking.service.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

PASS  spec/tests/app/import-service/validators/email.validator.spec.ts
FAIL  spec/tests/app/generic-competitions/generic-acro-ranking.service.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  functions/src/utils/GenericRankingServiceBackend.ts
  spec/tests/app/generic-competitions/generic-acro-ranking.service.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

PASS  spec/tests/app/import-service/validators/positive-integer.validator.spec.ts
FAIL  spec/tests/app/scoring/acro-specific-ranking.service.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  functions/src/utils/AcroSpecificRankingServiceBackend.ts
  spec/tests/app/scoring/acro-specific-ranking.service.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

PASS  spec/tests/app/import-service/validators/firestore-key.validator.spec.ts
PASS  spec/tests/app/import-service/validators/length.validator.spec.ts
PASS  spec/tests/app/import-service/validators/split-and-apply-multiple-validators.spec.ts
PASS  spec/tests/app/helpers.service.spec.ts
PASS  spec/tests/app/trampoline/judges-trampoline-service.spec.ts
PASS  spec/tests/app/generic-competition-admin/exercises-import.spec.ts
● Console
console.log
  Loading overlay was enabled:

  at MockLoadingOverlayService.enableOld (spec/tests/stubs/loading-overlay.stub.ts:5:13)

console.log
  text Adding exercises...

  at MockLoadingOverlayService.enableOld (spec/tests/stubs/loading-overlay.stub.ts:6:13)

console.log
  keepOnRouteDisable false

  at MockLoadingOverlayService.enableOld (spec/tests/stubs/loading-overlay.stub.ts:7:13)

console.log
  config {}

  at MockLoadingOverlayService.enableOld (spec/tests/stubs/loading-overlay.stub.ts:8:13)

Summary of all failing tests
FAIL  spec/tests/app/functions/util-helpers.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  spec/tests/app/functions/util-helpers.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

FAIL  spec/tests/app/scoring/team-ranking.service.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  functions/src/utils/AcroSpecificRankingServiceBackend.ts
  spec/tests/app/scoring/team-ranking.service.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

FAIL  spec/tests/app/generic-competitions/generic-acro-ranking.service.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  functions/src/utils/GenericRankingServiceBackend.ts
  spec/tests/app/generic-competitions/generic-acro-ranking.service.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

FAIL  spec/tests/app/scoring/acro-specific-ranking.service.spec.ts
● Test suite failed to run
Cannot find module 'firebase-admin/app' from 'functions/src/index.ts'

Require stack:
  functions/src/index.ts
  functions/src/utils/UtilHelpers.ts
  functions/src/utils/AcroSpecificRankingServiceBackend.ts
  spec/tests/app/scoring/acro-specific-ranking.service.spec.ts

  1 | 'use strict';
  2 |
> 3 | import { initializeApp, App } from 'firebase-admin/app';
    | ^
  4 | import { getFirestore } from 'firebase-admin/firestore';
  5 |
  6 | import 'firebase-functions';

  at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
  at Object.<anonymous> (functions/src/index.ts:3:1)

ESLint is also having problems:
https://stackoverflow.com/questions/69746672/unable-to-resolve-path-to-module-firebase-admin-app-eslint

Experiencing the same problem here: Cannot find module 'firebase-admin/firestore' from 'src/common/batch.ts'. whilst running unit tests.
"firebase-admin": "10.0.0",
"@types/jest": "27.0.2",
"jest": "27.3.1"
"ts-jest": "27.0.7"
"typescript": "4.4.4"

Issue with Jest is a known limitation in Jest's module resolver. See #1465 for more details including a link to the original bug report on the Jest project, and some workarounds. Good news is there's a good chance Jest might fix this issue in their next major version.
Issue with ESLint is a similar known problem with the corresponding tool. See https://github.com/firebase/firebase-admin-node/discussions/1359. You will most likely have to implement the workaround mentioned in that thread.


ContentUtils bulk update

DBFlow Version: 4.0.0-beta7
Issue Kind (Bug, Question, Feature): Question, Feature
Description:
I'm working with ContentProvider (for SyncAdapter) and at some point in the code I need to update many rows. As there isn't ContentUtils.bulkUpdate() method, I try to replace it by:
OperatorGroup where = getBaseWhereClause();
where.andAll(getSyncTypeColumn().eq(type), getSyncStatusColumn().in(STATUS_PENDING, STATUS_RETRY));
SQLite.update(getModelClass())
                .set(getSyncStatusColumn().eq(STATUS_IN_PROGRESS))
                .where(where)
                .executeUpdateDelete();

It isn't working while other operations made by ContentUtils work fine (Cause of CotentProvider I guess). I also try to execute a simple select as following:
SQLite.select().from(OrmAddressBook.class).where().queryList()
return 0 row while
ContentUtils.queryList(OrmAddressBook.CONTENT_URI, OrmAddressBook.class,nonGroupingClause(),null)
return all rows.
Is the problem come from ContentProvider ?
Is it planned to add a bulkUpdate() method to ContentUtils in the future ?
Thanks

SQLite wrapper objects do not know about the associated content provider model system, and thus do not sync with a content provider. in terms of what is supported for ContentProvider , it does not support  bulkUpdate. SQL wrapper updates are real sql queries, which content providers do not support directly.


Illegal redistribution of Android SDK

Hi,
Just an FYI it is illegal to create public docker images containing the Android SDK as per response from Google.
Cheers

Hi @Malinskiy,
First of all, thank you for the heads-up :)
In fact, this is not a problem for this implementation:

The legal concern has been already considered from the beginning of this repo:

not to redistribute the SDK is the legal behavior.

According to the philosophy, Android SDK shouldn't be part of the image, for several reasons:

You can maintain an external persistent SDK directory, and mount it to any container.

Instead of one dedicated Docker image per Android API level (which will end up with a ton of images), you just have to deal with one image.

The current Dockerfile is not tied to the Android SDK, except for the PATH environment variable, which is not a big deal.
Thanks again for your concern.
From the next major release on, we're going to remove Android SDK completely, as well as Gradle and Kotlin.


Example Missing

This article seems to be missing an example. Here is an example written in LinqPad that I think might be worth including.
void Main()
{
// Create objects and serialize them into an XML file.
Student s = new Student() {Name="Mark"};
Teacher t = new Teacher() {Name="Katie"};
SerializePerson(s, @"C:\Users\username\Desktop\student.xml");
SerializePerson(t, @"C:\Users\username\Desktop\teacher.xml");
// Deserialize the files just created into in-memory objects.
XmlSerializer xs = new XmlSerializer(typeof(Person));

Person p1;
using (Stream stream = File.OpenRead(@"C:\Users\username\Desktop\student.xml"))
	p1 = xs.Deserialize(stream) as Person;

Person p2;
using (Stream stream = File.OpenRead(@"C:\Users\username\Desktop\teacher.xml"))
	p2 = xs.Deserialize(stream) as Person;

Console.WriteLine($"The student's name is {p1.Name}.");
Console.WriteLine($"The teacher's name is {p2.Name}.");

}
[XmlInclude(typeof(Student))]
[XmlInclude(typeof(Teacher))]
public class Person { public string Name; }
public class Student : Person {}
public class Teacher : Person {}
public void SerializePerson(Person p, string path)
{
XmlSerializer xs = new XmlSerializer(typeof(Person));
using (Stream s = File.Create(path))
xs.Serialize(s, p);
}

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a4c29a31-52c5-1142-08d0-ef7cabc0053b
Version Independent ID: dd9e91fd-3ed4-d61e-4b8c-cba4baec3fcb
Content: XmlIncludeAttribute Class (System.Xml.Serialization)
Content Source: xml/System.Xml.Serialization/XmlIncludeAttribute.xml
Product: .net
GitHub Login: @dotnet-bot
Microsoft Alias: dotnetcontent

Hi @markdcarringer. Thanks for pointing that out. It seems that something happened in between migrations from one system to the other. This API used to have an example: https://msdn.microsoft.com/en-us/library/system.xml.serialization.xmlincludeattribute(v=vs.100).aspx#Examples
We'll add that back again to the docs.

I have opened an internal bug because it seems we're not currently supporting appropriately some ASP.NET file extensions like asmx, ascx, asax. etc.
Internal bug [295680(https://ceapex.visualstudio.com/SRE/_workitems/edit/29568)


InternVL Colab notebook

Welcome to try the InternVL Colab notebook:
https://github.com/Czi24/Awesome-MLLM-LLM-Colab/blob/master/MLLM/InternVL-colab/InternVL.md
It provides a convenient environment for testing and running the model.

It is 404 page, please fix this issue and give me proper link, I'm searching for Intern-VL2 40B colab notebook so badly, please if you can help me then do that!


Error importing modules

After installing the new toml file, I get some errors importing some modules. I have put a fix in #14 that works for me.

Got it, your suggested change makes sense and has been merged with the main branch. Thanks for pointing this out!


Update contributors list

What this PR does / why we need it:
Welcome @Forget-C & @fazledyn-or :tada:
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?:

How are users affected by this change:
Is this breaking change:
How to migrate (if breaking change):

お二人がまだ寝ないですか 🤣

対応早すぎるw


Interesting behavior when running in parallel

What is the issue?
Hi!
I am trying to embed a lot of documents with jina/jina-embeddings-v2-base-de. I am trying to use request parallelism with OLLAMA_NUM_PARALLEL=16. I have a multiprocessing script that spawns 16 processes. When watching the Ollama output I observe the following.

A lot of request got through fine.
Then suddenly an assertion is violates. See below.
Ollama then purges the model from GPU memory and reloads it.
My processes continue. I have some retry-mechanism.

Here is the assertion's output:
GGML_ASSERT: /go/src/github.com/ollama/ollama/llm/llama.cpp/llama.cpp:12063: seq_id < n_tokens && "seq_id cannot be larger than n_tokens with pooling_type == MEAN"
Any ideas?
OS
Linux
GPU
Nvidia
CPU
AMD
Ollama version
0.1.45

Do you see better behavior with a smaller parallel setting?  In the upcoming release (0.1.49) we'll be defaulting to 4 if the GPU has sufficient VRAM for the context.
How many CPU cores do you have?  Is it by any chance less than 16?  You might try setting it to a value equal or less than your core count and see if that improves reliability.

Please make sure to upgrade to the latest version, and try first with the default settings, and if you're still having troubles, please share more information about your setup and I'll reopen the issue.


Fix the issue of taking the zone parameter value in ZoneAwareClusterInvoker

What is the purpose of the change
#8451

Codecov Report

Merging #8521 (0b10e14) into master (6acd5a2) will decrease coverage by 0.74%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master    #8521      +/-   ##
============================================
- Coverage     61.06%   60.31%   -0.75%
+ Complexity      447      446       -1
============================================
  Files          1097     1097
  Lines         44401    44354      -47
  Branches       6469     6437      -32
============================================
- Hits          27113    26754     -359
- Misses        14302    14626     +324
+ Partials       2986     2974      -12

Impacted Files
Coverage Δ

...ster/support/registry/ZoneAwareClusterInvoker.java
0.00% <0.00%> (ø)

...gcenter/wrapper/CompositeDynamicConfiguration.java
0.00% <0.00%> (-73.34%)
:arrow_down:

...ookeeper/ZookeeperDynamicConfigurationFactory.java
0.00% <0.00%> (-66.67%)
:arrow_down:

...pport/zookeeper/ZookeeperDynamicConfiguration.java
0.00% <0.00%> (-62.86%)
:arrow_down:

...fig/configcenter/TreePathDynamicConfiguration.java
31.42% <0.00%> (-54.29%)
:arrow_down:

.../dubbo/metadata/report/MetadataReportInstance.java
12.50% <0.00%> (-40.63%)
:arrow_down:

.../configcenter/support/zookeeper/CacheListener.java
0.00% <0.00%> (-34.29%)
:arrow_down:

...gistry/client/migration/MigrationRuleListener.java
55.88% <0.00%> (-23.53%)
:arrow_down:

...ng/transport/dispatcher/all/AllChannelHandler.java
62.06% <0.00%> (-20.69%)
:arrow_down:

...mmon/config/configcenter/DynamicConfiguration.java
68.18% <0.00%> (-18.19%)
:arrow_down:

... and 57 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6acd5a2...0b10e14. Read the comment docs.


prod server

IMPORTANT: This repository's issues are reserved for feature requests and bug reports. Do not submit support requests here.
Steps to reproduce and a minimal demo of the problem
What steps should we try in your demo to see the problem?
Current behavior
the production server from the original seed project was removed
Expected/desired behavior
searching for a recommended way to run the production build
Other information
i assume there is a reason that the production server was removed from this fork, just wondering if there is a suggested method of running the production build now?

@dweitz43 - Production builds have varied requirements which is why serving a prod build was pulled from the base seed.
To add a basic production
Add: tools/tasks/project/serve.prod.ts
import {serveProd} from '../../utils';

export = serveProd

Modify: gulpfile.ts and add the following below the serve.e2e task
// --------------
// Serve prod
gulp.task('serve.prod', (done: any) =>
  runSequence('build.prod',
              'server.prod',
              done));


feat: 소통창구인 Discord 초대 링크를 추가합니다

Overview

es-hangul을 사용해주시는 개발자분들과 더 자유로운 소통을 위해 Discord 초대 링크를 추가합니다.
PR Checklist

[X] I read and included theses actions below

I have read the Contributing Guide
I have written documents and tests, if needed.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.53%. Comparing base (06a6e1c) to head (2515105).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #289   +/-   ##
=======================================
  Coverage   99.53%   99.53%
=======================================
  Files          38       38
  Lines         642      642
  Branches      156      156
=======================================
  Hits          639      639
  Misses          3        3

----
🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests
JS Bundle Analysis - Avoid shipping oversized bundles


AzureRM Terraform Provider Contribution Guide

This PR provides documentation to get started to contribute to the Azure RM Terraform Provider.

@grayzu thanks for your review. I've just pushed some updates according to your comments/suggestions. LMK if you see anything else that should be updated.

Thank you!


add Source Han Serif font

Motivation for this change
Source Han Serif font is an OpenType pan-CJK serif font family by Adobe.
It contains Japanese, Korean, Traditional Chinese, and Simplified Chinese glyphs.
The font is licensed under SIL Open Font License.
https://github.com/adobe-fonts/source-han-serif
https://source.typekit.com/source-han-serif
Things done

[X] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

Tested with Inkscape.

I am getting 404s when trying to build this:
> nix-build -A source-han-serif-japanese
these derivations will be built:
  /nix/store/snvnvwj2kzhvpq82b256ddv3c4iwxc3x-SourceHanSerifJ_EL-M.zip.drv
  /nix/store/yhrwin5z1hz1xw7igxb074k89kyi68k3-SourceHanSerifJ_SB-H.zip.drv
  /nix/store/f2xflwxgdl76rz7ljsvml9jdb57ha7wv-source-han-serif-japanese-1.000R.drv
building path(s) ‘/nix/store/9ln9zwri8l0w48m6cnwssrz1rxs96b8w-SourceHanSerifJ_EL-M.zip’
building path(s) ‘/nix/store/j077z4cg0x6fklpfgm2s2zpmbm1mgi94-SourceHanSerifJ_SB-H.zip’

trying https://github.com/adobe-fonts/source-han-serif/raw/f6cf97d92b22e7bd77e355a61fe549ae44b6de76/SubsetOTF/SourceHanSerifJ_SB-H.zip
trying https://github.com/adobe-fonts/source-han-serif/raw/f6cf97d92b22e7bd77e355a61fe549ae44b6de76/SubsetOTF/SourceHanSerifJ_EL-M.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download SourceHanSerifJ_SB-H.zip from any mirror
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download SourceHanSerifJ_EL-M.zip from any mirror
builder for ‘/nix/store/yhrwin5z1hz1xw7igxb074k89kyi68k3-SourceHanSerifJ_SB-H.zip.drv’ failed with exit code 1
cannot build derivation ‘/nix/store/f2xflwxgdl76rz7ljsvml9jdb57ha7wv-source-han-serif-japanese-1.000R.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/f2xflwxgdl76rz7ljsvml9jdb57ha7wv-source-han-serif-japanese-1.000R.drv’ failed

Fixed. It pointed wrong URLs. It happened to work on my machine because I saved archives to /nix/store manually with nix-store.

LGTM now, thanks.


use a bucket name that more clearly shows the user must replace it

Fixes internal issue 372853854

@cjac Mind taking a look? I made you an admin on this repo since you most recently contributed.


Pseudoinverse for multivectors

Would it be possible to add an option to use the matrix pseudoinverse instead of the normal matrix inverse into the method for computing inverses for multivectors? Having read the relevant chapter in Christian Perwass' book, he also mentions there that a pseudoinverse should work. I think it would be very nice to add an option to e.g. the .inv() method that allows something like
M.inv(pseudo=True)

given that in some situations this is still meaningful to do.

I assume this is for degenerate basis vectors, when sometimes there is no inverse?

Yes that's one case, and another one I've found is when performing bivector splits when the two bivectors in the split B = B_1 + B_2 start to become to similar. When they become identical the real inverse gives a LinAlgError, but the pseudoinverse returns B_1 = B_2 = B / 2. This is much friendlier ;).

If you update to the development version of clifford, you should find that the linear algebra method is not used at all.

I think we still use it for algebras greater than dimension 6?

I've taking the liberty of making a PR for this with a slightly different API then suggested above: I added a .pinv() method to really make it easy to distinguish. Let me know if you like it or want to see some changes!


mismatched cluster ID between containers from different VM

I got a environment. there are two VM which can communicate with IP like:192.168.XXX.XXX
Let's say I got two container in the VM1 and one in the VM2.
I try to bring up this three to a cluster.
The VMs export the responding ports to the containers. I can ping through the VM1 from VM2.
The trick is whenever I try to bring up the cluster, error: mismatched cluster will be raised.
As I found the on the two VMs, there are different cluster ID and the members have different ID two.
The config file as follow:
The VM1(<IP_ADDRESS>):
etcd
--name='node3'
--listen-client-urls "http://<IP_ADDRESS>:10805"
--advertise-client-urls "http://localhost:10805"
--listen-peer-urls "http://<IP_ADDRESS>:10810"
--initial-advertise-peer-urls "http://localhost:10810"
--initial-cluster "node=http://<IP_ADDRESS>:20822,node2=http://<IP_ADDRESS>:20824,node3=http://localhost:10810"
--initial-cluster-state "new"
--data-dir "/root/node"
--initial-cluster-token "ha-cluster"
The VM2(<IP_ADDRESS>):
etcd
--name "node1"
--listen-client-urls "http://<IP_ADDRESS>:20821"
--advertise-client-urls "http://<IP_ADDRESS>:20821"
--listen-peer-urls "http://<IP_ADDRESS>:20822"
--initial-advertise-peer-urls "http://localhost:20822"
--initial-cluster "node1=http://localhost:20822,node2=http://localhost:20824,node3=http://<IP_ADDRESS>:10810"
--initial-cluster-state "new"
--data-dir "./node1"
--initial-cluster-token "ha-cluster"
etcd
--name "node2"
--listen-client-urls "http://<IP_ADDRESS>:20823"
--advertise-client-urls "http://<IP_ADDRESS>:20823"
--listen-peer-urls "http://<IP_ADDRESS>:20824"
--initial-advertise-peer-urls "http://localhost:20824"
--initial-cluster  "node1=http://localhost:20822,node2=http://localhost:20824,node3=http://<IP_ADDRESS>:10810"
--initial-cluster-state "new"
--data-dir "./node2"
--initial-cluster-token "ha-cluster"
The log in VM1 as:
2017-12-14 08:10:02.659837 I | etcdserver: advertise client URLs = http://localhost:10805
2017-12-14 08:10:02.659842 I | etcdserver: initial advertise peer URLs = http://localhost:10810
2017-12-14 08:10:02.659866 I | etcdserver: initial cluster = node2=http://<IP_ADDRESS>:20824,node3=http://localhost:10810,node=http://<IP_ADDRESS>:20822
2017-12-14 08:10:02.670005 I | etcdserver: starting member ea419ac1c376565c in cluster cee48dfec45a1229
the log in VM2 as:
2017-12-14 08:09:44.928156 I | etcdserver: advertise client URLs = http://<IP_ADDRESS>:20823
2017-12-14 08:09:44.928186 I | etcdserver: initial advertise peer URLs = http://localhost:20824
2017-12-14 08:09:44.928237 I | etcdserver: initial cluster = node1=http://localhost:20822,node2=http://localhost:20824,node3=http://<IP_ADDRESS>:10810
2017-12-14 08:09:44.935114 I | etcdserver: starting member ac75c1372417afd2 in cluster d3013d4b1124c0e1
2017-12-14 08:09:37.680059 I | embed: listening for peers on http://<IP_ADDRESS>:20822
2017-12-14 08:09:37.680691 I | embed: listening for client requests on <IP_ADDRESS>:20821
2017-12-14 08:09:37.705180 I | etcdserver: name = node1
2017-12-14 08:09:37.705250 I | etcdserver: data dir = ./node1
2017-12-14 08:09:37.705272 I | etcdserver: member dir = node1/member
2017-12-14 08:09:37.705288 I | etcdserver: heartbeat = 100ms
2017-12-14 08:09:37.705304 I | etcdserver: election = 1000ms
2017-12-14 08:09:37.705320 I | etcdserver: snapshot count = 100000
2017-12-14 08:09:37.705343 I | etcdserver: advertise client URLs = http://<IP_ADDRESS>:20821
2017-12-14 08:09:37.705361 I | etcdserver: initial advertise peer URLs = http://localhost:20822
2017-12-14 08:09:37.705412 I | etcdserver: initial cluster = node1=http://localhost:20822,node2=http://localhost:20824,node3=http://<IP_ADDRESS>:10810
2017-12-14 08:09:37.718732 I | etcdserver: starting member f408e3cacb3f4275 in cluster d3013d4b1124c0e1
It seems that the two  in VM2 have the same cluster ID, but different from the one of VM2. I have no idea what is going on?

Don't use localhost in advertise URLs and initial-cluster flag.
Please see https://github.com/coreos/etcd/blob/master/Documentation/faq.md#what-is-the-difference-between-listen-clientpeer-urls-advertise-client-urls-or-initial-advertise-peer-urls.

yes, but this container only has the local ip. I can not use the IP of VM which will result in error. And the container ip is 10.8.2.X.  I don't think this ip can be connected from outside. Actually the connection is achieved only by the exposed port of VM. Anyway I will try with the 10.8.2.X.

OK, one more question just for the confirmation. Can I just use the local address of container with the exposed IP to be connected from external?

I don't think this ip can be connected from outside

You can open the host port to your container port (e.g. -p 2379:2379).

local address of container with the exposed IP to be connected from external?

Advertise URLs must be reachable by other nodes. How you set up local address in container depends on the container tool.

yes, actually I did export the port to container from host.  And the containers now across the VM can communicate with each other, because I got the mismatched cluster ID error. The problem now is I still got the error of mismatched cluster ID. But I can reach the cluster with etcdctl on one VM and cluster on the other VM. So I am confused.

@WandyLau Make sure clean up the data directory when you restart?

@gyuho yes. absolutely. Is it possible that different VM will produce different cluster ID according the different --initial-cluster parameters?  Because in different VM, this parameter is different. Just doubt it.


GIF exports

Expected Behavior
An option to make GIF.
Example : For any repo that can show their work effectively in their README file such as installation instructions.
Actual Behavior

Browser:

Code Snippet (If Applicable)

@V1NAY8 have you tried https://getkap.co/ for quickly creating a GIF? I don't think it would be feasible to create one from the browser, unless you have ideas 🙂

Kap is for MacOS, but I work on PC and Ubuntu :walking_man:

What would the advantage of a gif be compared to a png? A png can just as well be used in a README of a repo, or do you mean an animated gif? If so, what would you animate?

Just that carbon can be used for styling code. I wanted it also to behave as styling for Terminal or any UI.
Animating the Instructions will always help.

Example for Code: If the same is done in Carbon It looks Beautiful

These are nothing but a multiple output images from terminal.(If we could copy the output from terminal and paste it here and combine them). Just an Idea.

png can be used to demonstrate just the code itself. But when it comes to gif. We can demonstrate multiple commands or instructions in the Documentation of a Repo.

Ah okay, I see what you mean. Do you mean something like https://asciinema.org ? If so, how should it be different?

Oh yeah, That's such a beautiful one. As of now, that's the idea.
The above is present only for terminal via the installation of a package.
Thinking whether we can perform the same in Browser. https://getkap.co/ is the one suggested above, but its available for MacOs only.
The main idea is to use Carbon to create a beautiful environment for sharing code, along with these integrations

I'd like to add another perspective to GIF support. I make tutorial videos and I want them to be pretty. So, I'm thinking about using carbon when I show a piece of code. Although, having a window scale up animation would be great on that background which will produce either a GIF or a video file.
I think that can be a pretty good and needed enhancement.

I did a poor-mans approach to this on a fork carbon#feat/record-terminal-sesh using gif.js
There's a record button next to the settings. Press that and then all the keystrokes will be recorded and then "replayed" once you stop recording.
It's not polished. There are bugs and it is slow. It worked for my situation and lets me make pretty README gifs and twitter gifs, so I'm pretty stoked.
If this were to get polished, the next steps would be:

Improve speed (reevaluate approach of using setState)
Fix mixed-dimensions problem
Support highlighted lines
Add preview/save step separate from recording step
Add menu for gif options

@JessicaSachs this is amazing! 🎉 Really awesome work.
I would be perfectly happy to ship just a record feature on it's own. Highlighted lines and options are less important for an initial version, IMHO.
If you are up for integrating this into Carbon, I would be incredibly happy to help support the effort on my nights and weekends :+1: — or consult asychronously.
If you do not have the bandwidth, which I totally understand, I will try and get to it eventually. I looked at the diff, and it looks like it would not be too hard to integrate.

(Previous comment got deleted 😞)
@JessicaSachs this is amazing! 🎉 So cool that you got this working! I had never heard of gif.js before
If you would be up for it, I would absolutely love to help integrate this work into Carbon itself on my nights and weekends. Also happy to consult asynchronously if that works better 🙂I totally understand if you don't have the bandwidth — even if you pushed up a WIP PR rebased on master, that would be incredibly helpful.
Finally, FWIW, I would be more than happy to merge this with only a working record feature. No need for line-highlighting or menu options for an initial version, IMO.
I 🖤 the Carbon community!

Carbon.now.sh  use dom-to-image library to generate images and svg from any div (dom) , unfortunately gif export is not possible with this library.
By @neerajcodes

Carbon.now.sh use dom-to-image library to generate images and svg from any div (dom) , unfortunately gif export is not possible with this library.

This is true, however as @JessicaSachs demonstrated above, it is possible to combine this with a GIF library to create them. I haven't had time to implement this feature, though.

@mfix22 if I rebased, would you be able to take it over the line?

@mfix22 if I rebased, would you be able to take it over the line?

@JessicaSachs unfortunately, I don't have much time for feature development at the moment 😞 If you rebased and pushed up a PR, that could be good so I can work on it steadily in the background, though.

This is an interesting approach: https://www.animate-code.com/


Fix a racy read in test

Unfortunately I seem to have missed this wait() call in #3975, and it decided to assert itself recently:
https://buildkite.com/uberopensource/cadence-server/builds/11015#bf06bca6-2ad3-49fd-a063-de1dd67655b9
(races between line 363 and the suite's SetT() method)
This should fix it.  There aren't any other missing-wait()s AFAICT.

Pull Request Test Coverage Report for Build 6d6a5e6a-1650-4fb9-a36d-760faa60f3d8

0 of 0   changed or added relevant lines in 0 files are covered.
56 unchanged lines in 7 files lost coverage.
Overall coverage increased (+0.03%) to 59.777%

Files with Coverage Reduction
New Missed Lines
%

service/history/queue/timer_queue_processor_base.go
1
78.6%

common/task/parallelTaskProcessor.go
2
92.48%

service/matching/taskListManager.go
2
74.72%

common/persistence/cassandra/cassandraPersistence.go
3
50.62%

service/history/queue/timer_gate.go
3
95.83%

service/history/execution/mutable_state_task_refresher.go
14
70.54%

common/persistence/cassandra/cassandraPersistenceUtil.go
31
84.81%

Totals

Change from base Build 6a0816f9-b68a-48c1-9f27-82d25a3cd276:
0.03%

Covered Lines:
90384

Relevant Lines:
151201

💛  - Coveralls


Language

Hi,
This is great!
but can you fix the language in english when prompting file chooser?
Thanks alot for this plugin.
btw,
could you add geolocation support with this plugin?

Yeah , i do face same problem that file chooser use chinese language ,

@djks74  Hello, English is supported on version 0.0.7

@wenzhiming Thanks for this. :-)


Implement basic Sass outline from Pillars

We have a nice, clear Sass structure defined in Pillars but we don't have this added by default to Riggings.
Could we add a few basic Sass files (without any complex style rules) into Riggings to make it easier to meet the Pillars conventions and structure our Sass more consistently on new projects?

Yup, need to update Pillars as well with new learnings etc. A little bit outdated now.


Allow controlling if a field is marked as required

Sometimes it's better UX to not mark required fields with "*" and add an "optional" hint to non-required fields instead.
Default is still to mark required fields: https://www.nngroup.com/articles/required-fields/.
@danharrin, do you want me to add docs for this as well?

I feel like adding new attributes to the field wrapper is getting really messy and I don't see this as a priority feature. We should add this to v3 instead, where field wrappers have already been cleaned up.
Also yes, docs would be nice

I don't think it's such a big issue to add this to v2 now and merge it into v3. But sure, I'll PR it to v3 instead.

All instances should be MarkedAsRequired instead of MarkedRequired I think, with the setter being markAsRequired()

Agree, was in doubt about that.

Ah, good one!

This should be good to go now, @danharrin.


Feature request: IterableDataset.push_to_hub

Feature request
It'd be great to have a lazy push to hub, similar to the lazy loading we have with IterableDataset.
Suppose you'd like to filter LAION based on certain conditions, but as LAION doesn't fit into your disk, you'd like to leverage streaming:
from datasets import load_dataset

dataset = load_dataset("laion/laion400m", streaming=True, split="train")

Then you could filter the dataset based on certain conditions:
filtered_dataset = dataset.filter(lambda example: example['HEIGHT'] > 400)

In order to persist this dataset and push it back to the hub, one currently needs to first load the entire filtered dataset on disk and then push:
from datasets import Dataset

Dataset.from_generator(filtered_dataset.__iter__).push_to_hub(...)

It would be great if we can instead lazy push to the data to the hub (basically stream the data to the hub), not being limited by our disk size:
filtered_dataset.push_to_hub("my-filtered-dataset")

Motivation
This feature would be very useful for people that want to filter huge datasets without having to load the entire dataset or a filtered version thereof on their local disk.
Your contribution
Happy to test out a PR :)

+1

+1

+1, should be possible now? :) https://huggingface.co/blog/xethub-joins-hf

Haha we're working hard to integrate Xet in the HF back-end, it will enable coo use cases :)
Anyway about IterableDataset.push_to_hub, I'd be happy to to provide guidance and answer questions if anyone wants to start a first simple implementation of this

+1

+1


Description
Display more informations with -v and -verbose

the bug previously mentioned is an issue with how GO itself parses cli arguments, might fix later but would require a refacto of cli parsing.


Support querying metrics

Hi,
It would be great to be able to references pod metrics when querying. It would allow complex workflow not supported by simply setting limits.
Thanks!

Sounds incredible. I would like to make this a highlight of an upcoming release.


TypeError in plugin "terser"

Hi, any idea why this error is happening?
[17:24:11] TypeError in plugin "terser" Message:     The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined Details:     code: ERR_INVALID_ARG_TYPE     domainEmitter: [object Object]     domainThrown: false
when using the following code, which worked 2 weeks ago:
gulp.task('build:scripts', function () {     return gulp.src(config.app_files.js)         .pipe(sourcemaps.init())         .pipe(rollup({ }, { format: 'cjs', strict: false }))         .pipe(concat('script.js'))         .pipe(terser())         .pipe(rename(function (path) {             path.extname = ".min.js"         }))         .pipe(sourcemaps.write('.', {             mapFile: function (mapFilePath) {                                 return mapFilePath.replace('.js.map', '.map');             }         }))         .pipe(gulp.dest(config.build_dir)); });
Thanks!

This error is caused by the v5 version of terser, the terser.minify function was adjusted from synchronous to asynchronous. Now this problem has been fixed.

Thanks so much for your quick reply and fix. Have a nice day.

Thanks so much for your quick reply and fix. Have a nice day.

😀


Lots more specs

Git Commit Message
SQL
XML
XSLT
HAML

I'm wondering if there is already a collection with sample code for all common languages. Found this: https://github.com/isagalaev/highlight.js/tree/master/test/detect Not sure if we can just take it and use as a flat list. Maybe with permission.
Or the spec should only include languages that get bundled by Atom. https://github.com/atom/atom/blob/master/package.json#L130-L161. Maybe that's not enough.
Also, was thinking that the spec should have first a "typical" code example. So that theme authors can kinda see what it usually looks like. And then at the bottom, there could be tests and edge cases.

Well, that would definitely be easier than coming up with all the examples by hand. :)
highlight.js is released under the BSD license, so we'd be allowed to borrow their code as long as we attributed it back to them and retain the copyright notice. Regardless, I'd still suggest we obtain permission first.
On the subject of only including languages bundled with Atom, I'd say that probably isn't enough. We probably want to focus on looking great with all the bundled languages first but if we want the theme to be useful to the most amount of people, I'd suggest we stray beyond the core language set.
I think we're going to have a pretty difficult time coming up with a "typical" example in many cases. Every individual/organization is going to have their own patterns and practices, which make it difficult to nail down what typical code looks.

I think we're going to have a pretty difficult time coming up with a "typical" example in many cases. Every individual/organization is going to have their own patterns and practices, which make it difficult to nail down what typical code looks.

Yeah, it's less about if the code is correct or if it follows some patterns and practices. Maybe a better word would be a "basic" example. Just so that you don't see only edge cases, but also how syntax looks when it's used in a normal way. Anyways, I might add it to some that I know enough.


"local" port is not honoured

I have some local defined ports in /usr/ports/local (more or less slave ports into other ports, but with more than just some options tweaked, e.g. more patches applied).
When some of these ports are installed in the system, and synth tries to check which ports to build for a  "prepare-system", it fails for those ports:
Scan of local/XXXX failed, it will not be considered (when the installed ports lists its origin as local/XXXX).
Unfortunately, there is no detailled description even in the log files WHY the scan "failed".
Both "make" in those subdirs, and a portupgrade works fine with these local ports.
Any idea?
Thanks!
Armin

did you define local as a valid category in the custom makefile?

also, you might need local/Makefile to be created where all the SUBDIRS are defined.

Yes, both "local" is defined as a valid category, and there is /usr/ports/local Makefile which lists the ports subdir. Unfortunately, synth keeps saying
"Scan of local/XXX failed, ...".
Some switch for more verbosity may help tracking down the problem.. I've tried to dig into the source, but I'm not much familary with ADA :-/

It's not a question of verbosity.  If synth knows the problem, it will say so.  It has a number of obvious things to check.
In this case, I think /usr/bin/make is returning an error after trying to read the file.
What you can do is this (picking a random port):
> env ENTERAFTER=extract synth devel/gmake
(wait until you get a prompt)
> cd /xports/local/<portname>
> make -V PKGNAME

You should see the error

Unfortunately, not:
# env ENTERAFTER=extract synth test local/tmda Scan of local/tmda failed, it will not be considered. There are no valid ports to build.  (Synth must exit)
I have no indication why "scan failed"..

Sorry for the confusion.
I just ran "synth" under "truss" and analyzed the output.
I'm making heavy use of ZFS filesystems here, and I've forgotten that /usr/ports/local is a separate filesystem. So nullfs-mounting /usr/ports does not pick up the sub-filesystem and thus, /usr/ports/local is not visible in the workers shadow filesystem mounts..
Never mind :)  I will have to find a way how I can fix this.

To follow up, you didn't follow my suggestion correct.
I didn't want you to "synth test local/tmda", I wanted "synth test devel/gmake" which is a known good port.  After the break, you cd to local/tmda.  devel/gmake was just a doorway to an interactive prompt.

(you would have then realized that /exports/local didn't exist)


OAuth2: don't use oauth when an empty token is set

resolves #190

Would it make sense to print something even if it's only an info or debug level log message to "warn" people. I see the value in making it optional but in all of my use cases across multiple orgs they would be needed and this does not give any feedback to the user that they are not properly configured. Also I would like to point out that while it may be true that its not needed in some small or edgecases an unauthenticated user request gets a max of 60 requests in an hour due to their rate limiting which for many use cases is not enough.

Could you please provide the output of the case, when the token is required? Which HTTP code does github return? I believe terraform should print the error message in case in the auth is required.
As for the 60 requests in an hour, is it true for GitHub Enterprise as well? Do you suggest to add a warning box in the documentation?

@kayrus the doc says (accurately) that they both are optional, in the sense that while they must be provided, but they can also be sourced from environment variables (GITHUB_TOKEN, GITHUB_ORGANIZATION).
Making them optional in the API allows me to use environment variables without complaint from my IDE.

Could you please provide the output of the case, when the token is required?

I have not done it recently but anytime you do anything to a private repo without auth you will get a 404 indicating it does not exist when really its lack of authentication. This is similar to how aws gives a 403 to people obscuring if its lack of permission or missing with s3.
You can just make an ENV var with the prefix of TF_ for those env vars and it works out of the box as that's how I am using it. The benefit of leaving them as "required" is that if the env var is not present it prompts for it. I do agree its technically optional but I think that there are far more use cases where its wanted needed and you can always create a token without any oauth scopes granted.
I am not entirely opposed to allowing it to be optional but I think given that it will more likely negatively impact use cases than it helps I think it minimally needs to be printed out to warn the user that no auth is detected.

I can't say for sure how it works for github enterprise as I dont currently have one to look at but it seems that it can be enabled for authenticated users: https://help.github.com/en/enterprise/2.16/admin/installation/configuring-rate-limits my gut feeling is that if github enterprise allows unauthenticated requests at all then its probably enabled. You could probably hit:
$ curl https://api.github.com/rate_limit
{
  "resources": {
    "core": {
      "limit": 60,
      "remaining": 60,
      "reset":<PHONE_NUMBER>
    },
    "search": {
      "limit": 10,
      "remaining": 10,
      "reset":<PHONE_NUMBER>
    },
    "graphql": {
      "limit": 0,
      "remaining": 0,
      "reset":<PHONE_NUMBER>
    },
    "integration_manifest": {
      "limit": 5000,
      "remaining": 5000,
      "reset":<PHONE_NUMBER>
    }
  },
  "rate": {
    "limit": 60,
    "remaining": 60,
    "reset":<PHONE_NUMBER>
  }
}

Replacing the hostname with your github enterprise url

Here is what I get with the GitHub enterprise.
$ curl https://github.local/api/v3/rate_limit
{
  "message": "Rate limiting is not enabled.",
  "documentation_url": "https://developer.github.com/enterprise/2.14/v3/rate_limit/#get-your-current-rate-limit-status"
}

I am not entirely opposed to allowing it to be optional but I think given that it will more likely negatively impact use cases than it helps I think it minimally needs to be printed out to warn the user that no auth is detected.

What about keeping the github/config.go changes only? When a user explicitly set these parameters to an empty string, the oauth won't be used.

That would work for me, allow people to opt into disabling authentication as its I think more common to need auth then not. If someone explicitly configures no auth and they need it that's on them.

We should probably update the docs to indicate that setting this to an empty string will disable oauth

Does it look fine?
Setting the `token` to an empty string will lead the GitHub provider to work in an anonymous mode with the corresponding API [rate limits](https://developer.github.com/v3/#rate-limiting).

@majormoses Marking them optional in the API allows me to use environment variables without complaint from my IDE. Otherwise, the IDE is prompting me to create some means other than the env vars for providing these (even though the env vars will still work). For the token, especially, the env var is the best option.
The provider and doc should be in agreement.

The provider and doc should be in agreement.

Terraform docs or Github's docs? Personally I think that its ok to put guardrails in place that are helpful to users even at the cost of not matching the default of the upstream but that is an opinion and neither are right or wrong. I think this should be up to the community consensus but I don't really like introducing breaking changes that will harm more users than it helps. With the current implementation setting it to an empty string as a default will work in your editor and you can always override it via an ENV var.

@majormoses How is downgrading from Required to Optional a breaking change?
Assigning the token to "" does not work as you describe. It does placate the IDE, but the empty string is not be replaced by GITHUB_TOKEN.

Are you using the required  TF_VAR_ prefix for terraform to see environment variables? You keep saying it without the prefixes and it's not clear if that's why its not working for you. With the new behavior you could configure something like this:
# configure the provider
provider "github" {
  version      = "1.3"
  token        = "${var.github_token}"
  organization = "${var.github_organization}"
}

variable "github_organization" {
  description = "The github org you want to manage"
  default     = "<REDACTED>"
}

variable "github_token" {
  description = "export TF_VAR_github_token=$MY_TOKEN"
  default = ""
}

$ export TF_VAR_github_token=''<REDACTED>"
# run some terraform command that requires auth and you will see it be set properly
# if you wish to see the current behavior with this setup and setting the env var to an empty string and
# setting a valid token in the default and you you will get a:
# * data.github_team.<REDACTED>: data.github_team.<REDACTED>: GET https://api.github.com/orgs/<REDACTED>/teams?per_page=10: 401 Bad credentials []

By still "requiring" it but discarding if it's empty seems the best compromise to me as it allows you to opt in of using it without auth but for the 99% of use cases will get prompted for credentials should they forget rather than getting a misleading error.
It's breaking in the sense of providing false information back to the user when their authentication is not set and instead tells them that the resource does not exist for example or other rate limit issues outlined without helpful feedback.

I'm using GITHUB_TOKEN as documented by the GitHub provider. If the token is absent then GITHUB_TOKEN is used. (In this sense, token is optional.)
GITHUB_TOKEN is also used in at least one place in the terraform AWS provider code, by the way.
Thanks for pointing out that I can bypass GITHUB_TOKEN by using a different environment variable. I think your suggestion works well in situations where the user may want to run without a token. This option is not available when using GITHUB_TOKEN and running without a token is not something I want to do. I am just annoyed that the IDE thinks token is required.

Thanks for pointing out that I can bypass GITHUB_TOKEN by using a different environment variable. I think your suggestion works well in situations where the user may want to run without a token. This option is not available when using GITHUB_TOKEN and running without a token is not something I want to do.

By creating your own variable you can choose exactly how you want to handle it, as I would not set an empty string (added that specifically to show its possible) it will require it to come from the ENV or a prompt otherwise it will fail. I think this would also satisfy your editors confusion but I can't say for certain.

GITHUB_TOKEN is also used in at least one place in the terraform AWS provider code, by the way.

Did not know that, from a "quick" search it is only used here other than tests and documentation for the referenced code. It does not use this provider at all, it just uses the same variable to make an auth request. I'd argue it should allowed it to come via a terraform config key or an env var and should not dictate that but that's IMHO out of scope of the discussion as it does not use the provider being referenced here. I think it's worth bringing that up on the aws provider to add flexibility.

I am just annoyed that the IDE thinks token is required.

What IDE? As I said I think your problem could be addressed with or without this change. I don't think that our motivation for making it optional should be because of an IDE.

@majormoses As far as I know the IDE is any IDE with intellisense, which will complain because token and organization are declared Required in this source file, which is different from how they are declared in this documentation, which I thought was the motivation for #190.

As far as I know the IDE is any IDE with intellisense

I don't see this issue with VScode which has intellisense, but then again as I am configuring it differently I am probably sidestepping your problem entirely.

which will complain because token and organization are declared Required in this source file, which is different from how they are declared in this documentation, which I thought was the motivation for #190.

The motivation for #190 from what I see is to allow you to use it without anything defined for a real use case such as querying a public team on a public github setup. That being said I don't really think there are many real  use cases where this actually makes sense as you will very likely hit the 60 requests an hour limit without specifying authentication. It's true that there is a docs mismatch and this happens to bring it into alignment but I see that as a symptom rather than the root of the issue IMHO.
I think this looks good in its current state after some updates to the docs to indicate that its required but could be an empty string which disables authentication and uses anonymous calls.

UPD: this is bizarre. Calling the https://api.github.com/users/user/keys is counted by the rate_limit, but https://github.com/user.keys - doesn't

I thought about this and I suspect the reason for this is that https:///api.github.com/users/majormoses/keys is an API request where https://github.com/majormoses.keys is made against the UI (which in turn makes the appropriate api requests) and therefore are not subject to the same API rate limits. As everything in this module is API driven I still think that a vast majority of use cases really will need authentication in order to be useful even if all the requests are read only to public entities. Just my $0.02.

@majormoses nevertheless there are cases when github enterprise is used.

That doc update works for me.

@majormoses kindly ping

@majormoses kindly ping

@kayrus I can give it a look over but I am not a maintainer for this project just someone who uses it heavily.

@majormoses there is no maintainer list in the README.md and I don't know who can merge it.

@radeksimko can you take a look?

@apparentlymart @aeschright  any chance to merge this?

Hi @kayrus !
Thank you for this PR!  There have been many requests for this, and I think there are some different ways we can go about solving it.  After talking to the team, we thought it might be nice to add an anonymous bool on the provider block and validate that it’s either one or the other.
I’ve done a similar thing with organization in order to make it optional, so you can take a look and maybe follow a similar format for this (https://github.com/terraform-providers/terraform-provider-github/pull/242). Or if you don't have the time or inclination to make these changes, let me know, and we can try and pick up the changes ourselves so we can get this merged.
Let me know if you have questions or feedback, or if I can help with anything further.
Thanks!
Megan

@megan07 acked.

@megan07 kindly ping

Hi @kayrus !
This looks good!  Would you be able to add a few acceptance tests, please?  Here are a few things it might be nice to test:
• token and anonymous not set
• no token and anonymous set
• token and anonymous not set
• no token and anonymous not set
Please let me know if you don’t have time to do these tests and I would be happy to write them.
Thanks!

@megan07 feel free to add tests

Closing in favor of #255.


Support docker build -f

In my projects I have several Dockerfiles for only one image in order to create images from diferents OS, so i don`t have  a Dockerfile file, I have Dockerfile.Centos, Dockerfile.debian, Dockerfile.rhel...
I would need docker-api support "docker build --filename". Maybe this new feature could be included in Image.build_from_dir or in a new function Image.build_from_file.
If you believe that this new feature is interesting I have no problems in making a pull request. If you think that it is better to you implement it, I have no problems

@julianfernandez Can you try Docker::Image.build_from_dir('/path/to/dir', { 'dockerfile' => 'Dockerfile.Centos' })? This should work.

Perfect, it works. Many thanks @tlunter

I suggest to put it in the README

Added in https://github.com/swipely/docker-api/commit/61a4497a3af94d199b4c679aa13e56361acf1851


Triple added by IndirectContainer has object of the created resource, not the one pointed to by insertedContentRelation

Example: https://gist.github.com/cbeer/556b8877c0cefce6473f


bug: 'join-vertical md:join-horizontal' failed to set horizontal style border on nest join-item

What version of daisyUI are you using?
No response
Which browsers are you seeing the problem on?
Firefox
Reproduction URL
https://play.tailwindcss.com/X7oydx51nb
Describe your issue
border of nest join-item doesn't change into horizontal style in md breakpoint.

  <button class="btn join-item">111</button>

    <button class="btn join-item w-full bg-slate-200">222</button>

The join-item class must be used directly under the join class.
Example code:

  <button class="flex-none btn join-item">111</button>
  <button class="flex-1 btn join-item bg-slate-200">222</button>

The join-item class must be used directly under the join class.

That's not correct: Even if join-item is not a direct child of the group, it still gets the style. So it might be a bug.
I have to wrap the button in div for some reason and I solved it as follows:

  <button class="btn join-item">111</button>

    <button class="btn w-full bg-slate-200 rounded-[inherit]">222</button>


Arcyd: restore repo retry delays on success

If a repository has been temporarily disabled due to a failure and
subsequently succeeds on a retry, restore it's retry timers to the
original state.
This means that for an Arcyd instance that is running for weeks,
intermittent failures of a particular repository spaced out by days will
not cause the repository to fail. Instead it's 'lives' will be reset on
the next successful processing.
Test Plan:
$ ./precommit.sh
.. set retry delays to '1 sec, 10 secs, 1 min' for convenience ..
$ ./testbed/arcyd/test_shell.sh
    $ tail -f arcyd/var/log/info &
    .. 'arcyd started' streams to console ..
    $ mv origin2/ _origin2/ && read && mv _origin2/ origin2
    .. wait for repository to be disabled for the 1 sec delay ..
    .. wait for repository to be disabled for the 10 secs delay ..
    .. hit enter, to restore the dir ..
    .. observe that repo is re-enabled ..
    $ mv origin2/ _origin2/ && read && mv _origin2/ origin2
    .. observe repository is disabled for the 1 sec delay again ..

LGTM

@aevri  Can we added lgtm.co here by any chance?

Thanks!
Maybe, if our rate of PRs gets high enough :)


Fix systemd_timer_resource :remove action failures

Get the underlying action right
Signed-off-by: Tim Smith<EMAIL_ADDRESS>

This is going to fail in CI since we're actually running the chef 16 resource which has the same problem. It'll resolve once we release a new client there.


Why are vendor-prefixes mixins still used if depreciated and autoprefixer is requirement?

Documentation states that autoprefixer is required however shouldn't that replace use of vendor prefixes mixins? I thought I could simply remove that import from my _mixins.scss file but other components are still using those mixins.

They are used for the sake of backward-compatibility since the Autoprefixer requirement was added midway into Bootstrap v3's life. Didn't want to cause extra breakage for folks who don't have Autoprefixer in their build.
They have been fully eliminated in v4.


Improve audit logging performance

Instead of calling fmt.Sprintf with a large number of format arguments, we use strings.Join.
In addition to being 4X faster, I believe it is also more readable, especially as we add new fields to log.

Benchmark was run with go test -v -bench=BenchmarkLogDedupPacketAllow -run=XXX -benchtime=10s.
Results (old implementation):
goos: linux
goarch: amd64
pkg: antrea.io/antrea/pkg/agent/controller/networkpolicy
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkLogDedupPacketAllow
BenchmarkLogDedupPacketAllow-4   	16072234	       732.2 ns/op
PASS
ok  	antrea.io/antrea/pkg/agent/controller/networkpolicy	12.579s

Results (new implementation):
goos: linux
goarch: amd64
pkg: antrea.io/antrea/pkg/agent/controller/networkpolicy
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkLogDedupPacketAllow
BenchmarkLogDedupPacketAllow-4   	66314331	       182.1 ns/op
PASS
ok  	antrea.io/antrea/pkg/agent/controller/networkpolicy	12.309s

In addition to strings.Join, I have also tried strings.Builder. The strings.Join implementation is a bit faster (10%), and more readable.

/test-e2e

/test-all


TypeError: descriptor 'split' requires a 'unicode' object but received a 'str'

from indic_transliteration import xsanscript
Traceback (most recent call last):
File "", line 1, in
File "/home/pranav/.local/lib/python2.7/site-packages/indic_transliteration/xsanscript.py", line 122, in
_setup()
File "/home/pranav/.local/lib/python2.7/site-packages/indic_transliteration/xsanscript.py", line 74, in _setup
'vowels': sanscript.SCHEMES[DEVANAGARI]['vowels'] + s("""ऎ ऒ"""),
TypeError: descriptor 'split' requires a 'unicode' object but received a 'str'

I am getting this error when importing xsanscript. Kindly help me with the issue

had got an error with python2.X and python 3.X


The order index of new node of Editable-Children-enaled scene is not saved in .tscn file sometimes

Tested versions

Reproducible in v4.4.dev4.official [36e6207bb]

System information
Windows 11 - Godot Engine v4.4.dev4.official.36e6207bb - Vulkan 1.3.280 - Forward+ - Using Device #0: NVIDIA - NVIDIA GeForce RTX 3060 Ti
Issue description

When I add a new node Node2D_xxxx between Node2D3 and Node2D4 and save, the order index is not saved in main.tscn file so that when I re-open main.tscn, the Node2D_xxxx came after Node2D4.
Meanwhile, the Node2D_yyyy's order is saved correctly. So I guess there is a bug to cause such different behavior.
Steps to reproduce

Open MRP and open main.tscn
drag Node2D_xxxx between Node2D3 and Node2D4 and save。
re-open main.tscn and see the position of Node2D_xxxx

Minimal reproduction project (MRP)
test_child_order.zip

Can confirm

I've written up a fix for this, and will try open a PR for it later today/tomorrow when I get a spare moment. The specific condition that triggers this is when a local node is a direct child of a saved scene that is marked as "Editable Children", it's index is erroneously not stored.


Changes to get Triton driver accepted

Hi @edumur
Please confirm merge for this PR. It will help get the new Triton driver accepted into Qcodes_contrib_drivers repository.
Also, please always make a new branch from your master branch and make changes to the new branch. After committing your work on new branch you should make a PR from this branch and not the master branch of your repository. This will help us to better support you in contributions to the repository.
Thanks and Regards,
Akshita

Done.
Thanks for your remark on the branches, I will do ;) .


[cow2] Document cowbell strategy

Document that we want more cowbell to make things better.

we have enough cow for now, but thanks!


 ELI5

I cant find the default folder and I cant change the folder? Im windows 10 and using SinusBot

Looks like you're using a different program than youtube-dl. Please ask for SinusBot's developers for help.


Cleanup: Rename make_..._mask functions to make_..._biases

Clean up a TODO item as the term biases is more specific than mask.

Thanks!
Do you have an internal repo PR that bumps the axlearn version and passes the tests?

Yes! PR #14783


Hello , Can LMFlow support Qwen1.5-1.8B model Fine-tuning?

Hello , Can LMFlow support Qwen1.5-1.8B model Fine-tuning?

Thanks for your interest in LMFlow! We are integrating that feature right now, hopefully supporting it in 1-2 days. Please stay tuned for our latest update 😄

Hello , Can LMFlow support Qwen1.5-1.8B model Fine-tuning?

Hi, we've tested on Qwen1.5-1.8B and the script works fine.

Please make sure you include --lora_target_modules q_proj, v_proj in the finetune shell script:

Also, we strongly recommend you to:

Use a conversation dataset to finetune the model. You could either:

To test the workflow, try to download a conversation dataset from out data server via:cd data && ./download.sh alpaca && cd -

and specify the dataset to data/alpaca/train_conversation, or
Prepare your own conversation dataset (see here)

Specify the conversation template to qwen2 for better performance.


1.6 New PackageDeploymentManager methods not tagged with new v2 contract

cherry-pick New PackageDeploymentManager methods not tagged with new v2 contract #4663
https://github.com/microsoft/WindowsAppSDK/pull/4663

This is targeting main and thus bringing in 42 commits. It needs to target release/1.6-stable

Yeah. I created the PR last night and decided merge conflicts were better off for the morning and of course coffee and 3s was all it took to realize why ;P

/azp run

This is targeting main and thus bringing in 42 commits. It needs to target release/1.6-stable

Handled; see other comment. But Github's not giving me any "Resolve" button for this comment :-( At least I can tap 'Dismiss Review'.


Option to assume null for unset cells

Hello @nietras and thanks for this library!
Would it be possible to have an option for SepWriter to assume that any non-written fields are null/empty?
I have a scenario in which the CSV data contains lists like this (simplified example)

In order to write CSV like this, I have to call Set(null) on the empty cells:
for (int i = 0; i < rowCount; i++)
{
    using SepWriter.Row row = csvWriter.NewRow();

    if (i == 0)
    {
        row[0].Set(dto.Project);
        row[1].Set(dto.Author);
    }
    else
    {
        // Would like to get rid of this
        row[0].Set(null);
        row[1].Set(null);
    }

    if (i < contributorCount)
    {
        var contributor = dto.contributors[i];
        row[2].Set(contributor.Name);
    }
}

I could of course also just add this to the beginning of the new row:
for (int c = 0; c < 2; c++)
{
    row[c].Set(null);
}

but the cleanest option is to let Sep handle this...no? 🙂

Hi, thanks for using Sep and suggesting this, however, this is the kind of thing that I think is counter to the philosophy of Sep and not something I would want to support directly. Sep is minimal with focus on being straightforward and efficient and not about reducing "your" code to minimal possible ;)
Not the least this code can be written straightforwardly and simply by:
for (int i = 0; i < rowCount; i++)
{
    using var row = csvWriter.NewRow();
    row[0].Set(i == 0 ? dto.Project : string.Empty);
    row[1].Set(i == 0 ? dto.Author : string.Empty);
    row[2].Set(i < contributorCount ? dto.contributors[i].Name : string.Empty);
}

or similar e.g. if dto nullable then dto?.Project ?? string.Empty.
Hope that is clear :)

I understand, I just thought maybe it wouldn't be too complicated to not throw an exception if some cells were not set 😄
Your example is simpler but in my case it's more complicated and would result in very unreadable code.
In any case no worries, I've got it working with this simple loop:
// Set null everywhere initially just to be safe from expected column errors
for (int colIndex = 0; colIndex < colCount; colIndex++)
{
    row[colIndex].Set(null);
}


Vue is not defined.

Your userscript no longer works.

It still works, it is based on VueJS 2, just install the script and replace:
// @require      https://vuejs.org/js/vue.min.js
with
// @require<EMAIL_ADDRESS>(development version, includes helpful console warnings)
or
// @require<EMAIL_ADDRESS>(production version, optimized for size and speed)
Save, reload and enjoy ;)


[Snyk] Upgrade js-base64 from 3.7.6 to 3.7.7

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade js-base64 from 3.7.6 to 3.7.7.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 1 version ahead of your current version.
The recommended version was released 21 days ago, on 2024-02-21.

Release notes

Package name: js-base64

3.7.7 - 2024-02-21version 3.7.7

3.7.6 - 2024-01-20version 3.7.6

    from js-base64 GitHub release notes

Commit messages

Package name: js-base64

3d50276 version 3.7.7
ae00bf1 fix github actions
3fab4cd Revert "remove node 14 and 16 as they have reached EOL"
7c508e3 update & simply steps
26fdea6 remove node 14 and 16 as they have reached EOL
90989b4 rebuild base64.js
c19b960 Merge pull request #174 from Itaypa/bugfix/static-atob-check-does-not-work-in-web-worker
43f4a49 Caching the result of atob is causing webworker to crash

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Done in #17


Use Replication Protocol V3

What
Improve replication performance by being able to identify the last sequence at which CDTDatastore replicated to the remote.
How

Change the checkpoint document to match the format of V3 for the replication protocol.
If the last seq for local and remote don't match for the checkpoint, back track over the history object to identify a seq where they do match.

Issues
Part of #288

The copyrights need updating

@ricellis done.

I'm ok with this now, but it will need a second reviewer.

Will merge once we have the tests running again.

Rebased with latest changes which fix the build with cocoapods 1.1.1

@ricellis @emlaver
Tests are passing since the rebase, are you both still happy I merge this?

yes

since @emlaver is on ops, going to assume its still fine.


feat: add example for OpenAPI and mocking

Please add the example to the index on the readme.

Please add the example to the index on the readme.

I had totally missed that list. Adding it, thanks!


Stage3D creates Context3D even when it's not requested

Not only this create unnecessary context even when it's not needed, but also causes rendering issues on Starling v2.0.
Starling misunderstands that context is shared because Stage3D already has Context3D instance, and does not render anything. :worried:
https://github.com/openfl/openfl/blob/cfe82edd6f10575fee5d58d4040baabd25a4e31e/openfl/_internal/renderer/opengl/GLRenderer.hx#L62-L66
Looking at your code in openfl, there seems to be many piece of code that does the almost same thing.
https://github.com/openfl/openfl/blob/9b27b24317808d470ee6f0e1923575b3e2957806/openfl/display/Stage3D.hx#L220-L224
https://github.com/openfl/openfl/blob/9b27b24317808d470ee6f0e1923575b3e2957806/openfl/display/Stage3D.hx#L168-L173
Have you ever heard of a principle named "DRY(Don't repeat yourself)"?
It should be possible to reduce duplicates like that, instead of writing everything twice(WET). It's not easy for other people to understand WET code, and of course bad for maintainability. I'm not certain how many places I should modify to make some changes to WET code, because code is just copied and pasted everywhere...
I have hard time understanding your code... Why not just dispatch CONTEXT3D_CREATE event when context is created?? It's a lot easier.
https://github.com/openfl/openfl/blob/9b27b24317808d470ee6f0e1923575b3e2957806/openfl/display/Stage3D.hx#L154-L159
I'm not sure why Context3D needs RenderSession, because it looks unused.

I understand that the code seems to go through possibly unneeded steps.
When you requestContext3D in Flash, it does not dispatch immediately. If you request a context repeatedly, it will dispatch CONTEXT3D_CREATE events, so long as it has not dispatched the event yet. Request twice in a row, you will get one event, but request after the event, and you will get another. This is the behavior that the dispatch code in Stage3D is meant to emulate. In my tests, it behaves consistently with Flash Player, with one noted exception.
Flash Player has no hardware render context by default, until you requestContext3D. OpenFL does on desktop, mobile or WebGL, and optionally (but not by default) has a context on HTML5 -Ddom. Unlike Flash Player, we do have a context initialized. Like it or not, Stage3D on OpenFL GL targets are always using a shared context. Cooperation between different renderers (such as the OpenFL core renderer, Starling or other libraries) will need to improve more over time.

Is it really impossible if Context3D is not initialized instantly??
Let me have a look at those code. Also I'd like to find a solution without too much duplicates.

I think that just creating Context3D after 1ms delay works:

Don't schedule anything if one is pending
After delay, check if current renderer supports Stage3D. if not, dispatch error event.
Create Context3D if one is not there
Dispatch success event

That way you don't have to insert __createContext everywhere.
According to API reference, it throws exceptions when you request another context with different profile, if one is pending.

Making it available instantly allows for deeper integration between Stage3D, the core OpenFL renderer, and new features such as render-to-texture inside OpenFL
I'm not sure why it helps, you still have to deal with lots of state changes OpenFL renderer makes. It still gets broken regardless of when it's created, and we should find real solution instead of adding lots of workarounds, which may introduce other bugs...

I was thinking of creating OpenGL state wrapper that is shared by all OpenFL classes, which reverts all state changes at the beginning of rendering. You just have to make sure that you make state changes via this class. That should work regardless of Context3D creation timing, as long as you save/restore all states.

I've found that you are allocating Stage3D textures in GLRenderer, but why not allocating that by yourself? Also, this depends on single context setting. Does not look future-proof, I'm not sure what happens if you dispose Context3D or if you have multiple Stage3Ds.
The name "BitmapData.fromTexture()" might be a little confusing. It's not obvious whether it accepts GLTexture or openfl.display3D.textures.Texture.

Not sure why you want to create Stage3D textures inside GLRenderer. Maybe you just wanted to save rendered result to texture. Does anybody want that from Stage3D? Or it just doesn't have to be Stage3D, and you just tried to save yourself from some typing?
As I said before, rendering to texture is an expensive operation. Try not to use that when possible.
Starling does not do that for caching rendered results because it's expensive.
https://github.com/openfl/openfl/blob/cfe82edd6f10575fee5d58d4040baabd25a4e31e/openfl/_internal/renderer/opengl/GLRenderer.hx#L220-L250
There doesn't seem to be a way to disable this feature. Is everybody forced to use cached texture?
I just want to be able to use Stage3D without hidden implemantation difference like this.

Does anyone else have an opinion on this?

We can continue discussing this on Slack or the forums. This is not a bug, so I'm taking it down from Github Issues. Thank you so much for caring so much about this, I hope that we can find the best solution long-term :grin:


2.1.2 python 2.7 UnicodeDecodeError when loading docstrings

I'm running into the following error when trying to load libvips from python 2.7 on macOS:
import logging
logging.basicConfig(level = logging.DEBUG)
import pyvips

DEBUG:pyvips:Binary module load failed: No module named _libvips
DEBUG:pyvips:Falling back to ABI mode
DEBUG:pyvips:Loaded lib <cffi.api.FFILibrary_libvips.42.dylib object at 0x109023e10>
DEBUG:pyvips:Loaded lib <cffi.api.FFILibrary_libgobject-2.0.dylib object at 0x109023d90>
DEBUG:pyvips:Inited libvips
DEBUG:pyvips:
Traceback (most recent call last):
  File "vips-pngquant.py", line 4, in <module>
    import pyvips
  File "/usr/local/lib/python2.7/site-packages/pyvips/__init__.py", line 157, in <module>
    from .vimage import *
  File "/usr/local/lib/python2.7/site-packages/pyvips/vimage.py", line 134, in <module>
    class Image(pyvips.VipsObject):
  File "/usr/local/lib/python2.7/site-packages/pyvips/vimage.py", line 1174, in Image
    @_add_doc('ifthenelse')
  File "/usr/local/lib/python2.7/site-packages/pyvips/vimage.py", line 101, in _add_doc
    docstring = pyvips.Operation.generate_docstring(name)
  File "/usr/local/lib/python2.7/site-packages/pyvips/voperation.py", line 310, in generate_docstring
    result += argstr(name)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 25: ordinal not in range(128)

Also it's not using the binary module, even though pyvips was freshly installed with --no-cache-dir.
Using python 3 this does not occur and it's using the binary module.
Environment:

pyvips 2.1.2
cffi 1.11.5
pycparser 2.18
python 2.7.15 (homebrew python@2)
libvips 8.6.3 (homebrew)
macOS 10.13.4

Huh strange. I'll have a look.

It seems to be working for me:
john@katamata:~/try$ ./test.py
DEBUG:pyvips:Loaded binary module _libvips
DEBUG:pyvips:Inited libvips
DEBUG:pyvips:
john@katamata:~/try$ pip list
Package    Version
---------- -------
cffi       1.11.5
pip        10.0.1
pkgconfig  1.3.1
pycparser  2.18
pyvips     2.1.2
setuptools 39.1.0
wheel      0.31.0
john@katamata:~/try$ which python
/usr/local/bin/python
john@katamata:~/try$ which pip
/usr/local/bin/pip
john@katamata:~/try$ vips --version
vips-8.6.3-Thu Mar  8 15:18:35 UTC 2018

If I disable the binary module, it seems to fall back correctly too:
john@katamata:~/try$ mv /usr/local/lib/python2.7/site-packages/_libvips.so x
john@katamata:~/try$ ./test.py
DEBUG:pyvips:Binary module load failed: No module named _libvips
DEBUG:pyvips:Falling back to ABI mode
DEBUG:pyvips:Loaded lib <cffi.api.FFILibrary_libvips.42.dylib object at 0x10fbc8110>
DEBUG:pyvips:Loaded lib <cffi.api.FFILibrary_libgobject-2.0.dylib object at 0x10fbc8090>
DEBUG:pyvips:Inited libvips
DEBUG:pyvips:

My test program was just:
#!/usr/bin/env python

import logging
logging.basicConfig(level = logging.DEBUG)
import pyvips

if it makes any difference.

It appears to depend on the locale:

This fails: LANG=de_DE.UTF-8 python2 -c 'from pyvips import Image'
This works: LANG=C python2 -c 'from pyvips import Image'

I was just about to ask about your locale! Yes, I think you're getting a German translation in utf-8 and pyvips is not converting to ascii. I'll have a look at that.

I think git master pyviups may have a fix, could you test it?
It was not decoding strings in py2, and .format in py2 seems to need a unicode literal.
https://github.com/jcupitt/pyvips/commit/ac33aa70a04259f533beb0e11fed85cc55ac0171

It works on ac33aa70a04259f533beb0e11fed85cc55ac0171 but is broken again on b33ef40c4ceaad5a6ce55a7064f3a90ac9311ead.

Looks like there's a filename encoding issue on macOS as well.

Looks like there's a filename encoding issue on macOS as well.

AFAIK HFS+ uses unicode decomposition using NFD, so characters like ü are stored as u ̈, I don't know if the same applies for APFS. It can certainly cause trouble when comparing filenames to utf8 strings that have not been decomposed. This can usually be solved by converting them to the same normalization form.

This is strange, the following for me still breaks:
LANG=de_DE.UTF-8 python2 -c 'from pyvips import Image'

But this works fine on master:
LANG=de_DE.UTF-8 python2
>>> from pyvips import Image

The first example also works if I modify /usr/local/lib/python2.7/site-packages/sitecustomize.py and add sys.setdefaultencoding('UTF8') to change the default encoding for python2.7 from 'ascii' to 'utf8'.

@jcupitt The following patch on top of master works for me and the tests are green on 2.7 and 3.6:
diff --git a/pyvips/vobject.py b/pyvips/vobject.py
index 5fa2109..e407267 100644
--- a/pyvips/vobject.py
+++ b/pyvips/vobject.py
@@ -6,7 +6,7 @@ import gc
 import logging

 import pyvips
-from pyvips import ffi, vips_lib, gobject_lib, Error, _to_bytes, _to_string
+from pyvips import ffi, vips_lib, gobject_lib, Error, _to_bytes

 logger = logging.getLogger(__name__)

@@ -71,7 +71,7 @@ class VipsObject(pyvips.GObject):
         """Get the blurb for a GObject property."""

         c_str = gobject_lib.g_param_spec_get_blurb(self._get_pspec(name))
-        return _to_string(ffi.string(c_str))
+        return _to_bytes(ffi.string(c_str)).decode('utf8')

     def get(self, name):
         """Get a GObject property.
@@ -131,7 +131,7 @@ class VipsObject(pyvips.GObject):
         """Get the description of a GObject."""

         vo = ffi.cast('VipsObject *', self.pointer)
-        return _to_string(ffi.string(vips_lib.vips_object_get_description(vo)))
+        return _to_bytes(ffi.string(vips_lib.vips_object_get_description(vo))).decode('utf8')

 __all__ = ['VipsObject']


[Question]: Why we are not using Helper.switchCamera from flutter webrtc.

In the current implementation, we restart the track when we want to switch the camera (front/back). Is there any particular reason we are not using the static helper method Helper.switchCamera from flutter_webrtc?

@cloudwebrtc any idea?

yeah, I think it's worth considering, Helper.switchCamera will switch much faster on mobile devices

@cloudwebrtc shouldn't the above PR also update the capture options to notify the clients that a deviceId or camera position has been changed?

hi @xsahil03x ,Do you mean like that?
     if (fastSwitch) {
+       currentOptions = options. copyWith(deviceId: deviceId);
       await rtc.Helper.switchCamera(mediaStreamTrack, deviceId, mediaStream);
       return;
     }

@cloudwebrtc yes, something like this. But I'm confused about how to determine the new cameraPosition in case the switchCamera is called on web with deviceId.

Here is an example getUserMedia, to detect whether the web supports facingMode. If it supports, cameraPosition can be valid. If not, you need to enumerate mediaDevices and pass in deviceId to switch video source

@cloudwebrtc So, the flow should be something like this
FastSwitch ->
		if (web) ->
			if (facingModeSupported) -> setFacingMode
			else -> pass the deviceId

		else -> Native Call

Also, do you think we should use the Helper.setMicrophoneMute similarly for the microphone tracks instead of restarting them?


[Denovo] TOBE architecture and Kubernetes cluster sizing

The document describing migration in detail has been written and is now available internally.


Generate multiline assertion error message for better readbility

Instead of:
"Expected character's name to be <Sam> but was <Sammy>"

we would have:
"Expected character's name to be:
   <Sam>
but was:
  <Sammy>"

Done


[devicelab] remove twc enabled test and uncaught image test

Description
These are correctness tests which do not really need physical devices. Instead this behavior is covered by unit tests in the flutter_tools directory, and any additional tests should be added there.
The uncaught image error test also is tested in the framework - this was added by me for completeness but is ultimately unecessary.

Landing, since analysis has passed and this should reduce device-lab load more