Alpha seed implementation (New Feature)

Intro
We are looking for a savvy and experienced front-end Typescript developer to tackle this funded issue.
Please only apply if you believe you can provide a high-quality delivery that satisfies all the requirements outlined in this issue. Feel free to ask any questions about this issue in the conversation module on https://github.com/spacemeshos/cosmic/issues/2 - thanks for your contribution!
Story
App developers who need to ship their apps across all major platforms will use this seed as the base of their app's codebase because it will save them lots of time and effort writing boilerplate code to support these platforms with one codebase without having any duplicated app logic and UI code.
Requirements

Implement the seed using the architecture and frameworks described in the readme.
Please review the readme.md for an architectural overview of the different app components and modules and design the folder and project structure so it corresponds to the conceptual architecture modules
The seed should be buildable and runnable on all platforms - iOS, Android, Web and Desktop via Electron.
Please review the relevant projects in the wiki. For example - this is a working barebones seed for react-native-web typescript
We are expecting clean, well documented and polished Typescript code.

Demo App

Running the seed should launch a simple demo app
Demo App should have 2 screens with content

Use case 1 - Basic navigation - Clicking a button on the first screen should access the second screen. Clicking the button on the second screen should display the first screen
User case 2 - User Interaction - The first screen should also include a text input box accepting user provided text and a submit button . Submitting the text should display it in a label below the text box. All this functionality should be implemented using Redux for state modifications and UI updates based on state changes

Testing

Use Jest testing suite
There should be unit tests and integration tests for all implemented functionality
There must be a test that validates proper demo app startup on each supported platform. e.g display of first content screen without any errors

Documentation & Coding Style

All methods, models, types and interfaces must be documented via short and simple text comments
Write basic docs in readme.md that include instructions for setting up the dev env, building, testing and running the seed on all targets
Integrate tslint into the build workflow
All Typescript code must pass tslint

Guidelines

Use create-react-app-typescript and Eject the build scripts so they can be tweaked by hand and we can modify babel, etc...
Each module's files should be in a module specific folder and the overall folder structure should be well organized
Only use .js files when absolutely required a used framework - Typescript everywhere else
Please use yarn and not npm
PRs for this issue will be code reviewed and will only be accepted if all of the requirements outlined here are satisfied
Use flexbox layout for all react-native views and ui widgets

Developer Requirements

You must be an experienced Typescript and node.js developer with some experience with one or more of these frameworks: react, redux, react-native, react-native-web, react router.

Issue checklist

[x] All required functionality implemented
[ ] Demo app running on iOS device
[ ] Demo app running on Android device
[ ] Demo app running in web browser
[ ] Demo app running in an Electron app
[ ] Demo app navigation and use case works on all platforms
[ ] Build, test and run docs in readme.md
[ ] Well organized project folder structure that corresponds to the architecture
[ ] All methods, types, interfaces and modules are well comment documented
[ ] Typescript is used when possible instead of Javascript
[ ] Yarn build, test and run scripts work for all platforms
[ ] Full unit tests
[ ] Demo app tests for each supported platform
[ ] All Views and UI widgets are layed-out using flexbox.
[ ] Your name added to authors

Thinking about starting to work on this issue?  Please chat with us here https://gitter.im/spacemesh-os/cosmic

@avive this task is cool, am joining the gitter immediately

I think that this is the current status:

[+] All required functionality implemented
[+] Demo app running on iOS device
[ ] Demo app running on Android device
[+]  Demo app running in web browser
[+] Demo app running in an Electron app
[+] Demo app navigation and use case works on all platforms
[ ]  Build, test and run docs in readme.md
[ ]  Well organized project folder structure that corresponds to the architecture
[ ] All methods, types, interfaces and modules are well comment documented
[+] Typescript is used when possible instead of Javascript
[+] Yarn build, test and run scripts work for all platforms
[ ] Full unit tests
[ ] Demo app tests for each supported platform
[ ] All Views and UI widgets are layed-out using flexbox.
[ ] Your name added to authors

At the moment,
[x] Full unit tests
[x] Build, test, and run docs in readme.md,
[x] Demo app running on Android device

Looks like we are getting close.. updated the checklist above.

Let me know where can I collaborate and contribute.

@vikaskyadav You can work on this issue to familiarize yourself with the codebase  https://github.com/spacemeshos/cosmic/issues/7 - we are going to have additional funded issues soon as well.

sounds good!


adopt turbo

With turbo, the unit tests are running faster (mostly because they get to be parallelized, I guess?). It should be possible to split the test suite into 5 or 10 buckets and get more speed.
ref. https://turbo.build/repo/docs/reference/configuration
It also seems to fix the pesky terminal bug!
todo:

[ ] split tests into 4 or 8 buckets?
[x] fix coverage (reported as 0%)

using concurrently instead
superseded by #945


set values to role ID's in resourceCloudflareAccountMemberRead

Cloudflare provider is not able to give diff for the changes made via UI in role ID's. Fixing it by setting values to role ids in resourceCloudflareAccountMemberRead.

thanks for raising this PR! 🍰  before we land this, we need a couple of things:

CI to be all green
added test coverage for this change
a CHANGELOG entry (details on how to do this can be found at in changelog process)

Hi @jacobbednarz, any extra work need in the PR?

yes, this is still missing test coverage for the change

No file named resource_account_member_test. Where should I add test cases?

create one :) there are plenty of examples in the repository to use as
starting points.

we're green with the acceptance tests
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test $(go list ./...) -v -run "^TestAccCloudflareAccountMemberBasic" -count 1 -parallel 1 -timeout 120m -parallel 1
?   	github.com/cloudflare/terraform-provider-cloudflare	[no test files]
=== RUN   TestAccCloudflareAccountMemberBasic
--- PASS: TestAccCloudflareAccountMemberBasic (10.15s)
PASS
ok  	github.com/cloudflare/terraform-provider-cloudflare/cloudflare	10.648s
?   	github.com/cloudflare/terraform-provider-cloudflare/version	[no test files]

thank you for this work!


pyvespa: connection reset not handled

Feed problems using pyvespa - per @jobergum:

You are getting a connection reset which it looks like the underlaying http client used by pyvespa does not handle or retry:/ it’s expected that Vespa will recycle connections so that is really unfortunate.

Successful documents fed: 1000/1000.
Batch progress: 1/22.
Successful documents fed: 1000/1000.
Batch progress: 2/22.
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-4880' coro=<VespaAsync._feed_data_point_semaphore() done, defined at /home/jrs/anaconda3/lib/python3.9/site-packages/vespa/application.py:1090> exception=RetryError(<Future at 0x7fa286cac550 state=finished raised ClientOSError>)>
Traceback (most recent call last):
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 581, in write_bytes
    await self.body.write(writer)
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/aiohttp/payload.py", line 247, in write
    await writer.write(self._value)
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/aiohttp/http_writer.py", line 115, in write
    self._write(chunk)
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/aiohttp/http_writer.py", line 75, in _write
    raise ConnectionResetError("Cannot write to closing transport")
ConnectionResetError: Cannot write to closing transport

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/tenacity/_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/vespa/application.py", line 1082, in feed_data_point
    response = await self.aiohttp_session.post(end_point, json=vespa_format)
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/aiohttp/client.py", line 560, in _request
    await resp.start(conn)
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 899, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
  File "/home/jrs/anaconda3/lib/python3.9/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
aiohttp.client_exceptions.ClientOSError: [Errno None] Can not write request body for http://localhost:8080/document/v1/ZZZ/YYY/docid/1234

The following will reproduce the problem:
from vespa.package import ApplicationPackage, Field, FieldSet, RankProfile
from vespa.deployment import VespaDocker
from pandas import read_csv

app_package = ApplicationPackage(name="textsearch")

app_package.schema.add_fields(
    Field(name="id",    type="string", indexing=["attribute", "summary"]),
    Field(name="title", type="string", indexing=["index", "summary"], index="enable-bm25"),
    Field(name="body",  type="string", indexing=["index", "summary"], index="enable-bm25")
)

app_package.schema.add_field_set(
    FieldSet(name="default", fields=["title", "body"])
)

app_package.schema.add_rank_profile(
    RankProfile(name="bm25", first_phase="bm25(title) + bm25(body)")
)
app_package.schema.add_rank_profile(
    RankProfile(name="native_rank", first_phase="nativeRank(title, body)")
)

vespa_docker = VespaDocker()
app = vespa_docker.deploy(application_package=app_package)

docs = read_csv(
    filepath_or_buffer="https://data.vespa.oath.cloud/blog/msmarco/sample_docs.csv"
).fillna('')
docs.head()

for x in range(50):
    print("iteration: " + str(x))
    feed_res = app.feed_df(docs, asynchronous=True, batch_size=1000)

Batch progress: 7/10.
Successful documents fed: 1000/1000.
Batch progress: 8/10.
Successful documents fed: 1000/1000.
Batch progress: 9/10.
Successful documents fed: 963/963.
Batch progress: 10/10.
iteration: 28
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-615205' coro=<VespaAsync._feed_data_point_semaphore() done, defined at /Users/kraune/Library/Python/3.11/lib/python/site-packages/vespa/application.py:1092> exception=RetryError(<Future at 0x12a96ce50 state=finished raised TimeoutError>)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/tenacity/_asyncio.py", line 50, in __call__
    result = await fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kraune/Library/Python/3.11/lib/python/site-packages/vespa/application.py", line 1084, in feed_data_point
    response = await self.aiohttp_session.post(end_point, json=vespa_format)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 560, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 894, in start
    with self._timer:
  File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 721, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

The above exception was the direct cause of the following exception:

It seems to me that the async calls push enough load for items to time out eventually, and the workaround is to use asynchronous=False in the calls for now

Considered fixed with the above PR. Reopen if issue persists.


Add pkl to GitHub linguist

GitHub Linguist can detect the language of a file based on different methods.
It is related to syntax highlighting on GitHub and to generate language statistics inside GitHub repos:

I think it's a problem, that there already is another format called pickle with the file extension .pkl. Its Python related.
Would be great to add pkl grammar to linguist as it improves code readability on GitHub.
Linguist has quantitative requirements of how many public files of a specific language are hosted on GitHub, before they consider adding it, which might have not been reached yet.

It's in the works!

Support for Pkl in Linguist is being added by this PR. We just noticed this issue was open now so I'm adding a reference here for tracking.

Pkl support landed in linguist with https://github.com/github-linguist/linguist/pull/6730 and should be broadly available on github.com now.

Closing this one as it's now completed.


Remove commented-out dependency in gemspec. Add jetbrains IDE files to gitignore

I don't know whether these changes are useful to you but they were useful to me. :)

I am happy to remove the commented out dependency (I believe it was generated when the project was first generated as an example of adding a runtime dependency.)
We can leave the .gitignore file as is. A global .gitignore file should be used for IDE files. https://help.github.com/articles/ignoring-files/
If you revert the .gitignore file then I'll happily merge this.

Thanks <3


fix(orchestrator): fixes many security-related issues

Addresses:

https://issues.redhat.com/browse/FLPATH-1316

/approved

/approve

/lgtm

/approve


 and
 characters after transformation

After I transformed my xml file, I have
 and
 characters.
My config file

My original file

The result

Hi @KoosMos,
The XmlDocument class is encoding new lines in values when saving the document to a file. They are no options that i know of
to disable this behavior.
A workaround would be to remove the new lines in your original file or after the transformation do a replace of
 with a new line.


log old and new serials

This will log the old and new serial numbers.  Hmm.. the witespace is kinda weird, but I did do this as a patch off the 4.3.1 branch so maybe something changed upstream?

Thanks for the PR!  That looks like a nice printout addition.  Whitespace looks fine to me too.


Request of some advice to solve lld_pdu_get_tx_flush_nb HCI packet count mismatch (1, 2)

I tried to connect esp32 with xbox controller it again but failed with the following error.
lld_pdu_get_tx_flush_nb HCI packet count mismatch (1, 2)
E NimBLEClient: "Connection failed; status=574 "

The code (pClient->connect(true);) which causes the error.
https://github.com/asukiaaa/arduino-XboxControllerNotificationParser/blob/a0cf4052ee305cd6dfaf28cbf9c5c03044400d66/examples/printOnSerialWithNimBLE/printOnSerialWithNimBLE.ino#L248
I succeeded in it with your help before.
https://github.com/h2zero/NimBLE-Arduino/issues/172
My environment of PlatformIO
NimBLE: 1.3.1

PLATFORM: Espressif 32 (3.3.2) > Espressif ESP32 Dev Module
 - framework-arduinoespressif32 3.10006.210326 (1.0.6)
 - tool-esptoolpy 1.30100.210531 (3.1.0)
 - tool-mkspiffs 2.230.0 (2.30)
 - toolchain-xtensa32 2.50200.97 (5.2.0)

I rollbacked the versions as begin of this year but it also failed.
E NimBLEClient: "Connection failed; status=574 "

NimBLE: 1.1.0

PLATFORM: Espressif 32 (3.0.0) > Espressif ESP32 Dev Module
 - framework-arduinoespressif32 3.10004.210126 (1.0.4)
 - tool-esptoolpy 1.30000.201119 (3.0.0)
 - tool-mkspiffs 2.230.0 (2.30)
 - toolchain-xtensa32 2.50200.97 (5.2.0)

Could you give me some advice to solve this?
Thank you.

Please try removing the connection parameter setting at line 230 in your code and let me know if that has more success.

@h2zero
Thank you for the advice.
I comment out line 230.
    // pClient->setConnectionParams(12, 12, 0, 51);

After that it sometimes succeeded in connection but causes panic after that.
New client created
lld_pdu_get_tx_flush_nb HCI packet count mismatch (1, 2)
E NimBLEClient: "Connection failed; status=574 "
try connection again 2650
lld_pdu_get_tx_flush_nb HCI packet count mismatch (1, 2)
E NimBLEClient: "Connection failed; status=574 "
try connection again 4460
Connected
Connected to: 44:16:22:5e:b2:d4
RSSI: -42
E NimBLERemoteCharacteristic: "ble_gattc_disc_all_dscs: rc=3 "
E NimBLERemoteCharacteristic: "Error: Failed to get descriptors"
E NimBLERemoteCharacteristic: "ble_gattc_disc_all_dscs: rc=3 "
E NimBLERemoteCharacteristic: "Error: Failed to get descriptors"
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Thank you for reporting this, I see there is a bug in that section of the code that I will try to get fixed later today.
For a work around I suggest not using the discoverAttributes function, your afterconnect function should do that for you anyway.

I see there is a bug in that section of the code that I will try to get fixed later today.

Thank you,
I hope that you can fix this.

I suggest not using the discoverAttributes function,

Thank you for checking the code.
I commented out it.

@visicon-masc that message is a controller debug message indicating some packets were received that were not expected. This is basically reporting packet loss so the connection was not completed. There isn't much we can do about that as the causes are mostly external and the BLE controller is a closed binary source on the ESP32.
The bug reported in this thread is due to another cause that I have not had time to address yet, but the log message is not indicative of that.

I tried with the latest version of platforms.
The error still occurs but sometimes it succeeded in connection with xbox controller and can see the notification.
Platforms

PlatformIO 5.2.3
NimBLE 1.3.1
esp32

Platform espressif32
--------
Updating platformio/espressif32               3.4.0                              [Up-to-date]
Updating platformio/toolchain-xtensa32        2.50200.97 @ ~2.50200.0            [Up-to-date]
Updating platformio/framework-arduinoespressif32 3.10006.210326 @ ~3.10006.0        [Up-to-date]
Updating platformio/tool-esptoolpy            1.30100.210531 @ ~1.30100.0        [Up-to-date]
Updating platformio/tool-mkspiffs             2.230.0 @ ~2.230.0                 [Up-to-date]

Thank you.

The issue maybe caused by process in esp-idf.
https://github.com/espressif/esp-idf/issues/8303
So I close the issue on this library.

I traced timing of lld_pdu_get_tx_flush_nb HCI packet count mismatch (1, 2) and found that it is showed between calling ble_hs_hci_rx_evt.
The ble_hs_hci_rx_evt is registered by ble_hci_trans_cfg_hs.
https://github.com/h2zero/NimBLE-Arduino/blob/26e01302413e807bfbde435d6fe620eeac555fd0/src/nimble/nimble/host/src/ble_hs.c#L802
The ble_hci_trans_cfg_hs is defined in eps-idf for nimble.
https://github.com/espressif/esp-idf/blob/5c33570524118873f7bd32490c7a0442fede4bf8/components/bt/host/nimble/esp-hci/src/esp_nimble_hci.c#L57
I have to setup environment to debug esp-idf so I finish here today.

I would advise not spending too much time digging into this.
lld_pdu_get_tx_flush_nb HCI packet count mismatch (1, 2) is printed by the controller which is a closed source binary.
The error that's occurring is a timeout on the connection, which can be caused by many things. NimBLE host code simply sends the command to the controller and waits for its response. If the packets arrive out of sync the controller prints that message and returns the error code.
A short list of possible causes:

radio interference
low voltage
controller interrupted
poor timing control
controller source code bug

If you're inclined to dig deeper I'm all for it but I suspect you'll hit a road block at the controller or hardware level.
For reference this library has its own copy of the esp_nimble_hci.c file.

Thank you for the information.
I understood that the error may be returned by arriving packet at out of sync and it may be a problem of closed source library or hardware level.
Anyway I will dig esp-idf if I can keep interest to do that.
By the way, how about this PR?
https://github.com/h2zero/NimBLE-Arduino/pull/343

Note
Cleaning code sometimes solves the problem.
For platform io
pio run -t clean

lld_pdu_get_tx_flush_nb HCI packet count mismatch (1, 2) is printed by the controller which is a closed source binary.

My ESP32 stops working, if this message occuring. Is there a possibility to restart the ESP if this message is sent from the bt controller?

On my case, the error occurs on ESP32, but doesn't on ESP32S3 and ESP32C3.
I doubt some problem on IC.
The problem is discussed on esp-idf thread https://github.com/espressif/esp-idf/issues/8303 but it is still  unresolved.


WinRT Theme loading error not specific enough

Describe the bug
When loading a custom theme. If you specify only a "Light" theme and nothing esle the applicaiton wont start if your windowds is set to "Dark". Not in debug/release from VS or when packaged.
The probelm is that the error is generic and tell you it can't find the key it's looking for. Instead of it telling you there is no such ResourceDictionary with the key your are looking for.
Steps to reproduce the bug

Make a new project.
Load in a custom theme with only
Run applicaiton. Works fine.
Switch windows theme to "Dark"
Run applicaiton. Won't start.
Exception thrown at 0x00007FFDD6734F99 (KernelBase.dll) in Company.DesktopClient.exe: WinRT originate error - 0x80004005 : 'Cannot find a Resource with the Name/Key GradientBackground'.
onecore\com\combase\winrt\error\restrictederror.cpp(1016)\combase.dll!00007FFDD884C363: (caller: 00007FFDD8856049) ReturnHr(1) tid(2b78) 8007007E The specified module could not be found.
Exception thrown at 0x00007FFDD6734F99 (KernelBase.dll) in Company.DesktopClient.exe: WinRT originate error - 0x802B000A : 'Cannot find a Resource with the Name/Key GradientBackground [Line: 0 Position: 0]'.
onecore\com\combase\winrt\error\restrictederror.cpp(1016)\combase.dll!00007FFDD884C363: (caller: 00007FFDD8856049) ReturnHr(2) tid(2b78) 8007007E The specified module could not be found.
Exception thrown: 'Microsoft.UI.Xaml.Markup.XamlParseException' in WinRT.Runtime.dll

Expected behavior
I expected it to tell me that either no theme was found. Or that it would default to the only theme availbe.
Screenshots
No response
NuGet package version
0.8.4
Packaging type
Unpackaged
Windows version
October 2020 Update (19042)
IDE
Visual Studio 2019
Additional context
No response

I think if you named the resource dictionary "Default" it would work.


filesize_approx property is no logner proveded for vk.com, rutube.ru and others

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Russia
Provide a description that is worded well enough to be understood
Since yt-dlp 2023.06.21 there is no more filesize_approx attribute for videos json info.
In version yt-dlp 2023.03.04 there is such attribute.
Here is the commands to check the difference:
latest version
 docker run --rm -it jauderho/yt-dlp:2023.07.06 https://vk.com/video-211429367_456242294 -J | jq ".formats[].filesize_approx"
null
    null
        null
            null
                null
                    null
                        null
                            null
                                null
                                    null
                                        null
                                            %

version 2023.03.04
docker run --rm -it jauderho/yt-dlp:2023.03.04 https://vk.com/video-211429367_456242294 -J | jq ".formats[].filesize_approx"
104825011
         null
             204485132
                      null
                          410793074
                                   null
                                       669462405
                                                null
<PHONE_NUMBER>
                                                              null
<PHONE_NUMBER>
                                                                            %

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
docker run --rm -it jauderho/yt-dlp:2023.07.06 -vU
[debug] Command-line config: ['-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[b532a3481] (zip)
[debug] Python 3.11.4 (CPython x86_64 64bit) - Linux-<IP_ADDRESS>-microsoft-standard-WSL2-x86_64-with (OpenSSL 3.1.1 30 May 2023)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0
[debug] Optional libraries: mutagen-1.46.0, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: cedf44a9cf3c241520a692c1006d276e6f434d4d945727c47cb1a70d32953f4a
yt-dlp is up to date<EMAIL_ADDRESS>

This was intentionally removed due to its extreme inaccuracy. You can use tbr * duration if you want, which was how it was being calculated before

Hi, pukkandan, thank you for the answer.
There is 128 times difference between tbr*duration and filesize_approx.
What does this number stand for?

You probably got the units wrong. yt-dlp gives tbr in Kb/s and duration in secs. So multiplying them will give you approx filesize in Kb. If you want it in bytes, multiply with 1024/8=128

add --compat-option manifest-filesize-approx to your command if you want the filesize_approx value for manifest/fragmented formats


Kitura 0.13.6 doesn't build in Xcode

Hello,
Bringing this issue to a wider audience. Originally reported in Kitura-net:
Issue #28
Building Kitura 0.13.6 using swift build works ok. However when building from Xcode following error is returned when linking Kitura-net:
ld: library not found for -lCHttpParser for architecture x86_64
Xcode: 7.3.1 (7D1014)
Toolchain: Swift Development Snapshot 2016-05-03 (a)
I have previously installed and force-linked curl with
brew install curl && brew link curl --force

I had the similar issue (not with the same release): https://github.com/IBM-Swift/Kitura/issues/463; and followed this to get it resolved: https://github.com/IBM-Swift/Kitura/issues/459.

@michalkalinowski- There is no need to force link curl. Usually the problem is solved by deleting .build and Packages, rebuilding the application, and then doing a swift build -X to create the XCode project. There may have been some stuff remaining from older builds of the application.
In general when one wants to upgrade versions of dependencies with Swift Package Manager, one must delete the Packages directory.

I did one more test:

brew remove curl to use curl shipped with osx
I cloned a fresh copy of Kitura
swift build -X
opened project in Xcode
Built

I get the same linker error.
I don't see how configuration of my machine can have any effect on this error. Could anyone test the same steps?
If other people reproduce the same error it would mean Kitura-net needs to be fixed to work with Xcode with Development Snapshot 2016-05-03

@michalkalinowski- I appear to have been mistaken above. I can build KituraSample using XCode. I can't build Kitura using XCode.
I'm trying to understand what if any differences exist between the generated XCode projects.

What's funny with error, is that if one looks at the log, one sees that CHttpParser.dylib was built correctly and placed into the XCode build directory, which is not BTW ${SRCROOT}/.build/debug.
Having said that the link of Kitura-net fails, even though the link seems to include the CHttpParser.dynlib file.

Kitura-Sample depends on Kitura-0.12 -> Kitura-net-0.12
When I build the latest Kitura it resolves Kitura-net-0.13.5

@michalkalinowski- I believe it had something to do with SPM bug: https://bugs.swift.org/browse/SR-1444 (still being unresolved), particularly w/ the 05-03 snapshot. I got a weird error when using the Snapshot 05-03 for an iOS experiment (which wasn't happened in the 04-25).

@michalkalinowski- Which Kitura-Sample did you take? The one in the develop branch prereqs 0.13.X of Kitura....

@aquarchitect The issue with Kitura-TemplateEngine was solved in Kitura issue #459. If one is using the develop branch of Kitura et. al (i.e. 0.13.x) then the fix should be there.

I spent a little while yesterday trying to build different tags of Kitura-net with no luck. They all fail with the linking problem that makes me think it may be an issue with Xcode 7.3.1
I'll try to install 7.3 and try to build again this evening

I did some more testing. The issue doesn't lie within Xcode, it's the DEVELOPMENT-SNAPSHOT-2016-05-03-a toolchain:
I'm able to build Kitura-net in Xcode 7.3.1 using DEVELOPMENT-SNAPSHOT-2016-05-03-a if I generate Kitura-net.xcodeproj file using DEVELOPMENT-SNAPSHOT-2016-04-25-a toolchain. It means DEVELOPMENT-SNAPSHOT-2016-05-03-a changes the way xcodeproj file is generated. I will run a diff between the two to understand what has changed.
For now steps to reproduce my findings:

git clone https://github.com/IBM-Swift/Kitura-net.git
swiftenv local DEVELOPMENT-SNAPSHOT-2016-04-25-a
swift build -X
open Kitura-net.xcodeproj
change toolchain in Xcode to DEVELOPMENT-SNAPSHOT-2016-05-03-a
build -> success

Repeat the same steps but generate Kitura-net.xcodeproj with DEVELOPMENT-SNAPSHOT-2016-05-03-a and the build will fail

Ok, I've narrowed the problem down to CHttpParser package

create a new test project
add whatever dependency (Kitura-sys) for example
swift build -X with DEVELOPMENT-SNAPSHOT-2016-05-03-a
open in XCode and build --> success
add CHttpParser to Package.swift
swift build -X
try to build in Xcode --> linker error

This is a known issue we started seeing in 5/3. I don't believe there is a linker issue via CLI, only in Xcode. Please try the workaround in Step 2 in my recent blog post about contributing to Kitura.

you're right, issue is only in Xcode. I'm aware of the workaround, but I'm trying to find a root cause of this error.

Found it!
New Kitura fix:

clone Kitura, swift build -X, open in Xcode, build and you get well known linker error
cd ~/Library/Developer/Xcode/DerivedData/Kitura-net<something>/Build/Products/Debug
cp CHttpParser.dylib libCHttpParser.dylib

Voila, it builds!
It looks like they changed structure of project.pbxproj file, one of the chages is naming of frameworks (used to be lib and now they removed lib)
Now for some strange reason, in case of CHttpParser which is just a modulemap and c files Xcode still looks under an old name. I can not explain that any further, time to open a bug to package manager.

Found it!
New Kitura fix:

clone Kitura, swift build -X, open in Xcode, build and you get well known linker error
cd ~/Library/Developer/Xcode/DerivedData/Kitura<something>/Build/Products/Debug
- cp CHttpParser.dylib libCHttpParser.dylib
Voila, it builds!

It looks like they changed structure of project.pbxproj file, one of the changes is naming of frameworks (used to be lib[nameOfFramework] and now they removed lib)
Now for some strange reason, in case of CHttpParser which is just a modulemap and c files Xcode still looks under an old name. I can not explain that any further, but I'd say it's a bug with Xcode.

Great work figuring that out @michalkalinowski!! Has any bug been filed yet for this issue?

Closing this issue since a workaround for CHttpParser has been documented on the wiki. If you have a fix for the Xcode project generation, you should open a bug at bugs.swift.org or submit a PR against the Swift Package Manger.


add binary focal loss

Codecov Report

Merging #42 (127d69c) into main (fa81a34) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #42      +/-   ##
==========================================
+ Coverage   93.85%   93.92%   +0.06%
==========================================
  Files          35       35
  Lines        1416     1431      +15
==========================================
+ Hits         1329     1344      +15
  Misses         87       87

Impacted Files
Coverage Δ

src/hearth/losses.py
96.46% <100.00%> (+1.56%)
:arrow_up:

src/hearth/callbacks/monitors.py
94.59% <0.00%> (-2.71%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa81a34...127d69c. Read the comment docs.

Codecov Report

Merging #42 (127d69c) into main (fa81a34) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #42      +/-   ##
==========================================
+ Coverage   93.85%   93.92%   +0.06%
==========================================
  Files          35       35
  Lines        1416     1431      +15
==========================================
+ Hits         1329     1344      +15
  Misses         87       87

Impacted Files
Coverage Δ

src/hearth/losses.py
96.46% <100.00%> (+1.56%)
:arrow_up:

src/hearth/callbacks/monitors.py
94.59% <0.00%> (-2.71%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa81a34...127d69c. Read the comment docs.


Virtual Ethernet over USB (IDFGH-11409)

Is your feature request related to a problem?
Hi,
ESP32 S3 has no Ethernet integrated, only SPI Ethernet,
but has USB.
is is possible to make a Ethernet on USB, so if you plug into PC it has a IP Connection ?
Describe the solution you'd like.
No response
Describe alternatives you've considered.
No response
Additional context.
No response

https://github.com/espressif/esp-idf/tree/master/examples/peripherals/usb/device/tusb_ncm

cooooooooooooooool. want to test it !

https://github.com/espressif/esp-idf/tree/master/examples/peripherals/usb/device/tusb_ncm

had a look into this.
My next Question is:
if i have a internal File Server like this one:
https://github.com/espressif/esp-idf/tree/master/examples/protocols/http_server/file_serving
and i want to access this one over this Ethernet over USB,
how to do it ?
i do not want to brige Wifi to USB so that the host could access the Internet, but let the Host get a IP over DHCP and access the Internal Webserver.
Next Question would be if i want this Ethernet over USB and a Wifi AP on the ESP32.
so i can access the File Server either over USB or over Wifi AP

found this:
https://github.com/kaspernyhus/esp_tinyusb_test/blob/master/main/usb_ncm.c

found this:
https://github.com/buglloc/BoundBoxESP/blob/main/components/hardware/src/net_usb.cc

found this: https://github.com/kaspernyhus/esp_tinyusb_test/blob/master/main/usb_ncm.c

this helped.
working.
even with DHCP

so, tested on a Windows today, and get a unknown Device in Device Manager,
but was running fine on Linux.
where to get the Driver ?
Thanks !

@igrr any Help getting this als work under Windows ?
or any way i could ping someone who has such driver ?

working:
https://learn.microsoft.com/en-us/answers/questions/52386/usb-cdc-ncm-devices-not-assigned-to-usbncm-driver
https://learn-attachment.microsoft.com/api/attachments/91824-win10-native-ncm.pdf?platform=QnA


[refactor] move template variable handler to page

Coverage decreased (-0.001%) to 82.782% when pulling 604d733bf82dfaf18d9f365c1858530981fd178f on sunny4381:refactor-template-variable into 26b5fba66ba5321f76c23da4730d30e78bd98367 on shirasagi:master.

Coverage increased (+0.03%) to 82.798% when pulling 88fe2b0f75928e5468e150ca32faf611ad9d1e36 on sunny4381:refactor-template-variable into 24c08439a94d6c86bb92233342afe857c602d236 on shirasagi:master.


Quick question!

I was wondering if the model was compatible with android, I wanted to make an activity recognition app. Thanks for the help!

Hi, hiskuDN,
Personally, I haven't worked on the deployment part so I'm not sure about the answer.


342 geospatial reference data

https://github.com/uwlib-cams/MARC2RDA/blob/main/Working Documents/3XX.csv

@CatalogerCate, Penny and I have run into a question we hope you won't mind answering. Is there a difference between a prime meridian and a projection center? The context is MARC 342 $g and $h. Thank you in advance!

Also, would resolution on a map be a work or expression characteristic? @CatalogerCate @GordonDunsire

@CECSpecialistI: I don't know much about map projections, but a prime meridian (or any line of longitude) is a line, and I assume a projection centre is a point that is specified by the intersection of the meridian with a line of latitude. They pertain to different types of map projection as specified in the MARC 21 manual.
I assume that, in general, resolution is an expression characteristic, with the work covering the area represented, features selected, etc. But I suspect for some specific cases the decision would be that a difference in resolution results in a new work. This is pretty much the same as for all kinds of works and expressions. This should not be confused with printing or display resolution (e.g. 200 dots per inch), which is a manifestation characteristic.

"Thank you very much, @GordonDunsire, for the explanation. Can we do better than note on expression for $c - Latitude resolution (NR), $d - Longitude resolution (NR), and $t - Vertical resolution (NR)?
And I run into another question, for the second indicator, 7 - Method specified in $2, how can I address the mapping of 7 when it is combined with two first indicators and all subfields?" @CECSpecialistI @lake44me

@GordonDunsire @CECSpecialistI @lake44me There are many conditions in 342. I'd like to clarify these conditions for easy understanding, but not sure what approach I should take to explain these conditions. For example:
$g Longitude of central meridian or projection center
Longitude of the central meridian (the line of longitude at the center of a map projection, generally used as the basis for constructing the projection) when the second indicator contains the value 1 and subfield $a contains the text Albers conical equal area, Azimuthal equidistant, Equidistant conic, Equirectangular, Lambert conformal conic, Mercator, Miller cylindrical, Polyconic, Sinusoidal, Transverse Mercator, or Van der Grinten. Subfield $g contains a longitude of projection center (longitude of the point of projection for azimuthal projections) when the second indicator contains the value 1 and subfield $a contains the text General vertical near-sided projection, Gnomomic, Lambert azimuthal equal area, Orthographic, Robinson, or Stereographic.
342 | 01$aPolyconic$g0.9996$h0$i500,000$j0
$k - Scale factor
Used when the first indicator contains the value 1. If subfield $a contains the text Mercator, subfield $k contains the scale factor at equator (a multiplier for reducing a distance obtained from a map to the actual distance along the equator). If subfield $a contains Oblique Mercator, subfield $k contains the scale factor at center line (a multiplier for reducing a distance obtained from a map to the actual distance along the center line). If subfield $a contains Transverse Mercator, subfield $k contains the scale factor at central meridian (a multiplier for reducing a distance obtained from a map to the actual distance along the central meridian). If subfield $a contains Polar stereographic, subfield $k contains the scale factor at the projection origin (a multiplier for reducing a distance obtained from a map to the actual distance at the projection origin).
342 | 12$aUniversal Transverse Mercator$p13$k0.9996$g-105.00$h0.00$i500,000$j0.0
Could you please give me some advice or examples?

Hi @pennylenger ! This is a complex field! Let's work through some of these conditions together on Thursday. It will be a good introduction to writing mapping conditions and will be easier than trying to explain with a comment.

@pennylenger Hi Penny, just to clarify, for the transform note on $k and when $a contains "mercator", the condition should be "contains mercator AND not contains oblique mercator and transverse mercator"? I.e. $a universal transverse mercator should only have the text ”scale factor at central meridian“ and not ”scale factor at equator“?

Yes, you are right.


Toggle Button Invisibility in Bright Mode

Problem: The toggle button is not visible when the application/website is in bright mode.
Expected Behavior: The toggle button should be clearly visible and functional regardless of the display mode (bright or dark).

i can fix this issue please assign me


Smartcard CBA Completed Flow

Note
New smartcard CBA features will be merged into the feature branch melissaahn/smartcardCBAFeature until the feature is ready for the next Broker and MSAL releases.
The Feature
This PR builds on my second PR by using the information retrieved by the certificate picker and the PIN dialog to attempt CBA with a certificate located on a YubiKey device. The parts of the smartcard CBA feature that are being address in this PR are (with corresponding PBIs linked):

Given the slot where a chosen certificate resides and a verified PIN, authentication via a modified private key can be attempted.

Get main auth success scenario working for smartcard certificates

Testing and handling alternative flows, including early unplugging and no certificates on device (exceptions are handled with a general error dialog, but they have not been tested yet).

Test For User Case Extensions and Alternate Flows

Changes Made
Main changes:

YubiKit and PIV version bumped up to 2.1.0-alpha.1, which include methods that help facilitate client TLS authentication without having to copy the private key from the YubiKey device.
useSmartcardCertForAuth method

Adds PivProvider in Security static list of security providers
Uses a PivProvider to set up KeyStore instance
Sets up PivPrivateKey and certificate chain
Attempts to authenticate using PivPrivateKey object and certificate chain as parameters to request.proceed()

SmartcardErrorDialog class takes in two string resource ids and displays a dialog when an error occurs.

Result
When the user chooses a cert in the picker and successfully enters their PIN, they are either logged in successfully or met with an error page from the server.
If, at any point when a dialog is showing, the user decides to unplug their key, an error dialog will be shown reminding the user to replug their key in. The current CBA flow will be cancelled.
If the user does not have any certificates on their YubiKey device, they will be shown an error dialog informing them they do not have any relevant certificates on their device. The current CBA flow will be cancelled.
Follow Ups
A private build of Authenticator with the smartcard CBA feature will be tested in private preview.
Need to ensure before GA that none of the logging statements will contain PII.
Testing
Below are the general testing steps I'm planning on using to help form manual tests for Broker and MSAL. I'll need to contact Labs to see if they can provide test AAD accounts for CBA.
Post private preview, Robolectric will be used to test scenarios where exceptions can be thrown.
Beginning of all scenarios:

Download AzureSample.
Go to Multiple Account Mode.
Click "Get Graph Data Interactively".
Type the upn associated with the cert on key (or basically any other upn that brings up the "sign in with a certificate" link on the next page). Click "Next".
Plug in the YubiKey.
When the permission dialog pops up, click "OK".
Click "Sign in with a certificate". The custom cert picker should pop up with a listing of the certs currently held on the YubiKey.

Success Scenario:
8. Select the cert associated with the upn and click "Continue". The pin dialog should pop up.
9. Enter the YubiKey's pin and click "Unlock". The dialog should dismiss, and the page should be redirected to the Multiple Account fragment, where the user should successfully be logged in.
10. Repeat steps 3-9 with a second account.
Incorrect Pin Scenario:
8. Select the cert associated with the upn and click "Continue". The pin dialog should pop up.
9. Enter an incorrect YubiKey pin and click "Unlock". The EditText should clear and should have a red message underneath that says "The PIN you entered was incorrect".
10. Enter the YubiKey's pin and click "Unlock". The dialog should dismiss, and the page should be redirected to the Multiple Account fragment, where the user should successfully be logged in.
Max Pin Attempts Allowed Scenario:
8. Select the cert associated with the upn and click "Continue". The pin dialog should pop up.
9. Enter an incorrect YubiKey pin and click "Unlock". The EditText should clear and should have a red message underneath that says "The PIN you entered was incorrect".
10. Repeat step 9 two more times. The pin dialog should be replaced with another dialog that states "Too many failed attempts, try again later."
11. Click "Ok" on the dialog and it should dismiss.
12. Click the back button.
13. Retype the upn and click "next".
14. Click "Sign in with a certificate". The same "Too many failed attempts" dialog should pop up, and the error page should be seen.
Early YubiKey Unplugging Scenario:
8. Unplug the YubiKey. The cert picker dialog should dismiss and a "Try Again" dialog should be shown.
9. Replug the YubiKey and click "Ok" when the permission prompt comes up.
10. Click the back button.
11. Retype the upn and click "next".
12. Click "Sign in with a certificate". The custom cert picker should pop up with a listing of the certs currently held on the YubiKey.
13. Select any cert and click "Next". The Pin dialog should pop up.
14. Unplug the YubiKey. The pin dialog should dismiss and a "Try Again" dialog should be shown.
16. Repeat steps 9-12.
17. Follow the "Max Pin Attempts Allowed Scenario", and when the error dialog shows up, unplug the YubiKey. The dialog should dismiss and a "Try Again" dialog should be shown.
Switch To On-Device Certs Scenario:
8. Click "Cancel". The dialog should dismiss and the error page should be shown.
9. Unplug the YubiKey.
10. Click the back button.
11. Retype the upn and click "next".
12.  Click "Sign in with a certificate". The default Android on-device cert picker should pop up.
All of the scenarios above should be repeated with brokerHost installed as well.

Can you also verify the following in the static pivprovider callback?

what happened if you force throw an exception (runtime/non-runtime). Does the webview crashes? Or is it handled gracefully?

(Regardless, we should still log every exception in that block, and rethrow if needed)

if it's handled gracefully, what's the behavior? Is it the same as calling cancel()?

Can we display a dialog in that operation?

Tried a few things:

Attempted to show dialog straight in callback (within and outside of runOnUiThread -> crashed webview and app (for the app part, probably has to do with UI for adapter)
Attempted to cancel Request, then invoke Result.failure on callback -> crashed webview and app due to request being cancelled twice.
Attempted to just invoke Result.failure in callback -> webview closes, see MsalServiceException "ERR_SSL_CLIENT_AUTH_SIGNATURE_FAILED" in body of AzureSample. No fatal errors, but notably this is NOT the same as calling request.cancel().

For private preview, the last option is probably the best. I probably have to experiment and debug a bit more to see what can be done otherwise.

Addressed feedback by implementing changes and responding to comments.
If anyone has a YubiKey and wants to test:

I need to provide you a cert and account, so ping me.
MSAL branch: melissaahn/YubiKeyConfigChanges
broker branch: melissaahn/YubiKitOverrideIntegration
I've been using AzureSample and BrokerHost.

Added Telemetry to report if PivProvider already exists in the Security static list in one of the latest commits. Please let me know if I'm going about this the correct/wrong way. I'm thinking about creating a custom Event for CBA in a subsequent PR post private preview.


Deprecate history panel

Thank you, history panel! You gave us so much insights and guided us to Activity Log.
Happy retirement!

Victory! 🚢


adding coralogixOperator.secretKey function

If secretKeyReference is defined then use the secret key name, otherwise default to CORALOGIX_API_KEY

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Do omnisharp has a build error cache?

I'm converting a legacy code to net core and during this process, I need adjust some issues. When I finally resolve it, the editor keeps a part of the code underlined, as if it's still in error...
Environment data
dotnet --info output:
.NET Command Line Tools (1.0.1)
Product Information:
Version:            1.0.1
Commit SHA-1 hash:  005db40cd1
Runtime Environment:
OS Name:     Windows
OS Version:  6.1.7601
OS Platform: Windows
RID:         win7-x64
Base Path:   C:\Program Files\dotnet\sdk\1.0.1
VS Code version:
Version 1.11.2
Commit 6eaebe3b9c70406d67c97779468c324a7a95db0e
Date 2017-04-13T08:03:11.395Z
Shell 1.4.6
Renderer 53.0.2785.143
Node 6.5.0
C# Extension version: 1.10.0-beta2
Steps to reproduce
I can't reproduce this...
Expected  behavior
Actual behavior
It looks like it has an error cache

When I do Reload Window the problem desapears...

I've seen this occasionally but haven't been able to reproduce it either. My suspicion is that the problem is actually in VS Code itself, but I haven't been able to trigger the problem under the debugger to know for sure.

Thanks @DustinCampbell
This problem is more often in beta versions!

And to specifically answer your question, no we don't cache diagnostics in OmniSharp or C# for VS Code. The only cache of diagnostics is in the DiagnosticCollection that VS Code provides us with (code).
If you keep running into this problem, would you mind providing your OmniSharp Log? It'd also be helpful if you set the "omnisharp.loggingLevel" setting in VS Code to "debug" to provide more information.
Some things to watch out for:

Are the diagnostics that seem to be hanging around produced by typing, or by building?
Are there particular diagnostics that seem to hang around more than others?
Are there particular activities that seem to leave diagnostics hanging around? For example, running a code action from a light bulb might be such an activity.

OmniSharp Log:
omnisharp.log.txt

Q: Are the diagnostics that seem to be hanging around produced by typing, or by building?
A: The problem occurs when I'm typing but still after build.
Q: Are there particular diagnostics that seem to hang around more than others?
A: Yes. In my case, I had a virtual method called "Remover", but it isn't virtual anymore. So I need to changed it to a new method "CriarComandoRemover". When I do this, the problem occurs.
     But the problem also occurs in other cases...

Q: Are there particular activities that seem to leave diagnostics hanging around? For example, running a code action from a light bulb might be such an activity.
A: Normally I clean unused usings (ctrl + .) and refactoring the class.

You type, and the errors from the build don't go away. Is that correct?

Yes, but only on Editor. On the "Problems" output doesn't show anymore.

@bernardbr: Is this still an issue for you? If so, have you been able to lock down a repro?

Hello, @DustinCampbell I'm using the last beta version on OmniSharp and it works fine!
I'm closing this issue.
Thanks!

Glad to hear it -- thanks!


Hello @pmeier. :wave:

Hello @pmeier. :wave:
Hope you are doing well.
It looks like dIoU PR is almost finished (except the refactoring that will be started later I guess).
In the meantime, is there something I can work on? Thanks for your guidance and time.
最初由 @yassineAlouini 在 https://github.com/pmeier/pmeier/issues/5 发布

Hey @aipwe15 :wave: It seems this contact is genuine. I'm not sure why you chose to open this issue in a repository I'm not affiliated with in any way? If you want to reach out to me, please open an issue here. If it was indeed a mistake, please also close this issue here to avoid adding noise on an issue tracker.


Defer 'nc_time_axis' import, making dependency optional.

Hopefully closes #2113

Thanks @pp-mo! Do we also want to remove nc_time_axis from the conda requirements file? Or is that one dependency removal too far?

failing examples will still fail

I think I've sorted this out now...
As nc_time_axis is an optional dependency, the tests should run without it.
They now do (and before was not absent from the 'minimal' test set).
Meanwhile, the examples can still require it, as they are only run with the full dependencies.

I think that this looks ready to merge.
The use of the IrisError makes sense to me, as Iris is unable to function in this case, due to a dependency problem.  Unless there's an established pattern for this kind of error, I'm content with this one
@pp-mo anything outstanding from your p.o.v.?

@pp-mo
this is a 1.10 issue, please target the v1.10.x branch, rather than master?

Hi @pp-mo
please may you remove the reference to nc_time_axis from
minimal-conda-requirements
as part of this PR
thank you

@marqh please may you remove the reference to nc_time_axis from minimal-conda-requirements
Some confusion here? : this was already done

Some confusion here? : this was already done

so I see, my mistake
thank you


Fix recursive exceptions from _RateLimitFilter

We have a very useful piece of code, _RateLimitFilter, which is designed to avoid sending us a billion error emails in the event that a Zulip production server is down in a way that throws the same exception a lot.  The code uses memcached to ensure we send each traceback roughly once per Zulip server per 10 minutes (or if memcached is unavailable, at most 1/process/10 minutes, since we use memcached to coordinate between processes)
There are a few problems with it:
(1) There are no tests, which makes iterating on the code a risky option.
(2) If memcached is down, there end up being some logging.error calls internal to the Django/memcached setup that happen inside the cache.set() call, and those aren't caught by the except Exception block around it.  This ends up resulting in infinite recursion, eventually leading to Fatal Python error: Cannot recover from stack overflow., since this handler is configured to run for logging.error in addition to logging.exception.
We should fix this as follows:

Eliminate the "infinite recursion" problem by adding an outer try/finally block that sets a global (or maybe thread-local?) variable handling_exception = True at the start, and sets it back to false at the end.
Add unit tests that can reproduce this error condition (I think basically we just need to mock.patch the cache.get call to either throw an exception or do a logging.error call for the main corner cases)
Let's look at the actual logging.error message Django gives when memcached is down; I have a recollection that it has a large random ID in it, which makes the duplicate-filtering we use not work at all.

Can I give this issue a try?

@Arunhari333 this is likely a difficult issue requiring a lot of expertise; given your question, I think this is likely not a good issue for you to get involved with.

@zulipbot claim

PR submitted https://github.com/zulip/zulip/pull/13123


Chapter 7 - postInsert error

When trying to submit a new post, I'm getting a 'postInsert method not found' error and the post fails to submit. I've checked my code and it seems identical to that in your book.
Any ideas?
Adrian

Hi,
Do you know the answer? I met the same question.

lib/collections/posts.js
change the code like this:

check(Meteor.userId(), String);

check(this.userId, String);


Feature: Restart on Failure

Feature Description:
sometimes you want to run a long running command and make sure it is always up.
if your command fails or an unexpected thing happen you want to command to still run and if it crash then, is should be restarted
Expected Behavior:
create a flag -r, --restart-on-failure  the will listen for the running command and make sure to restart it if it exits with a non-0 status code

#44 Addresses this issue


Using with a custom model not on HuggingFace

Your question
This is an awesome project, kuddos to you. I hope you can help me use it!
I've created a custom model for multi text classification that I've exported as an .onnx file (originally made in sklearn). I've tried to import this file a bunch of different ways to use it and I've thus far been unsuccessful.
Can you provide some guidance about how one might use a custom model? I can't find much useful documentation on how to put together config.json and such needed here... plus, it seems like most of the documentation right now is geared toward models published on HuggingFace -- rather than proprietary custom models.
End goal here is I have a bunch of training data that takes NLP and assigns it 0 or more custom (known) tags. I'd like to be able to use this model in a PHP request to create tags from input text.
Context (optional)

model = clr.fit(
    cv_clean.transform(X_train_clean),
    y_train
)

initial_type = [('float_input', FloatTensorType([None, cv_clean.transform(X_train_clean).shape[1]]))]

onnx_model = convert_sklearn(model, initial_types=initial_type, target_opset=20, options={type(clr.estimator): {'zipmap': False}})

onnx_model_path = "model_quantized.onnx"
with open(onnx_model_path, "wb") as f:
    f.write(onnx_model.SerializeToString())

Reference (optional)
No response

To be a little more clear my thoughts here are:

Create a model (in this case I used sklearning and then converted it to a .onnx)
Deploy the model in my PHP app by storing it where transformers-php expects to find it (in my case, storage/app/transformers/coogle/tags-classification

Ultimately I'm trying to create an interface such that:
$classifier = pipeline('text-classification', $model);
$result = $classifier($input);

Where $result will be the tags the model predicted should be applied to $input.
FWIW this is literally my very first attempt at training my own model, so it's entirely possible I have no idea what I'm doing here and any pointers in the right direction to getting this specific use case working would be extremely appreciated.

Hi @coogle ,
Thanks for your interest in TransformersPHP! It's great to see you exploring custom model deployment. You're right - our docs mostly focus on HuggingFace models since that's the common use case. There's a little part that discussed using custom models though, but I must admit that it's not as detailed as it needs to be.
The pipeline function may not be the best thing to use in this case. It's more of a helper method that wraps the complex processes involved in inferencing from preprocessing inputs to post processing outputs. For text classification however, the process is fairly simple - tokenization, inference proper, and post-processing. For custom usage, it's recommended to use the individual model classes.
Tokenizer
First, you’ll need to load the tokenizer with AutoTokenizer. Ensure you have a tokenizer.json in your model folder, so the class picks it up locally instead of trying to reach the Hub. The structure of this file depends on the tokenization process you used when training.
Here’s a breakdown of the top-level keys in tokenizer.json:

Normalizer: This step involves normalizing the input text (e.g., lowercasing, Unicode normalization). Example: BertNormalizer.
Pre-Tokenizer: This step splits the input text into tokens. Example: Whitespace.
Model: This is the core of the tokenizer, mapping tokens to IDs. The model type can be Unigram, WordPiece, or BPE (Byte-Pair Encoding). Example: WordPiece.
Post-Processor: This step modifies the tokens after initial tokenization. Example: TemplateProcessing.
Decoder: This step converts token IDs back to text. Example: BPEDecoder,  ByteFallback, FuseDecoder, MetaspaceDecoder

TransformersPHP supports these types, and you should match the values in your tokenizer.json accordingly. Check out the toxic-bert tokenizer for an example. In my experience, the Model is the most important, for the rest, you can make them null
Here’s an example structure for a tokenizer.json file:
{
  "normalizer": {
    "type": "BertNormalizer",
    "clean_text": true,
    "handle_chinese_chars": true,
    "strip_accents": null,
    "lowercase": true
  },
  "pre_tokenizer": {
    "type": "Whitespace"
  },
// "pre_tokenizer":  null (it could be null too)
  "model": {
    "type": "WordPiece",
    "vocab": {
      "[PAD]": 0,
      "[UNK]": 1,
      "[CLS]": 2,
      "[SEP]": 3,
      "[MASK]": 4,
      "the": 5,
      "to": 6,
      // additional vocabulary entries
    },
    "unk_token": "[UNK]",
    "max_input_chars_per_word": 100
  },
  "post_processor": {
    "type": "TemplateProcessing",
    "single": "[CLS] $A [SEP]",
    "pair": "[CLS] $A [SEP] $B:1 [SEP]:1",
    "special_tokens": {
      "[CLS]": 2,
      "[SEP]": 3
    }
  },
  "decoder": {
    "type": "WordPiece",
    "cleanup": true
  }
}

If everything is setup right, then you can create the tokenizer like so:
$tokenizer = AutoTokenizer::fromPretrained('coogle/tags-classification');

PS: Could you please explain your tokenization process? This way, I can list the supported types for each stage that match your method.
Model
Next, load your model with AutoModel::fromPretrained. You'll need a config.json in your model directory,  or you can just pass an array with necessary configs and it'll skip looking for the JSON file. If you check example config.json like this, there are a lot of keys as well. Most of them are relevant for different tasks. In this case (text-classification), the most relevant keys are the  label2id and id2label (which map string labels to numerical ids that the model will output and vice versa respectively).
$model = AutoModel::fromPretrained(
    'coogle/tags-classification',
    config: [
        'label2id' => ['tag1' => 0, 'tag2' => 1, 'tag3' => 2],
        'id2label' => ['0' => 'tag1', '1' => 'tag2', '2' => 'tag3']
    ],
    modelFilename: 'model_quantized.onnx'
);

Inference
Tokenize your input text and perform inference:
$modelInputs = $tokenizer->tokenize($textInputs, padding: true, truncation: true);
$outputs = $model($modelInputs);
// The proceed compute softmax of the output logits to get the predictions, then map the ids to corresponding labels.

Using Pipeline with Custom Models
Although not in the documentation, you can actually manually construct the pipeline (without the pipeline helper method this time around). You'll use the model and tokenizer instance this time around.
$classifier = new TextClassificationPipeline('text-classification', $model, $tokenizer);
$result = $classifier($input);

This lets you leverage the pipeline's functionality while using your custom model and tokenizer.
Feel free to reach out if you have more questions.

Thanks for getting back to me! I really appreciate this help it's exactly what I'm missing.
When I tried to do as you suggested I'm getting an error:
($labels is my labels as an id => string array -- my model_quantized.onnx is in coogle/tags-classifier/onnx/model_quantized.onnx)
            $modelObj = AutoModel::fromPretrained(
                $model,
                config: [
                    'id2label' => $labels,
                    'label2id' => array_flip($labels),
                ],
            );

 TypeError  Cannot assign null to property Codewithkyrian\Transformers\Utils\AutoConfig::$modelType of type string.

--
 () at vendor/codewithkyrian/transformers/src/Utils/AutoConfig.php:27

It seems that AutoConfig requires modelType to not be null here -- I tried looking into this code but I didn't know enough about what exactly you were doing here to understand if modelType should be able to be null and it's a typing issue, or if I was doing something wrong here.
Regarding the tokenizer, I'm using:
from sklearn.feature_extraction.text import TfidfVectorizer # Tokenizes the data based on a term-frequency-inverse-document algorithm

I haven't figured out how to generate a tokenizer.json from this yet (suggestions most welcome)

Just to give you some more insight here's the python script I'm using right now... pardon the horrible code as I hack around (you can see where I was playing around with generating a tokenizer.json which is obviously wrong)
#!/usr/bin/env python
# coding: utf-8

# In[1]:

import warnings
import os
import pandas as pd # For loading data
import nltk # NLTK stands for Natural Language Took Kit
from nltk.corpus import stopwords
import re # Regex
import string # String library has more string manipulation features
import numpy as np # Numerical Python library
import matplotlib.pyplot as plt # For graphing
import seaborn as sns # For graphing
from sklearn.feature_extraction.text import TfidfVectorizer # Tokenizes the data based on a term-frequency-inverse-document algorithm
from sklearn.feature_extraction.text import CountVectorizer # Implements tokenization and counts all in one
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.metrics import multilabel_confusion_matrix
from sklearn.multioutput import MultiOutputClassifier # Multi-output wrapper import
from sklearn.naive_bayes import MultinomialNB # Naive Bayes import
from sklearn.svm import LinearSVC # Support Vector Machine import
from sklearn.metrics import precision_score, recall_score, f1_score, accuracy_score # Metrics
from sklearn.linear_model import LogisticRegression # Logistic regression import
from skl2onnx import to_onnx
import onnxruntime as rt
import onnx
import skl2onnx
from skl2onnx.common.data_types import FloatTensorType
from skl2onnx import to_onnx
from skl2onnx import convert_sklearn
import json
from transformers import AutoTokenizer

# In[2]:

warnings.filterwarnings("ignore")

# In[3]:

nltk.download('stopwords')

# In[4]:

df = pd.read_csv('model-data/tags-training-data.csv')
df = df.convert_dtypes()
df

# In[5]:

df_tags_trans = df.iloc[:, 1:].T.sum(axis=1).sort_values(ascending=False).copy()
df_tags_trans

# In[6]:

stop_words = stopwords.words('english')

def clean_text(text):
    text = str(text).lower() # Lowercases everything
    text = re.sub('<.*?>+', '', text) # Removes everything between '<' and '>'
    text = re.sub(r'[^\w\s]', '', text) # Removes all punctuation
    text = re.sub('\n', '', text) # Removes any breaklines
    text = re.sub('\w*\d\w*', '', text) # Removes any words that contain numbers

    # Remove Stopwords
    text = ' '.join(word for word in text.split() if word not in stop_words)

    # Removing non english characters
    text = ''.join(line for line in text if line in string.printable)

    # Fixing extra whitespaces between words
    text = ' '.join(text.split())
    return text

# In[7]:

df['query_cleaned'] = df['query'].apply(clean_text)
df[['query', 'query_cleaned']]

# In[8]:

df = df.sample(df.shape[0], random_state=43)
df = df.reset_index(drop=True)
df

# In[9]:

percent = 0.9
num_training_examples = int(df.shape[0] * percent)
df_train = df.iloc[:num_training_examples]
df_test = df.iloc[num_training_examples:]

# In[10]:

df_train.shape

# In[11]:

df_test.shape

# In[12]:

X_raw = df['query']
X_clean = df['query_cleaned']
y = df.columns[1:]

X_train_raw = df_train['query']
X_train_clean = df_train['query_cleaned']

X_test_raw = df_test['query']
X_test_clean = df_test['query_cleaned']

y_train = df_train.iloc[:, 1:-1].astype(int)
y_test = df_test.iloc[:, 1:-1].astype(int)

# In[32]:

tfidf_raw = TfidfVectorizer()
tfidf_raw.fit_transform(X_train_raw)

tfidf_clean = TfidfVectorizer()
tfidf_clean.fit_transform(X_train_clean)

cv_raw = CountVectorizer()
cv_raw.fit_transform(X_train_raw)

cv_clean = CountVectorizer()
cv_clean.fit_transform(X_train_clean)

# In[29]:

def smaller_classification_report(y_true, y_pred, floating_point = 2):
    """
    Function to help us print a nicer classification if we just want selected metrics
    """
    print("Accuracy \t\t{}".format(round(accuracy_score(y_true, y_pred), floating_point)))
    print("Precision (weighted) \t{}".format(round(precision_score(y_true, y_pred, average='weighted'), floating_point)))
    print("Recall (weighted) \t{}".format(round(recall_score(y_true, y_pred, average='weighted'), floating_point)))
    print("F1-score (weighted) \t{}".format(round(f1_score(y_true, y_pred, average='weighted'), floating_point)))

# In[33]:

# Creating the chain model
clr = MultiOutputClassifier(
    LogisticRegression(max_iter=500)
)
model = clr.fit(
    #cv_clean.transform(X_train_clean), # Uncomment to try and comment out below
    tfidf_clean.transform(X_train_raw),
    y_train
)
# Making predictions
y_pred = model.predict(
    #cv_clean.transform(X_test_clean), # Uncomment to try and comment out below
    tfidf_clean.transform(X_test_raw)
)

smaller_classification_report(y_test.values.astype(int), y_pred)

# In[34]:

# Define the initial types for the ONNX model
initial_type = [('float_input', FloatTensorType([None, tfidf_clean.transform(X_train_clean).shape[1]]))]

# Convert the trained model to ONNX format
onnx_model = convert_sklearn(model, initial_types=initial_type, target_opset=20, options={type(clr.estimator): {'zipmap': False}})

# Save the ONNX model to a file
onnx_model_path = "build/Arvee/tags-classifier/onnx/model_quantized.onnx"
with open(onnx_model_path, "wb") as f:
    f.write(onnx_model.SerializeToString())

print(f"ONNX model saved to {onnx_model_path}")

labels = list(y_train)
id2label = {i: label for i, label in enumerate(labels)}
label2id = {label: i for i, label in enumerate(labels)}

# Create the configuration files for transformers library
config = {
    "model_type": "text-classification",
    "vocab_size": len(tfidf_clean.vocabulary_),
    "hidden_size": tfidf_clean.transform(X_train_clean).shape[1],
    "id2label": id2label,
    "label2id": label2id
}

# Save the configuration to a JSON file
config_path = "build/Arvee/tags-classifier/config.json"
with open(config_path, "w") as f:
    json.dump(config, f, indent=2)

print(f"Configuration saved to {config_path}")

tokenizer_path = "build/Arvee/tags-classifier/"
# Load a predefined tokenizer and save its files
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
# Save the tokenizer to the current directory
tokenizer.save_pretrained(tokenizer_path)

print(f"Tokenizer Metadata saved to {tokenizer_path}")

# Load the ONNX model using ONNX Runtime
session = rt.InferenceSession(onnx_model_path)

# Prepare the test data for ONNX Runtime
X_test_onnx = tfidf_clean.transform(X_test_clean).toarray().astype(np.float32)

# Execute the model against the test data
input_name = session.get_inputs()[0].name
label_name = session.get_outputs()[0].name

y_pred_onnx = session.run([label_name], {input_name: X_test_onnx})[0]

# Print the classification report
smaller_classification_report(y_test.values.astype(int), y_pred_onnx)

Great info!
First off, you could fix the error by setting model_type as custom in the config array you passed to the AutoModel. But then, in your training script, I noticed you were already saving a config.json, so you needn't pass the config array to the AutoModel anymore. Just make sure the saved config.json is in the right directory and it'll load the config from there.
labels = list(y_train.columns)
id2label = {i: label for i, label in enumerate(labels)}
label2id = {label: i for i, label in enumerate(labels)}

config = {
    "model_type": "custom",
    "vocab_size": len(tfidf_clean.vocabulary_),
    "hidden_size": tfidf_clean.transform(X_train_clean).shape[1],
    "id2label": id2label,
    "label2id": label2id
}

config_path = "build/Arvee/tags-classifier/config.json"
with open(config_path, "w") as f:
    json.dump(config, f, indent=2)

Then for the tokenizer, I'd suggest constructing one from scratch for now, instead of reusing the BertTokenizer. I'll try and walk you through the different parts of it, but do some research to back it up later as well.
Normalizer
Your clean_text function performs several normalization steps like lowercasing, removing HTML tags, punctuation, line breaks, and stopwords. This can be represented using a combination of normalizers. For eg.
"normalizer": {
  "type": "Sequence",
  "normalizers": [
    {"type": "NFD"},
    {"type": "Lowercase"},
    {"type": "StripAccents"}
  ]
}

But there's a predefined normalizer, BertNormalizer that can handle lowercasing, stripping accents, and handling Chinese characters so I'd go for that instead.
"normalizer": {
    "type": "BertNormalizer",
    "clean_text": true,
    "handle_chinese_chars": true,
    "strip_accents": true,
    "lowercase": true
  },

For the stop words and removing HTML Characters, you can use a couple of Replace normalizers. The Replace normalizer expects a pattern with a String or Regex key. Eg.
{
  "normalizer": {
    "type": "Sequence",
    "normalizers": [
      {
        "type": "Replace",
        "pattern": {
          "Regex": "<.*?>"
        },
        "content": ""
      },
      {
        "type": "Replace",
        "pattern": {
          "Regex": "\\b(" + "|".join(stop_words) + ")\\b"
        },
        "content": ""
      },
      {
        "type": "BertNormalizer",
        "clean_text": true,
        "handle_chinese_chars": true,
        "strip_accents": null,
        "lowercase": true
      }
    ]
  }
}

Note: There is a known issue where the Replace normalizer expects the value of pattern to be a regex and not an object with the regex. It'll  be fixed soon.
Pretokenizer
This step splits the input text into tokens. You can use Whitespace for this step since it aligns with your cleaning process(splitting the text based on spaces).
"pre_tokenizer": {
  "type": "Whitespace"
}

Model
Since you used TfidfVectorizer, this handles the tokenization and vectorization, which isn’t directly compatible with tokenizers like WordPiece or Unigram. But TransformersPHP allows for legacy tokenizers with just a vocab so you could try that
"model": {
  "vocab": tfidf_clean.vocabulary_
}

Post Processor and DEcoder
In your own case, ie a text classification task, these are not neccesary. So,
{
...
"post_processor": null,
"decoder": null
}

All these config should happen in your python script after which you save to file. Here's how the final tokenizer looks
import json

# Save the tokenizer configuration
tokenizer_config = {
    "normalizer": {
        "type": "Sequence",
        "normalizers": [
          {
            "type": "Replace",
            "pattern": {
              "Regex": "<.*?>"
            },
            "content": ""
          },
          {
            "type": "Replace",
            "pattern": {
              "Regex": "\\b(" + "|".join(stop_words) + ")\\b"
            },
            "content": ""
          },
          {
            "type": "BertNormalizer",
            "clean_text": true,
            "handle_chinese_chars": true,
            "strip_accents": null,
            "lowercase": true
          }
        ]
      },
      "pre_tokenizer": {
        "type": "Whitespace"
      },
      "post_processor": null,
      "decoder": null,
      "model": {
        "vocab": tfidf_clean.vocabulary_
      }
    }

tokenizer_path = "build/Arvee/tags-classifier/tokenizer.json"
with open(tokenizer_path, "w") as f:
    json.dump(tokenizer_config, f, indent=2)

I hope this helps

This is great info! Thank you because now the tokenizer.json file makes more sense than it did. Unfortunately though, I am still having problems that started requiring me to hack the library specifically in PretrainedTokenizer (see below). I also saw that you don't support Whitespace as a pre_tokenizer so just to move things alone I switched it to WhitespaceSplit, and that things broke because I also didn't have a tokenizer_config.json file, which I had to create as just an empty JSON object.
Here's what happened when I fixed up the .json files and tried to run the model and the code I had to change before I got stuck:
line 382: $tokens2 was null so I had to change it to:
new PostProcessedOutput(tokens: array_merge($tokens, $tokens2 ?? []));

line 306: $maxLength was null ... I was able to get past the error by passing in a max length calculated from the max length of sequences (e.g. max(array_map(fn($x) => count($x['input_ids']), $encodedTokens)), but not sure if that was even the right fix :))
That all ended up with me still getting this exception:
 Codewithkyrian\Transformers\Exceptions\MissingModelInputException  The following model inputs are missing: float_input.
With all of this in mind is part of the issue we're having here that I'm just doing things in a kind of weird and odd way when I make my model in the first place (that's what this seems like -- e.g. using tfidf which you described as "legacy"). At the end of the day what I am most interested in here is simply training a model to accept some text input from the user and assign it N relevant tags (including zero). If there is a faster approach that is more compatible with the way Transformers wants to deal with things I'd be willing to do that too (as long as I can find a good example to work off from). Would you happen to have good suggestions on an less-legacy way to build my model that would also be a lot easier to work with in your library?

Wanted to give an update here..
I went back to the beginning and rather than building my own model from scratch, I fine tuned a BERT model with my data set using PyTorch ... from there it was pretty easy to convert into an ONNX and when I used my new model things went a LOT easier for me.... based on the code you provided I was able to get the model working with code such as the following:
            $modelObj = AutoModel::fromPretrained($model);
            $tokenizer = AutoTokenizer::fromPretrained($model);

            $id2label = $modelObj->config->config['id2label'];

            $modelInputs = $tokenizer->tokenize($input, padding: true, truncation: true);
            $outputs = $modelObj($modelInputs);

            $retval = [];

            $scores = $outputs['logits']->toArray()[0];

            foreach($scores as $id => $score)
            {
                if($score > 0)
                {
                    $retval[] = $id2label[$id];
                }
            }

            return $retval;

I couldn't figure out how to use the pipeline directly as you suggested -- if there is an easier way to get from scores to the labels baked into the code base already that I can use please let me know.
That said, I got it working!

I was going to suggest fine tuning a smaller Bert model like Albert or Distilbert but seems you've taken care of that. Nice!
In that case, provided you saved the configs et al, the pipeline helper function is super compatible with it as shown on the docs. Did you try that?

Yes, but the problem I ran into is I'm looking for multi-label outputs... so for example when I try:
$classifier = pipeline('text-classification', $model);
$classifier($input, multiLabel: true);

I get one result when I should get two. When I try zero-shot, it throws an exception:
   TypeError  str_replace(): Argument #2 ($replace) must be of type array|string, null given.

--
 () at vendor/codewithkyrian/transformers/src/Pipelines/ZeroShotClassificationPipeline.php:99

Honestly it's not a big deal for me that I had to use a more verbose path vs. just using the pipeline helper -- I just cleaned up my outputs with a sigmoid and applied a threshold of > 0.5 which worked well -- probably shouldn't throw that exception though when I tried to use it as a zero-shot :)

I'm trying to understand what you mean by multi-label outputs. Normally, the pipeline returns only the top label. If you'd like to return the scores for all labels, or a specific number of labels, you can pass a topK argument into the pipeline invocation as indicated in the docs.
$classifier($input, topK: 3);

Setting topK to -1 returns all labels and their scores.
Regarding multi-label classification in text classification, it works similarly to the multiLabel of zero-shot classification. For the classification pipeline to use multi-label, the code looks for a problem_type key in the model config set to multi_label_classification.
You may wonder why I didn't expose a multiLabel argument similar to the zero-shot classification pipeline to force multi-label classification when invoking the model. The reason is that zero-shot classification models are generally more versatile and handle classifying based on labels they were not explicitly trained to use. Not all models are suitable for zero-shot classification. For example, your model wouldn't work for zero-shot classification since it lacks the necessary configurations.
Text classification models are mostly fine-tuned for a specific set of labels. If they are to be used for multi-label scenarios, they require additional training to function properly in such contexts, which is why this has to be indicated in the model config. While it's possible to expose the multiLabel argument to the text classification pipeline, forcing it on models not fine-tuned for multi-label classification could lead to performance degradation.
Actually, if you look into TextClassificationPipeline, the only difference for multi-label is that I use a sigmoid function to scale the probabilities between 0 and 1, while for single-label, I use softmax. I hope this explanation clarifies things for you!


Lazy loading not working

When using SanityImage I sometimes only get the blurry preview image and never the real image.
It seems to be the same or very similar issue as https://github.com/gatsbyjs/gatsby/issues/21932 which is closed with no solution.
To replicate please go to https://nifty-jepsen-2ac991.netlify.app and click "past projects" in the menu. On other pages lazy loading seems to work, but not on the past projects screen. Reloading the page triggers image loading. Same with manually removing the loading="lazy" attribute in dev tools.
Dependencies from package.json

    "@sanity/block-content-to-react": "^2.0.7",
    "babel-plugin-styled-components": "^1.12.0",
    "dotenv": "^9.0.0",
    "gatsby": "^3.4.1",
    "gatsby-image": "^3.9.0",
    "gatsby-plugin-anchor-links": "^1.2.1",
    "gatsby-plugin-image": "^1.4.1",
    "gatsby-plugin-manifest": "^3.4.0",
    "gatsby-plugin-react-helmet": "^4.4.0",
    "gatsby-plugin-sanity-image": "^0.8.0",
    "gatsby-plugin-sharp": "^3.4.2",
    "gatsby-plugin-sitemap": "^4.0.0",
    "gatsby-plugin-styled-components": "^4.4.0",
    "gatsby-source-filesystem": "^3.4.0",
    "gatsby-source-sanity": "^7.0.4",
    "gatsby-transformer-sharp": "^3.4.0",
    "normalize.css": "^8.0.1",
    "react": "^17.0.1",
    "react-dom": "^17.0.1",
    "react-helmet": "^6.1.0",
    "react-slick": "^0.28.1",
    "styled-components": "^5.3.0"

This is covered in the README, though it might not be clear.

⚠ Minor gotchas with deferred loading
SanityImage is relying on browser-native deferred image loading. This generally works fine in browsers that support it, but there are situations where the unloaded image is hidden or covered, resulting in the full image never loading.
If this happens, you can override the styles set on the full-size image using the img[data-loading] selector. This image sits immediately adjacent to the spaceball image and has the following default styles while loading:
position: absolute;
width: 10px !important; /* must be > 4px to be lazy loaded */
height: 10px !important; /* must be > 4px to be lazy loaded */
opacity: 0;
zindex: -10;
pointerevents: none;
userselect: none;

If you inspect the DOM on your website you'll see that your div.slick-list element has overflow: hidden on it. Because of this, the full-size image we are trying to lazy-load is not visible in the viewport, so the browser never attempts to load it.

To resolve this, reposition the img[data-loading] element. For example, setting bottom: -20px should do the trick in this scenario.

Hi Corey! Thank you so much for your quick reply. Apologies for not spotting the note in the readme.
I am actually doing the exact same thing without issues on another site (Gatsby + Sanity + Slick), but with an earlier version.

To resolve this, reposition the img[data-loading] element. For example, setting bottom: -20px should do the trick in this scenario.

Maybe I'm being slow, but I didn't actually understand the solution you mentioned. Hiding the preview image completely like this works, and I understand why:
.galleryImage {
    img {
      &:not([loading]) {
        display: none;
      }
    }
  }

However positioning the image while loading (&[data-loading] in my example) does not seem to work and I don't actually understand why it would. Wouldn't the preview still be in front of it, causing the exact same problem?
If you have a second to clarify I would appreciate it and I'm sure for anyone else using the very popular Slick it would be helpful as well.
Thanks!

The full-size image is outside of the bounds of .slick-list and therefore clipped. Instead of hiding the preview image, just use bottom: -20px to position the full-size image within the visible bounds of .slick-list. I.e.:
.galleryImage {
  img[data-loading] {
    bottom: -20px;
  }
}

(note: your example is missing the data- prefix on the attribute)
This works because img[data-loading] already has position: absolute set and comes later in the DOM, so is therefore drawn above the preview image (which does not have position: absolute or position: relative set). Further, the .slick-slide ancestor has position: relative set, creating a new positioning context that matches the bounds of the .slick-list element. This means that setting bottom: -20px on img[data-loading] will position it 20px above the bottom of the .slick-list container, which is in the view. I tested this on your website before my initial response and it worked, but if you did not have success with it I'd love to know why or have a chance to diagnose myself.


fix: back button in add account screen terminates app

Description
When the user taps on add account from the drawer, we were calling the configurations_screen with Navigator.pushReplacement. Therefore, the app terminates when back-pressed from the add account screen.  So, I have updated it and added an AppBar to show the back button with 0 elevations to match UI.
Screenshot:

@harchani-ritik PTAL and suggest changes if any.

So, the second account comes on top of the first account in the navigation stack. Is this a viable behaviour? OR we could pop off the configurations screen and then replace the first account with the second account. @tusharojha @pratikbaid3 suggest, please.

I think the option to pop off the configuration screen and then replace the account suits better as this would probably be the desired action from the users perspective.

Although, we can replace the initial account with the new one but I think in that case we would not call it as "add account" and there will be no use of showing the list of accounts inside the drawer.
Also, please let me know if I am misunderstanding the thing.

Ah, no brother, the "add account" feature is similar to adding a google account in the Gmail app. So, after a user adds another account the expected behaviour of the back button will be to close the app rather than navigating to the previous account which will be the case with your changes.

Got it!
Consider a case in which user taps on the add account button but doesn't login to another account and tries to go back. In that case rather than returning to homescreen, the app will close.

Yes, I guess the issue is made for handling this case only. Instead of replace, you push the conf screen on top of it. But when a user logins, it should pop off the conf screen and then replace the first account with the second account. I hope I am able to convey the requirements for this issue.

Yes I get it!
I will push new commits to this PR addressing the above discussion.

@harchani-ritik I have made the change. PTAL.
Now, if the user presses the back button from add account screen he/she will be navigated to homescreen rather than closing the app. If the user logs into the account successfully then also, the user gets navigated to homescreen with the default account switched to the newly added.


Fix Travis Build as it's failing

Build now passes, removed failing tests. Tests (or lack of) need to be sorted out but haven't had time.