ISPN-5607 Preemptively invalidate near cache after writes

https://issues.jboss.org/browse/ISPN-5607
[Master ONLY!]

Near cache stale reads can't be fully guaranteed since there's always
the possibility of a read to come in when the server has already
executed a write operation but it's in process of sending back the
response.
However, we can't at least guarantee that within a single
thread, a read after a write will read the written value. The change
proposed in this PR addresses this, by preemptively invalidating data
after a successful write.

@wburns This should be fixed now

pulling

merged, thanks @galderz


Introduce initial empty vscode extension

This will introduce initial empty vscode extension created from yo tools.
ONE-vscode-DCO-1.0-Signed-off-by: SaeHie Park<EMAIL_ADDRESS>

From draft #14

I think package-lock.json file should be added as well. Please see https://github.com/Samsung/ONE-vscode/pull/17/files#r698430764.

I'll rebase this after #17 lands

Added package-lock.json file too. Please take a look again :)


Description of the cmp operator seems weird

In this section: https://docs.perl6.org/type/Order#infix_cmp
it says "Coerces it's arguments to Stringy."
However, in the other section: https://docs.perl6.org/routine/cmp#(Operators)_infix_cmp
it says "Compares strings with string semantics, numbers with number semantics, Pair objects first by key and then by value etc."
Which one is the correct description ?

From looking at the code in Rakudo (See this link) it looks like it will primarily use the stringy form to compare it's arguments, but will use the same code that <=> does when it gets an Int type.
From this it looks like the description given in the Operators section of the docs is correct, and the description in the Order type needs to be changed.


Menu, autocomplete, and datepicker should remove CSS classes from inert panel element

If you have
<md-menu #options class="borange">
  ...
</md-menu>

The css classes are copied to the panel element inside the overlay. However, the classes also remain on the empty <md-menu> element in the page. We should remove all classes from the md-menu element.
This behavior should apply to menu, autocomplete, and datepicker.

Does this affect md-basic-chip as well? I had a similar problem when using class with md-basic-chip but worked fine when ng-class was used instead.
Issue


Memory issues on 0.5.2?

Several users are reporting issues with LND 0.5.2.
CPU and memory overuse caused by errors in LND and NBXplorer since 0.5.2. (see https://github.com/btcpayserver/btcpayserver/issues/605)
I personally did not encountered this problem though. Is there a commit around which would fix this?

Might be related to https://github.com/lightningnetwork/lnd/issues/2551

Nothing really changed much in 0.5.2...however are these people also running bitcoind 0.17? Perhaps something changed with the RPC or ZMQ connections to cause the ZMQ connection to drop more often?

I am not sure I will follow #2658 and #2686. It might be related.

@NicolasDorier I posted a screenshot on BTCPay Server chat. I was experience 6GB RAM usage with nbxplorer.
I ran btcpay update as suggested, and seems to have fixed the issue. So thought I would share this here if others find themselves on this issue.

@ccdle12 this is unrelated. The problem I am reporting is high RAM consumption from LND.

This is likely due to the massive increase of the mainnet channel graph. We are aware of the issue. Closing as duplicate of #2658.


[list-item][media-list] Caret shown at wrong position

This is a (multiple allowed):

[x] bug

[ ] enhancement

[ ] feature-discussion (RFC)

Framework7 Version: 2.0.10

Platform and Target: iOS

Live Link or JSFiddle: https://framework7.io/kitchen-sink/
choose 'List View' and scroll all the way down where media-lists are shown

Expected Behavior
Caret (at the right) should be positioned in the vertical center of the list-item
E.g. as background-image in

Actual Behavior
Caret is placed as background-image in

This also results in the fact, that a media-list's item-text is a bit too wide
Here's a screenshot taken from https://framework7.io/kitchen-sink/

This is by design, assuming media list needs more space for text. This is how native iOS mail looks

Imo Mail is not a good example, because it is not really a media-list, but a mashup.
If you go to settings and watch a true media-list, you'll see the caret is centered:

Here also the item-after is centered as well. Anyway, this won’t be changed, because it can be a breaking change. It is easier to apply custom styles for single app to get the desired effect.
I will add some additional class for list to have the chevron in centered

OK - great thanks!

Great!
P.S. I had already reported it about a year ago... #1358


Reader HID multiclass SE RP10 not work

with forV2Board Date 29.11.2019 the reader works as Wiegand - only webinterface is not stayable.
with update to newer build forV2Board or generic.bin i cant get the reader to work. Same Pinout same config - it wont work.
Reader Type= Wiegand
Pin D0= GPIO-5  green cabel from reader
Pin D1= GPIO-4  white cable from reader
Pin code request= false
WiFi LED= GPIO-2
Door Status Pin= GPIO-16 official board
if i change back to old version it works again - with newer one it wont read any badge.
any idea why ??
(testet on 3 official green boards)
br rupert

Hey @stupsi099 could you please try using the latest dev build instead? You can find it here: https://github.com/esprfid/esp-rfid/actions/runs/3404792413 at the bottom of the page.
Let me know if that helps! Thanks.

did not help - not work with the new version.
i set wlan to the internal led and can see the esp8266 is rebooting if you go to the log tab and switch between access and log tab...
-> i see fetching data -> reboot -> in log i see reboot!
14 | INFO | sys | System setup completed, running |   | 1
13 | INFO | wifi | WiFi is connected | WLanZ1 <IP_ADDRESS> | 1
12 | INFO | door | Door Closed |   | 2
11 | INFO | sys | System setup completed, running |   | 2
10 | INFO | wifi | WiFi is connected | WLanZ1 <IP_ADDRESS> | 2
9 | INFO | websrv | Login success! | <IP_ADDRESS> | 2022-11-06-18:59:23
8 | INFO | door | Door Closed |   | 11
7 | INFO | sys | System setup completed, running |   | 11
6 | INFO | wifi | WiFi is connected | WLanZ1 <IP_ADDRESS> | 11
5 | INFO | sys | System is going to reboot |   | 2022-11-06-18:59:06
br rupert

thank you for the feedback, I'm going to check

i try this build "Add pin code only option to open door #71 platformio
Option: RFID Card without asking Pin code
not working!! (users -> add card -> nothing, did not read)
br rupert

with platformio #104 same problem.
card wont read  - my config if i did anything wrong with original green board.
},
"hardware": {
"readertype": 1,
"wgd0pin": 5,                                   green cable
"wgd1pin": 4,                                   white cable
"sspin": 0,
"rfidgain": 32,
"wifipin": 2,
"rtype": 1,
"ltype": 0,
"rpin": 13,
"rtime": 400,
"beeperpin": 255,
"ledwaitingpin": 255,
"openlockpin": 255,
"doorbellpin": 255,
"accessdeniedpin": 255,
"useridstoragemode": "decimal",
"requirepincodeafterrfid": false,
"allowpincodeonly": false,
"doorstatpin": 255,
"maxOpenDoorTime": 0,
"numrelays": 1,
"relay2": {
"rtype": 1,
"ltype": 0,
"rpin": 4,
"rtime": 400
},
"relay3": {
"rtype": 1,
"ltype": 0,
"rpin": 4,
"rtime": 400
},
"relay4": {
"rtype": 1,
"ltype": 0,
"rpin": 4,
"rtime": 400
}
},
"general": {
"hostnm": "esp-rfid",
"restart": 0,
"pswd": "-----",
"openinghours": [
"111111111111111111111111",
"111111111111111111111111",
"111111111111111111111111",
"111111111111111111111111",
"111111111111111111111111",
"111111111111111111111111",
"111111111111111111111111"
]

Hey @stupsi099, just checking, are you referring to this build here: https://github.com/esprfid/esp-rfid/actions/runs/3980992874 ?
Anyway thank you for your settings, I'm going to check soon

yes i use the last one #105 from pfatformio

with the old one from the release (29.Nov.2019) it works with this pinout and reader.

newer did not scan the card. the reader read but no output on the "user tab" to add a new user

br rupert

Gesendet: Montag, 23. Januar 2023 um 20:29 Uhr
Von: "Matteo Giaccone" @.>
An: "esprfid/esp-rfid" @.>
Cc: "stupsi099" @.>, "Mention" @.>
Betreff: Re: [esprfid/esp-rfid] Reader HID multiclass SE RP10 not work (Issue #542)

Hey @stupsi099, just checking, are you referring to this build here: https://github.com/esprfid/esp-rfid/actions/runs/3980992874 ?
Anyway thank you for your settings, I'm going to check soon
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

Sorry, one more question, when you say that the reader reads, do you see anything on the serial logs? Or the reading is just a "beep" that you hear?
I don't have an idea on what's wrong on the new build and unfortunately I don't have the reader that you have to test :(
If you have a spare one to ship I'm more than happy to fix it for you, if you don't have any better option!

Beep from reader. But its internal - i have spare reader! You can have one

Cool, feel free to send me an email to my public email (on Github profile) and I'll let you know my address. Thanks!

Hey @stupsi099 have a go with this: https://github.com/esprfid/esp-rfid/pull/582
Thank you very much for sending over the reader, I've learnt new stuff around Wiegand readers :)

now the reader works with all cards i have 125khz, 13.xMhz Desfire and old one.
but there are strange ID`s when i read the badge/fob
old version:<PHONE_NUMBER>         new software: 959324802             Desfire EV2 8k        Serial 04:45:5C:F2:E4:5E:80
if i switch to hex
new software: 3b3e7782    Mifare Classic 1k   Serial 04:EF:7C:76
back to dec
new software: PICC Type 32    1816220      125kHz Fob       Serial<PHONE_NUMBER>   (-50%)
old version: PICC Type 20      3632440                  same as on the chip
(every card on old version are PICC Type 20 - on new version are PICC Type 32)
-> but all cards readed by new software!!! -> problem is only wrong ID. ->by 125kHz Chip its 50% from the number :-)
hope it helps
br rupert

new version works now same as old version. All cards accepted and readed.
thx for your work
br Rupert

Thank you very much for your help! I've learned something new and important :)


add django-dynamic-preferences

closes #240
Todo:

[x] user notification settings for responsible things
[x] settings on EventType (defaults for visible_for, responsible_users and responsible_groups)
[x] hide fields on event creation
[x] views for global settings
[x] views for event types
[x] think about another location for user settings? aka in the profile
[x] only display the responsible settings for users that are responsible
[x] think about splitting the user settings by section (notification settings, other settings)
[ ] fix design of "advanced settings" section for event creation/update
[ ] write some tests (yay)

Coverage increased (+1.03%) to 81.214% when pulling 565d5461c34daed8cfb5fc689a5bb6502fef56c1 on feature/settings into 89cb43107e98b6b6ae2ae9bebddea95f95161d4d on main.


updated PreciseNode

I don't know why this failed, works for me.

Jenkins is using a strictly enforced KSP version of 1.0.0 so he fails this since he cannot use the supplied .ckan file nor can he find a version that works for 1.0.0.
Is this precisenode version strictly locked to 1.0.2 or would it also work with 1.0.x? If the latter is the case maybe consider changing the ksp_version to 1.0 as outlined here

It's hard to tell according to change log
1.1.3, 2015-05-02
- Fixed NullReferenceException in map mode.
- Updated for KSP 1.0.2.

1.1.2, 2014-12-16
- Updated for KSP 0.90.0.
(...)

I'll change ksp_version to 1.0
Anyway I think Jenkins behaviour should be tweaked :)

I'm considering switching Jenkins to a 1.0 install but I'm not sure that's better or worse than 1.0.2

Dunno, Jenkins side of ckan is magic for me :)

Me to but you do pick things up here and there so I'm trying to learn as to not be handicapped when stuff like this happens :p

Meh, the same problem with Kerbal Alarm Clock :C I'll open an issue


kube-router SDN not working

Bringing the discussion from #3 to a proper issue.
The error with seen with kube-router is:
ubuntu@osh-sh-ci-01:~$ kubectl logs  kube-router-hkf2d -n kube-system
panic: nodes "kubernetes" not found

goroutine 1 [running]:
panic(0x1596120, 0xc42040a450)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/cloudnativelabs/kube-router/app/controllers.NewNetworkPolicyController(0xc420315540, 0xc420314960, 0x0, 0x0, 0x0)
	/home/kube/go/src/github.com/cloudnativelabs/kube-router/app/controllers/network_policy_controller.go:785 +0x413
github.com/cloudnativelabs/kube-router/app.(*KubeRouter).Run(0xc4203c1660, 0xc4203c1660, 0x0)
	/home/kube/go/src/github.com/cloudnativelabs/kube-router/app/server.go:120 +0x710
main.main()
	/home/kube/go/src/github.com/cloudnativelabs/kube-router/kube-router.go:37 +0x13c
ubuntu@osh-sh-ci-01:~$

kube-router upstream has been changed to hopefully support anything for a nodeName with kube-router run as a pod.  So you might want to docker pull it again and try it out.  I would try it out but I haven't had time to deploy any Ubuntu servers or VMs yet.

sorry took me so long to get back to this @bzub...was (and am) in the process of moving to Denver, CO. i'm on break this week and should be able to test this, and get back to you. thanks for resolving some of those issues upstream though!

No problem @v1k0d3n. I also hope to try this out soon.
By the way, I think you should try the CNI configuration with hairPinMode set to true.  I'm still testing this out but so far it seems to fix some issues with pods talking to master components on the same host.  After testing it will go into the kube-router repo officially.
So it would like:
{
  "name":"kubernetes",
  "type":"bridge",
  "bridge":"kube-bridge",
  "hairpinMode": true,
  "isDefaultGateway":true,
  "ipam": {
      "type":"host-local"
  }
}


[docs] Shift aliasing responsibilities from babel to webpack

Move aliasing responsibilities from babel to webpack. Leverage the webpack resolver aliasing capabilities. Webpack is more suitable for this task as it provides the modules runtime. It acts as a linker, so it should be in charge of the resolution process.
Let next-transpile-modules compile workspace dependencies

This avoids the need to specify the aliasing again for transpiled modules.
In an ideal world we'd avoid aliasing altogether. In Toolpad we avoid them by building and watching each of the workspaces separately.

What about scripts executed with babel-node? There is a couple of them that import from @mui/ scoped packages. They may need the aliases defined in Babel config.

@michaldudak FYI this PR may be superseded by https://github.com/mui/material-ui/pull/35197

Superseded by https://github.com/mui/material-ui/pull/40792/files


Duplication BUG

This bug is appearing in many places site-wide. See:

[ ] Duplicate methods in sidebar https://github.com/participedia/usersnaps/issues/200
[ ] Duplicate components in sidebar https://github.com/participedia/usersnaps/issues/199
[x] Duplicate location in sidebar https://github.com/participedia/usersnaps/issues/163
[x] Duplicate bookmarks on profile page https://github.com/participedia/usersnaps/issues/167

Duplicate method in results after editing (original AND new edited version shown in results) https://github.com/participedia/usersnaps/issues/210

fix for the methods/components duplication issues https://github.com/participedia/frontend/pull/913


check for returned error

Signed-off-by: MattCosturos<EMAIL_ADDRESS>Description
Check the return value of performTableDissemination for errors
Issue reference
Please reference the issue this PR will close: #7031
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Unit tests passing
[ ] End-to-end tests passing
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Specification has been updated / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]
[ ] Provided sample for the feature / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

@holopin-bot @MattCosturos


npm install stop working

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type:   bug
Enter Task Name: npm install
Environment

Server - Azure Pipelines

Agent -  window 2019

Issue Description
The task npm install stop working. I got the error below. it is not always. So if I run once it will succeed but if I run the pipeline again the task will fail.
I have two steps
// Clean the cache because I thought the problem was caused by a cache data
steps:

task: Npm@1
displayName: 'npm custom'
inputs:
command: custom
verbose: false
customCommand: 'cache clean --force'

// I was using the npm install but with this one always fail.
steps:

task: Npm@1
displayName: 'npm custom'
inputs:
command: custom
workingDir: src
verbose: false
customCommand: 'clean-install --force'

Task logs
2020-06-14T21:48:38.3747727Z ##[section]Starting: npm custom
2020-06-14T21:48:38.3888690Z ==============================================================================
2020-06-14T21:48:38.3889004Z Task         : npm
2020-06-14T21:48:38.3889372Z Description  : Install and publish npm packages, or run an npm command. Supports npmjs.com and authenticated registries like Azure Artifacts.
2020-06-14T21:48:38.3889754Z Version      : 1.169.2
2020-06-14T21:48:38.3889971Z Author       : Microsoft Corporation
2020-06-14T21:48:38.3890277Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/package/npm
2020-06-14T21:48:38.3890786Z ==============================================================================
2020-06-14T21:48:39.2542528Z SYSTEMVSSCONNECTION exists true
2020-06-14T21:48:39.2549149Z SYSTEMVSSCONNECTION exists true
2020-06-14T21:48:40.2559871Z [command]C:\windows\system32\cmd.exe /D /S /C ""C:\Program Files\nodejs\npm.cmd" --version"
2020-06-14T21:48:40.2560664Z 6.14.4
2020-06-14T21:48:41.8345322Z [command]C:\windows\system32\cmd.exe /D /S /C ""C:\Program Files\nodejs\npm.cmd" config list"
2020-06-14T21:48:42.5961410Z ; cli configs
2020-06-14T21:48:42.5962021Z metrics-registry = "https://registry.npmjs.org/"
2020-06-14T21:48:42.5962377Z scope = ""
2020-06-14T21:48:42.5963050Z user-agent = "npm/6.14.4 node/v12.17.0 win32 x64"
2020-06-14T21:48:42.5963370Z
2020-06-14T21:48:42.5964176Z ; environment configs
2020-06-14T21:48:42.5964559Z cache = "C:\npm\cache"
2020-06-14T21:48:42.5964949Z prefix = "C:\npm\prefix"
2020-06-14T21:48:42.5965290Z userconfig = "D:\a\1\npm\12907.npmrc"
2020-06-14T21:48:42.5965518Z
2020-06-14T21:48:42.5966148Z ; builtin config undefined
2020-06-14T21:48:42.5966595Z
2020-06-14T21:48:42.5967085Z ; node bin location = C:\Program Files\nodejs\node.exe
2020-06-14T21:48:42.5967440Z ; cwd = D:\a\1\s\src
2020-06-14T21:48:42.5967827Z ; HOME = C:\Users\VssAdministrator
2020-06-14T21:48:42.5968058Z ; "npm config ls -l" to show all defaults.
2020-06-14T21:48:42.5968208Z
2020-06-14T21:48:42.5969009Z [command]C:\windows\system32\cmd.exe /D /S /C ""C:\Program Files\nodejs\npm.cmd" clean-install --force"
2020-06-14T21:50:52.3276932Z npm WARN using --force I sure hope you know what you are doing.
2020-06-14T21:50:52.3277995Z npm WARN tarball tarball data for<EMAIL_ADDRESS>(sha512-CDwa3FJd0XJpKDbWCST484H+mCNjF26dPrU+xnREW+upR0UODjMEfXPl3bxWuAwZIX6c2ASg1plLO7jP8ehWeA==) seems to be corrupted. Trying one more time.
2020-06-14T21:50:52.3278646Z npm ERR! code EPERM
2020-06-14T21:50:52.3278934Z npm ERR! syscall unlink
2020-06-14T21:50:52.3279403Z npm ERR! path D:\a\1\s\node_modules\npm\man\man1\npm-publish.1
2020-06-14T21:50:52.3279905Z npm ERR! errno -4048
2020-06-14T21:50:52.3280377Z npm ERR! Error: EPERM: operation not permitted, unlink 'D:\a\1\s\node_modules\npm\man\man1\npm-publish.1'
2020-06-14T21:50:52.3281139Z npm ERR!  [OperationalError: EPERM: operation not permitted, unlink 'D:\a\1\s\node_modules\npm\man\man1\npm-publish.1'] {
2020-06-14T21:50:52.3282143Z npm ERR!   cause: [Error: EPERM: operation not permitted, unlink 'D:\a\1\s\node_modules\npm\man\man1\npm-publish.1'] {
2020-06-14T21:50:52.3282800Z npm ERR!     errno: -4048,
2020-06-14T21:50:52.3283119Z npm ERR!     code: 'EPERM',
2020-06-14T21:50:52.3283439Z npm ERR!     syscall: 'unlink',
2020-06-14T21:50:52.3283857Z npm ERR!     path: 'D:\a\1\s\node_modules\npm\man\man1\npm-publish.1'
2020-06-14T21:50:52.3284301Z npm ERR!   },
2020-06-14T21:50:52.3284544Z npm ERR!   errno: -4048,
2020-06-14T21:50:52.3284872Z npm ERR!   code: 'EPERM',
2020-06-14T21:50:52.3285211Z npm ERR!   syscall: 'unlink',
2020-06-14T21:50:52.3285613Z npm ERR!   path: 'D:\a\1\s\node_modules\npm\man\man1\npm-publish.1'
2020-06-14T21:50:52.3286026Z npm ERR! }
2020-06-14T21:50:52.3286250Z npm ERR!
2020-06-14T21:50:52.3286612Z npm ERR! The operation was rejected by your operating system.
2020-06-14T21:50:52.3287680Z npm ERR! It's possible that the file was already in use (by a text editor or antivirus),
2020-06-14T21:50:52.3290941Z npm ERR! or that you lack permissions to access it.
2020-06-14T21:50:52.3291209Z npm ERR!
2020-06-14T21:50:52.3291552Z npm ERR! If you believe this might be a permissions issue, please double-check the
2020-06-14T21:50:52.3292051Z npm ERR! permissions of the file and its containing directories, or try running
2020-06-14T21:50:52.3292464Z npm ERR! the command again as root/Administrator.

Greetings!
We have been having a similar issue since Friday. Pipelines that completed successfully the day before, fail w/identical configuration and code with this error:
npm WARN tarball tarball data for<EMAIL_ADDRESS>(sha512-CDwa3FJd0XJpKDbWCST484H+mCNjF26dPrU+xnREW+upR0UODjMEfXPl3bxWuAwZIX6c2ASg1plLO7jP8ehWeA==) seems to be corrupted. Trying one more time.

Yes, the same happens to us.

Hi, everyone,
the way I fix the problem was not using npm to install packages. I am using yarn.
steps:

powershell: 'yarn install  --network-timeout=30000'
displayName: 'yarn install'

Hi,
For this issues, we encourage you do

NpmAuthenticate
script: call npm install-clean --force || call npm install-clean --force

For more info,  See docs  (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/package/npm-authenticate?view=azure-devops)

Thanks for the info. I decided to move to yarn.


Removing unused exec service.

Description
As part of refactoring executor service inside 'ControllerTask` is not longer required. Removing it.

Key changed/added classes in this PR

MSQControllerTask

This PR has:

[x] been self-reviewed.

LGTM (non-binding)


FIX-#1263: Fix 'monotonic' implementation to allign new MapReduce beh…

…avior
Signed-off-by: Dmitry Chigarev<EMAIL_ADDRESS>
What do these changes do?

[x] commit message follows format outlined here
[x] passes flake8 modin
[x] passes black --check modin
[x] signed commit with git commit -s
[x] Resolves #1263
[x] tests passing

The thing that was changed from the previous implementation is that now instead of a bool scalar we return DataFrame with a single column named the same as the Series.

@devin-petersohn we already can use window since there is a .rolling function in the Series API, the problem is that it seems that there is no standard function that allows us to determine is window monotonic or not, we can do it only with .apply by passing a lambda function that will compute it, but not all of the backends can execute UDFs python functions, so we can't do it this way. Looking forward, any implementation of the Window operator does not allow us to do this, since we forced to pass a lambda that determines that the window is monotonic.

@dchigarev I think I was not clear about implementation details.
is_monotonic_increasing and is_monotonic_decreasing should be in the query compiler. From here, those functions should call the window algebraic operator in the Modin DataFrame. This algebraic operator has not been implemented yet, and we have been using apply_full_axis for it.
So should we wait until the window algebraic operator is available from Modin dataframe (which will be faster than apply) or implement with apply immediately? Do you have an opinion?

@devin-petersohn what is the point then of using a rolling window if we can use MapReduce? I think that it would perform even slower than MapReduce since we need to apply is_monotonic to a 2*(len(ser)-1) amount of windows.
The current approach applies is_monotonic to every partition at the map phase and then do all on the results at reduce phase + apply is_monotonic to all of the edge elements of partitions that were memorized at the map phase. What's wrong with that logic and why do we need to replace it with a rolling window, that may involve much more computations?

The main point is the correctness of doing window by hand with mapreduce vs a more general approach. The implementation of window will likely be very similar, but it will not require you to handle the corner cases as you do here, rather you can just call the algebraic operator with the is_monotonic function.
If you would like, you can generalize your approach here and we can add it to the Modin Dataframe layer. The window size will be important so we know how to handle the corner cases of window. We will want to reuse the window functionality for the other window functions for performance.

@dchigarev what is the status of this PR? Should we close or do we plan to restart development?


Better time zone workflow for JS target

Right now Time_zone.local attempts construction using name guesses (strings) from Timedesc_tzlocal.local,
but this would fail if tzdb.none backend is picked
Run time retrieval of tzdb json file without being typed to a Lwt promise was not possible iirc,
so perhaps one way forward is to expose the string guesses as an alias (Timedesc.Time_zone.local_string perhaps).

@glennsl Any thoughts?

Hmm, I'm not sure where that would get us. Once you have the guesses, what would you do with them?
Also, as I mentioned in #42, it is possible to query the JS environment's time zone database using Intl.DateTimeFormat, but that requires abstracting away the lookup table as you have to query the offset for every timestamp. And I'm not sure if this would be too limited for how the the time zones are currently used.

This would allow loading the appropriate JSON file of the time zone, instead of loading every single time zone in tzdb.

Ah, so off-loading the async work to the consumer. Yeah that would be a nice option to have. It does have some significant architectural repercussions though, as the consumer will either have to pass this time zone around everywhere, or somehow deal with the possibility that the local timezone is not yet known.

That would likely be too slow at least for timere, correct.

It's definitely a trade-off, but apparently not bad enough that the successor of the most widely used JS date time library makes any note about a noticeable performance impact in its documentation. It would be a nice option to have, and also nice to have some numbers to evaluate alternatives.

For timedesc, it may just be easier to ask user to construct a JS time and convert into Timedesc.t, if desirable at all.

Then you'll just have a fixed offset timezone with all the problems that come with that, right?
A third option might be to offer a reduced time zone database without all the historical baggage. Spacetime manages to get down to 47kB that way, which is quite acceptable. Compared to moment-timezone at 479 kB and date-fns-timezone at 922 kB. And compared (unfairly) to timere's own unminifed full tzdb at 5.4 MB.

It's definitely a trade-off, but apparently not bad enough that the successor of the most widely used JS date time library makes any note about a noticeable performance impact in its documentation. It would be a nice option to have, and also nice to have some numbers to evaluate alternatives.

Timere needs access to the (full) transition table to resolve pattern matching queries.
Timedesc is comparable to Luxon yes, though I'll have to think about how this would work out - we'll have to replace the stored transition tables with an oracle that interacts with Intl from what I understand.

Then you'll just have a fixed offset timezone with all the problems that come with that, right?

True.

A third option might be to offer a reduced time zone database without all the historical baggage. Spacetime manages to get down to 47kB that way, which is quite acceptable. Compared to moment-timezone at 479 kB and date-fns-timezone at 922 kB. And compared (unfairly) to timere's own unminifed full tzdb at 5.4 MB.

Approach of Spacetime (https://github.com/spencermountain/spacetime/blob/master/zonefile/iana.js) seems to be just restricting ourselves to the immediate next 12 months-ish, namely for Sydney we have
  "australia/sydney": {
    "offset": 10,
    "hem": "s",
    "dst": "04/03:03->10/02:02"

which is just the next transition (without even year).
Similarly for https://github.com/vvo/tzdb. Both libraries seem assume the application will always load the latest rebuilt library from NPM, and you don't compute anything more than one year into the future.
If that is indeed what suffices for JS usage, I am happy to add a tzdb-js backend that only includes very recently months of data.

Timere needs access to the (full) transition table to resolve pattern matching queries.

I see. Yeah that certainly makes this trickier.

Timedesc is comparable to Luxon yes, though I'll have to think about how this would work out - we'll have to replace the stored transition tables with an oracle that interacts with Intl from what I understand.

Yeah, for Intl it seems to be a function timestamp -> offset.

which is just the next transition (without even the year).

Does the transition really happen at different times every year? Or is it just that there's a (very small) possibility that it will?

If that is indeed what suffices for JS usage, I am happy to add a tzdb-js backend that only includes very recently months of data.

I think it's at least very unlikely that most front-end applications will deal with datetimes before 1970 and after, say, 10 years into the future. So filtering out anything outside that should already reduce the size quite a bit.
For our specific use case, which is quite time-centric, we have some historical data going back to the 90s I believe. But that's just used for machine learning on the back end. On the front end we don't show historical data from before 2015, and only look about a year into the future. And if we were to display historical data somehow it would still just be with dates (or more likely weeks at the smallest granularity), so no offset is really needed anyway.
I can also imagine other formats for the time zone database that would allow more flexibility with a smaller footprint, such as a rule-based format that enables computing the table on demand. A rule could be in the form { from:<PHONE_NUMBER>, to:<PHONE_NUMBER>, offset: 36000, transitions: ["04/03:03->10/02:02"] }. And even that could be compressed further by inferring from or to from the the next or previous entry. But this probably requires both an encoder and decoder, as I assume the source you use for the time zone database is in table form already.

Does the transition really happen at different times every year? Or is it just that there's a (very small) possibility that it will?

For most places afaict, yes - I think government bodies tend to define it around "nth weekday of some month" or something like that? But no strict rule on a specific pattern being followed.
I actually don't know if the date then becomes fixed if we switch to ISO week calendar - I'll have to check.

I think it's at least very unlikely that most front-end applications will deal with datetimes before 1970 and after, say, 10 years into the future. So filtering out anything outside that should already reduce the size quite a bit.

For our specific use case, which is quite time-centric, we have some historical data going back to the 90s I believe. But that's just used for machine learning on the back end. On the front end we don't show historical data from before 2015, and only look about a year into the future. And if we were to display historical data somehow it would still just be with dates (or more likely weeks at the smallest granularity), so no offset is really needed anyway.

Fair enough - sounds like a tzdb-slim (or some better name) will be a good addition.

I can also imagine other formats for the time zone database that would allow more flexibility with a smaller footprint, such as a rule-based format that enables computing the table on demand. A rule could be in the form { from:<PHONE_NUMBER>, to:<PHONE_NUMBER>, offset: 36000, transitions: ["04/03:03->10/02:02"] }. And even that could be compressed further by inferring from or to from the the next or previous entry. But this probably requires both an encoder and decoder, as I assume the source you use for the time zone database is in table form already.

Transitions in the general case cannot be algorithmically generated, might be the case for contemporary periods using ISO week calendar as suggested above  - don't know for certain yet.

I checked and it seems to be the case for Australia/Sydney for 2021/2022 that you can state the split using a fixed ISO week date (varying year), but I don't have time to write code to check if that's always the case for all time zones in recent +/- N years at the moment.

I think government bodies tend to define it around "nth weekday of some month" or something like that?

Ah, of course they do :roll_eyes:

Transitions in the general case cannot be algorithmically generated,

I don't see why not. In the worst case you'd just have a "rule" for every transition, which is effectively status quo (with a bit more overhead). Anyway, I'm not trying to say that the specific rule format I suggested would be optimal, or even better than a table, but just seed the idea that the database doesn't have to be stored as a bulky lookup table, but could be (partly) computed on demand based on some kind of rules. Coming up with the actual rules, format and encoding process is of course still a non-trivial task.

I checked and it seems to be the case for Australia/Sydney for 2021/2022 that you can state the split using a fixed ISO week date (varying year), but I don't have time to write code to check if that's always the case for all time zones in recent +/- N years at the moment.

Promising. Thanks for checking! This is just optimization, and so not very time sensitive, but I might take this on as a fun challenge if and when I get some time.

Another (simpler) idea is to simply pass the table through a compression pass, which might already yield a fair bit of savings.

There's definitely lots of potential for compression, but they ought to be transferred in gzipped form already, and so I don't imagine traditional compression would do much for transfer size. It could help with memory use though, which is especially important on mobile.
It would also be easy to remove quite a bit of bloat just by transforming the entries from
[ "3920576400", { "is_dst": false, "offset": 3600 } ]

to
[<PHONE_NUMBER>, 0, 3600 ]

Re compression: I was primarily thinking of at the marshalled level (so it'd be without the clutter of JSON already).

Right, yeah I had no idea what that process was, so thanks for the context!
Here's an interesting idea. Try running this code on one of the json objects:
{
  var last = 0;
  tz.table.map(([time, {offset}]) => {
    let delta = time - last;
    last = time;
    return [delta, offset];
  })
}

It's not hard to see the recurring patterns, but also that there's a very small set of unique entries. Which means you could encode this in a two-level lookup table. E.g.:
{
  "L1": {
    1: [ 16934400, 3600 ],
    2: [ 14515200, 0 ].
    ...
  },
  "L2": [1, 2, 1, 2, 3, 2, 3, 4, 1, 2, 1, ...]
}

with the entries in L2 indexing into L1. And from this you could easily build the full lookup table on demand.
I imagine you could do much better than this by taking the recurring patterns into account, but this could already reduce the database size by 80% if we're lucky, I think.

Hm...adjusting to a different binary encoding would indeed save us a lot of space, but then the initialisation of timedesc could be significantly slower compared to just unmarshalling the string - I don't know if this is the case.
is_dst is largely for completeness - I don't really recall if it's actively used in the suite, but don't think it's used to any meaningful extent.

Would it be hard to unmarshal on demand instead?

A few examples of the potential effectiveness of this encoding:

Eire has 353 entries, 41 unique.
Australia/Sydney has 267 entries, 25 unique.
Europe/Oslo has 266 entries, 19 unique

Come to think of it, the main table is just a pair of array, so really as efficient as one can be when maximising for both space efficiency and initialisation efficiency.

Would it be hard to unmarshal on demand instead?

The change would be transparent as type Timedesc.Time_zone.db is abstract, so we're free to do the on demand tricks internally.

A few examples of the potential effectiveness of this encoding: ...

Ah right, you're mapping that way...right, we can then use a byte/char as the index, with dynamic unmarshalling...yeah okay this sounds like a good idea.

Yeah okay, this is very doable, just that adding the relevant testing will take some time (on top of me lacking a functional desktop right now for fuzzing).

Cool! Let me know if you want me to pitch in with something. I could for example write the encoding script. I don't know what you use as a source though.

Actually, this would probably be useful for many other projects as well, so perhaps we should set up a separate repository for the time zone database, with ocaml and js packages to encode and decode. That would give it more real world testing, some help with keeping it up to date etc.

Cool! Let me know if you want me to pitch in with something. I could for example write the encoding script. I don't know what you use as a source though.

We need a Timedesc.Time_zone.t -> string serialiser and string -> Timedesc.Time_zone.t deserialiser with roughly the following format
| uniq offset count (int8) | uniq offset #0 in minutes (int16) | uniq offset #1 in minutes (int16) | ...
| timestamp in seconds since unix epoch (int64) | index into uniq offset count (int8) | is DST (int8) | ...

We can just use string String_map.t to represent the encoded db and unmarshal into an entry for Time_zone.Db.entry String_map.t on the fly (or something like that).

Actually, this would probably be useful for many other projects as well, across languages, so perhaps we should set up a separate repository for the time zone database, with ocaml and js packages to encode and decode. That would give it more real world testing, some help with keeping it up to date etc.

Hm...yeah interesting idea - our current representation is copied from Rust chrono-tz crate (I hope I got the name right), I'm curious if they had similar concerns since they could be targeting use cases in WASM.

We can then just use string String_map.t to represent the encoded db and unmarshal into an entry for Time_zone.Db.entry String_map.t on the fly (or something like that).

Sounds like a good plan. I don't quite understand the rationale behind this format though. A lookup table for offsets would offer some savings, but only 50% of the offset field at most, since it's replacing an int16 with an int8. The offset is only 2/7ths the size of the entry as a whole, however, which means it would amount to a 15% reduction in total size at most, even disregarding the overhead of the L1 table.
What I show above is that, assuming is_dst is always in sync with offset, the number of unique entries converted to relative time is only 1/10th the full number of entries. So you could have an L1 table that is 10% the size of the full table, and an L2 table that is 15% the size of the full table (because we replace 7 bytes with a 1 byte index into L1). Which in total is a 75% reduction in size. Quite an improvement from 15%!
That would be something more like this:
| uniq entry count (int8) | delta #0 (int64) | is_dst #0 (int8) | offset #0 (int8) | delta #1 (int64) | ...
| index into uniq entry #0 (int8) | index into uniq entry #1 (int8) | index into uniq entry #2 (int8) | ...

I'm curious if they had similar concerns since they could be targeting use cases in WASM.

Rust WASM is already pretty bloated (I say as my jsoo bundle size just passed 10MB, unminified and whatnot, but still), so my guess would be no :laughing: It's a good question, and definitely something that would be worth looking into. But from a quick search of their issues I can't find anything.

One issue with that is then we'll have to put extra time into making it future proof etc (which uh would exceed my free time budget : D).

Ah, true! Maybe not at first then, but something we could have in mind for the future!

But from a quick search of their issues I can't find anything.

Oh, searched the wrong repo! I found two issues mentioning size, neither of them related to WASM. The solution they focus on seem to be various ways of filtering which timezones are included.

The original time zone database actually seems to be in a rule based format already
Definitely not a simple format though. I think I prefer the compression scheme we've come up with here, even if it ends up a bit bulkier.

What I show above is that, assuming is_dst is always in sync with offset, the number of unique entries converted to relative time is only 1/10th the full number of entries. So you could have an L1 table that is 10% the size of the full table, and an L2 table that is 15% the size of the full table (because we replace 7 bytes with a 1 byte index into L1). Which in total is a 75% reduction in size. Quite an improvement from 15%!

Right relative time, right okay. This requires investigation on the exact upperbound of the gaps...which will need another set of code...though I think 32bit is a safe bet, and we can just make the code gen raise an exception when it's not 32bit.

I was thinking we might just keep using int64, for simplicity. That will also allow using 0 as the starting point while still starting at "the beginning of history" (i.e. the first delta being Int64.min). Otherwise we'll also need a starting time field I think. Going from int64 to int32 will also just yield 1.5 percentage points or so additional reduction, because we've already reduced the total number of timestamps by 90%.

I added the serialisation side (with some more conservative choices in the layout for now), and the size of tzdb_marshalled.ml shrunk from 2.9M to 929K : D (on branch alt-tzdb-encoding)
Deserialisation is not implemented yet.

Swapping from 16bit to 8bit for index shrinks to down to 667K

Made some further adjustment to reduce width of delta and offset, now size is at 485K

The format should be relatively obvious from Time_zone.Compressed_table.add_to_buffer should you want to make further adjustment or roll a deserialiser (personally prefer using angstrom).

That's awesome! Almost 85% reduction in size! I'm surprised the encoding of offset needs to be this involved though, but I guess that make sense for arcane historical reasons.
I see you've added a deserializer too now, so seems you have full control! I continue to be impressed about how fast you manage to make things happen after I suggest something. Thanks so much for this!

That's awesome! Almost 85% reduction in size! I'm surprised the encoding of offset needs to be this involved though, but I guess that make sense for arcane historical reasons.

Yeah, always some oddities in tzdb sadly.
I'll make a different PR for reducing the number of years we include in tzdb (reorganising the tzdb backends a bit).

I see you've added a deserializer too now, so seems you have full control! I continue to be impressed about how fast you manage to make things happen after I suggest something. Thanks so much for this!

: D thanks!
Right now I'm doing the most boring part - testing the (de)serialisation (laptop not very good at running ever so slightly heavy tests...)

Okay finished debugging - quickcheck catches a lot of errors as usual, namely Int64.of_int32 repeats the sign bit (forgot this entirely) and overflow/underflow in some tables.

I see it's been merged now. Nice work! It seems that the whole database is decompressed at the same time though, which means you'll still have to pay the memory and cpu cost of all the time zones even if you most likely just need one of them. This is especially unfortunate on mobile, which is generally more memory and cpu-constrained, but also power-constrained. And just having things in memory consumes quite a bit of power, as I understand it. Would it be possible to have a map with time zones compressed individually, and only decompress them as needed?
Also looking forward to see how much removing historic a far future time zone transitions will affect the size now!

I see it's been merged now. Nice work!

: D

It seems that the whole database is decompressed at the same time though, which means you'll still have to pay the memory and cpu cost of all the time zones even if you most likely just need one of them. This is especially unfortunate on mobile, which is generally more memory and cpu-constrained, but also power-constrained. And just having things in memory consumes quite a bit of power, as I understand it. Would it be possible to have a map with time zones compressed individually, and only decompress them as needed?

Yep that's the intended direction, just wanted to make that a separate PR.

Also looking forward to see how much removing historic a far future time zone transitions will affect the size now!

Indeed, what's a good default you reckon? +- 20 years?

Yep that's the intended direction, just wanted to make that a separate PR.

Awesome!

Indeed, what's a good default you reckon? +- 20 years?

I think most application's needs are likely asymmetric, and my gut feeling is +10/-30 years. That'd cover the nineties, and therefore the whole mainstream digital/internet era. I also can't imagine most applications needing to project more than 10 years into the future.

Hey @glennsl I've adjusted the generator pipeline and shrunk the year range to 1990 to 2040, tzdb_compressed.ml is now at 242k, and the compiled size is 95K if I'm reading it right (I'm looking at tzdb_full.cmxs - is that correct?

Hey. I'm not sure how I'd be able to confirm those numbers independently, but it certainly seems plausible!
I wonder how much it would actually increase if you include more recent history. As long as the pattern remains the same, it would just add a few more entries to the L2 lookup table. Would it be easy to check how much difference going back to 1970 would make, for example?

Yeah I streamlined the year range specification, so you can do that in the Makefile gen section
.PHONY: gen
gen :
        cd gen/ && dune build gen_time_zone_data.exe
        dune exec gen/gen_time_zone_data.exe -- 1990 2040 full

Do a make gen to run and parse the zdump output, then make to make sure everything's rebuilt properly.
Finally do ls -lh _build/default/tzdb-full to check the size (I believe that's the right place):
(The order here is different since I was doing the ls outside of the container)
$ ls -lh _build/default/tzdb-full/
total 532K
-r--r--r-- 1 darren darren  381 Mar  5 02:33 dune
-r--r--r-- 1 darren darren  84K Mar  6 00:48 timedesc_tzdb_full.a
-r--r--r-- 1 darren darren  89K Mar  6 00:48 timedesc_tzdb_full.cma
-r--r--r-- 1 darren darren  914 Mar  6 00:48 timedesc_tzdb_full.cmxa
-r-xr-xr-x 1 darren darren  95K Mar  6 00:48 timedesc_tzdb_full.cmxs
-r--r--r-- 1 darren darren  231 Mar  4 03:39 timedesc_tzdb.ml
-r--r--r-- 1 darren darren  104 Mar  4 03:39 timedesc_tzdb__timedesc_tzdb_full__.ml-gen
-r--r--r-- 1 darren darren 242K Mar  6 00:48 tzdb_compressed.ml

For 1850 - 2040, make gen && make && ls -lh _build/default/tzdb-full/ yields
total 848K
-r--r--r-- 1 root root  381 Mar  4 15:33 dune
-r--r--r-- 1 root root  231 Mar  3 16:39 timedesc_tzdb.ml
-r--r--r-- 1 root root  104 Mar  3 16:39 timedesc_tzdb__timedesc_tzdb_full__.ml-gen
-r--r--r-- 1 root root 134K Mar  6 13:15 timedesc_tzdb_full.a
-r--r--r-- 1 root root 139K Mar  6 13:15 timedesc_tzdb_full.cma
-r--r--r-- 1 root root  914 Mar  6 13:15 timedesc_tzdb_full.cmxa
-r-xr-xr-x 1 root root 145K Mar  6 13:15 timedesc_tzdb_full.cmxs
-r--r--r-- 1 root root 408K Mar  6 13:15 tzdb_compressed.ml

For 1850 - 2100, the command yields
total 1012K
-r--r--r-- 1 root root  381 Mar  4 15:33 dune
-r--r--r-- 1 root root  231 Mar  3 16:39 timedesc_tzdb.ml
-r--r--r-- 1 root root  104 Mar  3 16:39 timedesc_tzdb__timedesc_tzdb_full__.ml-gen
-r--r--r-- 1 root root 159K Mar  6 13:08 timedesc_tzdb_full.a
-r--r--r-- 1 root root 164K Mar  6 13:08 timedesc_tzdb_full.cma
-r--r--r-- 1 root root  914 Mar  6 13:08 timedesc_tzdb_full.cmxa
-r-xr-xr-x 1 root root 169K Mar  6 13:08 timedesc_tzdb_full.cmxs
-r--r--r-- 1 root root 498K Mar  6 13:08 tzdb_compressed.ml

For 1970 - 2040, the command yields
total 628K
-r--r--r-- 1 root root  381 Mar  4 15:33 dune
-r--r--r-- 1 root root  231 Mar  3 16:39 timedesc_tzdb.ml
-r--r--r-- 1 root root  104 Mar  3 16:39 timedesc_tzdb__timedesc_tzdb_full__.ml-gen
-r--r--r-- 1 root root  99K Mar  6 13:09 timedesc_tzdb_full.a
-r--r--r-- 1 root root 104K Mar  6 13:09 timedesc_tzdb_full.cma
-r--r--r-- 1 root root  914 Mar  6 13:09 timedesc_tzdb_full.cmxa
-r-xr-xr-x 1 root root 110K Mar  6 13:09 timedesc_tzdb_full.cmxs
-r--r--r-- 1 root root 293K Mar  6 13:09 tzdb_compressed.ml

For 1970 - 2100, the command yields
total 796K
-r--r--r-- 1 root root  381 Mar  4 15:33 dune
-r--r--r-- 1 root root  231 Mar  3 16:39 timedesc_tzdb.ml
-r--r--r-- 1 root root  104 Mar  3 16:39 timedesc_tzdb__timedesc_tzdb_full__.ml-gen
-r--r--r-- 1 root root 125K Mar  6 13:18 timedesc_tzdb_full.a
-r--r--r-- 1 root root 130K Mar  6 13:18 timedesc_tzdb_full.cma
-r--r--r-- 1 root root  914 Mar  6 13:18 timedesc_tzdb_full.cmxa
-r-xr-xr-x 1 root root 135K Mar  6 13:18 timedesc_tzdb_full.cmxs
-r--r--r-- 1 root root 384K Mar  6 13:18 tzdb_compressed.ml

For 1990 - 2040, the command yields
total 532K
-r--r--r-- 1 root root  381 Mar  4 15:33 dune
-r--r--r-- 1 root root  231 Mar  3 16:39 timedesc_tzdb.ml
-r--r--r-- 1 root root  104 Mar  3 16:39 timedesc_tzdb__timedesc_tzdb_full__.ml-gen
-r--r--r-- 1 root root  84K Mar  6 13:12 timedesc_tzdb_full.a
-r--r--r-- 1 root root  89K Mar  6 13:12 timedesc_tzdb_full.cma
-r--r--r-- 1 root root  914 Mar  6 13:12 timedesc_tzdb_full.cmxa
-r-xr-xr-x 1 root root  95K Mar  6 13:12 timedesc_tzdb_full.cmxs
-r--r--r-- 1 root root 242K Mar  6 13:12 tzdb_compressed.ml

For 1990 - 2100, the command yields
total 708K
-r--r--r-- 1 root root  381 Mar  4 15:33 dune
-r--r--r-- 1 root root  231 Mar  3 16:39 timedesc_tzdb.ml
-r--r--r-- 1 root root  104 Mar  3 16:39 timedesc_tzdb__timedesc_tzdb_full__.ml-gen
-r--r--r-- 1 root root 109K Mar  6 13:13 timedesc_tzdb_full.a
-r--r--r-- 1 root root 114K Mar  6 13:13 timedesc_tzdb_full.cma
-r--r--r-- 1 root root  914 Mar  6 13:13 timedesc_tzdb_full.cmxa
-r-xr-xr-x 1 root root 120K Mar  6 13:13 timedesc_tzdb_full.cmxs
-r--r--r-- 1 root root 341K Mar  6 13:13 tzdb_compressed.ml

Hmm, interesting. The size increases with about 50% of the increase in time for 1990 vs 1970. Much bigger than I thought it would be. Compared to ~33% for 1990 vs 1850 and ~25% for 2040 vs 2100. Much better than 100% (or more) of course, but still quite expensive to do so just in case. Unless there are good reasons for extending the range I think 1990 is a good default.
Thanks for getting the numbers and explaining the procedure!

The size increases with about 50% of the increase in time for 1990 vs 1970

I don't think I follow this

The number of years between 1990 and 2040 is 50, and between 1970 and 2040 is 70. That's a 40% increase. And the size for those ranges is 532K and 628K respectively, which is an 18% increase. Therefore the increase in size is about half that of the increase in time covered.
Hope that's easier to follow (and also free of mistakes!)

The total is all files combined though - we're only interested in comparing the sizes of one of the compiled object files or tzdb_compressed.ml.
So for say 1970 to 2040, my understanding is that it's roughly 110K when compiled.

Ah yes, sorry. The difference is roughly the same though. For tzdb_compressed.ml it's 21%, and for the .a and .cxms files it's 17%

I am leaning toward 1970 - 2040 for the final publish, what you think?
I believe the tests cover everything we've added

Qc.to_of_compressed in desc-tests/time_zone_tests.ml (this is a new one)
Alco.tzdb_make_all in desc-tests/tzdb_tests.ml tries construction of all possible time zones provided by the tzdb backend (defaulting to tzdb.full)

lookup_record in desc/time_zone.ml is made more complicated for lazy loading:
let lookup_record name : record option =
  match M.find_opt name !db with
  | Some table ->
    assert (check_table table);
    Some (process_table table)
  | None ->
    match M.find_opt name compressed with
    | Some compressed_table ->
      let table =
        Compressed_table.of_string_exn compressed_table
      in
      assert (check_table table);
      db := M.add name table !db;
      Some (process_table table)
    | None -> None

but implicitly tested by the tzdb_make_all test.
If you don't spot anything missing I am going to finalise timedesc.0.7.0 and submit later.

Sweet! Can't think of anything, no.

I am leaning toward 1970 - 2040 for the final publish, what you think?

1970 is a very natural point in time that is likely to conform to users expectations thanks to Unix time. I still think 1990 is sufficient for 99% of use cases though, and that it's relatively expensive to extend it to 1970 given that.Then again, we've already reduced it by quite a lot, so perhaps it's time to relax a bit :smile:

Completed with timedesc (and timedesc-tzlocal-js) 0.8.0
One additional adjustment since then is removal of use of Marshal entirely, to make timedesc-tzdb a standalone package. The compressed form is stored entirely in repo now, so construction of a marshaled tzdb through timedesc is no longer needed.


do not require Content Security Policy (CSP) style-src 'unsafe-inline'

To not trigger a Content Security Policy (CSP) 'unsafe-inline' violation we have to modify CSSOM directly. This is what jQuery .css() method does. Therefore code like $('<div style="width: 100"'); should be avoided to not force consuming applications to lower CSP rules.

I think this is also much cleaner JS/jQuery code. :+1:

Thanks!

@mkoryak Do you mind to release a patch version?

I don't normally release with so little changes, but since you took time to make a PR, here is your new version:
https://github.com/mkoryak/floatThead/releases/tag/1.4.3

Thanks a lot!


Unsuccessful installation: npm ERR! function not implemented, symlink '../mkdirp/bin/cmd.js'

NodeJS Version: v7.10.0
Ubuntu 14.04 and x86_64:
Are you having trouble installing and you checked the Installation Special Cases docs?
YES and YES
Are you using Electron and have you checked the Electron Docs?:
NO and NO

Summary of Problem
Unsuccessful installation
Steps and Code to Reproduce the Issue
In terminal, execute:
sudo npm install serialport
Response:
`[sudo] password for lex:
prototipo@ /media/lex/Datos/Proyectos/Asuka/prototipo
└── (empty)
npm WARN prototipo@ No description
npm WARN prototipo@ No repository field.
npm WARN prototipo@ No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/mkdirp):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: ENOSYS: function not implemented, symlink '../mkdirp/bin/cmd.js' -> '/media/lex/Datos/Proyectos/Asuka/prototipo/node_modules/.bin/mkdirp'
npm ERR! Linux 4.4.0-79-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "install" "serialport"
npm ERR! node v7.10.0
npm ERR! npm  v4.2.0
npm ERR! path ../mkdirp/bin/cmd.js
npm ERR! code ENOSYS
npm ERR! errno -38
npm ERR! syscall symlink
npm ERR! ENOSYS: function not implemented, symlink '../mkdirp/bin/cmd.js' -> '/media/lex/Datos/Proyectos/Asuka/prototipo/node_modules/serialport/node_modules/node-pre-gyp/node_modules/.bin/mkdirp'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     https://github.com/npm/npm/issues
npm ERR! Please include the following file with any support request:
npm ERR!     /home/lex/.npm/_logs/2017-06-28T22_17_18_111Z-debug.log
`
THANK YOU

Thank you.
According with your suggestion, I'm proceeding to install node8.
I'll notify further results.

I found such problem could be I was trying to install in a device with a  file system distinct than EXT4 (a partition with a different file system).

The partition is FAT32. But now it does not care. Actually I'm working in the main partition, EXT4.


add link to list of allowed max versions close to addon upload link

Describe the problem and steps to reproduce it:
(Please include as many details as possible.)
my local TB installs addon xpi with forbidden max versions.
So for clarity, it would be helpful to have the link to the allowed max versions close to the upload button.
That would help to avoid unnecessary rejected runs through the linter. And even if the linter tells that the max version is not accepted .. where do i find the accepted values if that link is not visible.
What happened?
What did you expect to happen?
Anything else we should know?
(Please include a link to the page, screenshots and any relevant files.)

I am pretty sure this information (list of accepted min / max versions) is here:
https://addons.thunderbird.net/en-US/thunderbird/pages/appversions/

yes, my proposal is to include that link next to the addon upload button, so that it has higher visibility

We also added it to the list of important resources here:
https://developer.thunderbird.net/add-ons/resources


@W-9919770 Bug fix in UIAutomator locator static check

We already support this locator - "new UiScrollable(new UiSelector().scrollable(true)).scrollIntoView(resourceId("com.salesforce.chatter:id/app_launcher_menu_item"))"
This fix is to support - "new UiScrollable(new UiSelector().resourceId("com.salesforce.fieldservice.app:id/work_view")).scrollIntoView(new UiSelector().resourceId("com.salesforce.chatter:id/app_launcher_menu_item"))"
UiScrollable can use resourceId or other supported methods.

@helenren @lizaiv77 could you please review?

@jimevanssfdc since Liza is OOF, could you pls have a review? Thanks.


Add animation on scroll on whole website

i can  add beautiful animation on all page
improve user experience
sample
https://github.com/user-attachments/assets/88a9984e-019c-421c-83ee-3b3fd2b69db1

not needed at all , we like our website minimalistic and simple


Feature Request: Re-trigger Jenkins builds on fail

From my personal experience with Greenkeeper, the issues it opens are most of the time caused by some interruption in the build services, e.g. SauceLab. In this case, I often trigger a rebuild and it works again. It would be great if Greenkeeper could also do that before opening an issue.

Since it is hard to decide whether a build failure is caused by a legit error in the tests or something else, we’d rather not auto-restart CI services.


Contributed - web UI/UX and themes - elegant & aqua

All Submissions

[ ] Have you followed the guidelines in our Contributing document?
[ ] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions

[ ] Does your submission pass tests?
[ ] Have you lint your code locally prior to submission?

Changes to Core Features

[ ] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your core changes, as applicable?
[ ] Have you successfully ran tests with your changes locally?

Hey! @DalpatRathore, thank you for contributing to this project 🚀.
Before I review this PR, Can please:

Write a small description or some bullets about the changes you have made
Some relevant screenshots or GIFs representing your graphical changes (if any)

This will help while reviewing the code.

Hello! @Ashutosh00710
I have contributed the below-given changes in the “GitHub-Readme-Activity-Graph”  repo:

Web UI/UX Changes:

Made UI Changes with logo and footer

Added Loading Animation - when user data fetching for graph

Added footer section with text & link.

Added Two Themes:

Elegant

Aqua

Thank you for developing this activity graph. It not only looks good but also persuades programmers to keep coding and contributing more for plotting this elegant graph on their Github profile.
Thanks & Regards.

Thank you @Ashutosh00710  for appreciating my contributions.
Stay happy & blessed.
Happy Coding!


Option for rss to not warn on empty or missing data

I use a non-standard rss source. It provides new rss stories only once.
Because of this, flexget outputs a warning in the logs on every check:
Task didn't produce any entries. This is likely due to a mis-configured or non-functional input.

I'd like to hide that warning.
Unfortunately, setting silent: yes as per the docs does not seem to do the trick.
Could we add an option like noWarnEmpty: yes and/or make silent to hide that warning?

Well, that was easy to fix. Just needed to set no_entries_ok: yes up one level on the task. I expected to see that in the rss docs.


Companion from cengage

ADD ISSUE DESCRIPTION HERE
My professor has a companion that i seem not to have which allows him to look and the assignment and submit as well. Currently  I have to switch between webpages and submit assignments through email instead of through the course by screenshotting
Version: 1.85.2
Commit: 8b3775030ed1a69b13e4f4c628c612102e30a681
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
Embedder: codespaces

/causedByExtension
Please use your course support channel.

This issue is caused by an extension, please file it with the repository (or contact) the extension has linked in its overview in VS Code or the marketplace for VS Code. See also our issue reporting guidelines. If you don't know which extension is causing the problem, you can run Help: Start extension bisect from the command palette (F1) to help identify the problem extension.
Happy Coding!


Duplicate call log entries and SMS backups due to app or email changes

I have lots of duplicate call logs, like this:
13s (00:00:13)
1234567890 (outgoing call)
13s
1234567890
with the same date and time, presumably created by different versions of the app?
I also have duplicate SMS backups, where one will be saved under a phone number subject line and the other will be saved with a name subject line, after I've added their name to my address book.  Is there any way the program could delete the older duplicates when backing up the newer ones?

When backing up the second time did you say Backup ALL or ignore the current one's to backup?

I don't know what you mean by that.  "Items per backup" is set to "All".

It asked me if I want to only backup from where I last backed up or everything.

the last backup flag got probably reset, so it backed up old messages again. there's no deduplication in the app, i wrote an external dedup (https://github.com/jberkel/sms-backup-plus/blob/master/contrib/dedup.py) however haven't used it an while and it's potentially dangerous :)


Failed to parse abi: abi: purely anonymous or underscored field is not supported

Can be invoked by parsing ABI for contract: 0x00a8b738E453fFd858a7edf03bcCfe20412f0Eb0

Functions that start with _ are not supported. In order to sort this out we have to ensure only EXTERNAL and PUBLIC visibility types are packed for ABI. Internal and private functions/vars, etc... should not be in the ABI 😂

We are hit by this issue lately. Do we have an estimate of when this will land in the main release?

@shinewu-smlxl Next release is targeted for end of January. What I can do is by EOW put the fixes into the main branch.


signed-compose: [ART-1065] use same method for el7 as el8

Currently el7 signed composes are built from advisories; this means they only get arches we are shipping. If built from a "signed tag" like el8 they would get all arches. This makes them build the same.
See also:  https://coreos.slack.com/archives/CFFJUNP6C/p1569602175283300
On hold initially due to lack of permissions
(And obviously because of that I haven't been able to fully test it, but would appreciate reviews.)

tags and permissions are set up and I was able to test:
http://download.eng.bos.redhat.com/rcm-guest/puddles/RHAOS/AtomicOpenShift-signed/4.2/latest/
since we kinda need this for tomorrow's 4.2 builds i'm gonna merge this and related ocp-build-data changes.


Add Cognito branch

Add branch replacing Auth0 with Amazon Cognito Identity.
http://docs.aws.amazon.com/cognito/latest/developerguide/what-is-amazon-cognito.html
http://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-with-identity-providers.html
https://github.com/aws/amazon-cognito-identity-js

https://github.com/jch254/serverless-node-dynamodb-ui/issues/7 instead


GGLJ

Use the preview tab to see the example given below. Edit the relevant information and preview the changes before submitting. You may delete these instructions.

Key
Value

ID
GGLJ

Source
Vaaran Bhai Gurdas Ji

Page
26

Line
scY qKiq invwsu hY; swDsMgiq scKMif vsMdw [

Correction
THIS ≠ THAT

PROOF (EDITION)
Image:

Correction:
scKMif ≠ sc KMif
Proof:
Vaaran Bhai Gurdaas Ji | November 2011 | Shamsher Singh Ashok | SGPC, Sri Amritsar
Image:


HoloAnim prevents e2 from being spawned

I can't find a lua error :s, they're not showing up for some strange reason, otherwise i'd have just put in a pull request and fixed it myself.
Any E2 that has holoAnim code in it will not spawn.

Can you provide an example E2?


Improvements for the search page

Redesign of the search page to use "smart search" like in Google or ADS. Like the latter, offer the palette of search terms above the input field.
Possible queries are like:

10 20 300 - cone search at ra=10 dec=20 with 300 arcsec radius
Vega r=10 - cone search within 10 arcsec around Vega
10 20 300 after="2023-01-01" window=1 - cone search within 300 arcsec from Vega in 24 hours after given moment
ZTF22abzanvj - exact search for objectId
ZTF23abshpmf r=100 - cone search within 100 arcsec around the alert
ZTF22abzan - partial match search for objectIds starting with given pattern
TRCK_20231212 - partial match search for tracklets (effectively, all tracklets from a given date)
class=Unknown - last 100 alerts of a given class
class=Unknown last=10 - last 10 alerts of a given class
class=Unknown before="2023-01-01 01:12:23" - last 100 alerts before given moment

Everything is somehow working, except for SSO - this one is waiting for backend API improvements as discussed with Julien. GET api (does anyone actually use it?..) is changed to reflect underlying query parser, so the urls that work now look like:

/?action=conesearch&ra=10&dec=20&r=300
/?action=class&class=Unknown&last=10

Requires installation of two additional packages from pip - regex (used for partial matches) and dash-autocomplete-input (used for dropdown menu for class search).
Also requires updating dash-mantine-components to latest version from pip.
Documentation (mostly the help popup inside the search bar itself) is not yet updated.

Card view added in latest commits actually quickly hits the limit of deploying with finite number of workers in Gunicorn (as suggested by README) due to its lazy loading of cutouts and lightcurves - as many component update requests actually perform one more request to /api/ under the hood, and block until it returned. So when there are many simultaneous ones the worker pool is exhausted and everything just hangs.
Solution might be to deploy a number of dedicated workers for /api/ that do not serve anything else. I did not find any built-in option in Gunicorn for that, so for now ended with deploying two instances on different ports, and pointing APIURL to the second one that is not visible from outside.

With the last commit, the problem of exhaustion of web workers pool should be solved. The commit isolates all API access outside of /api/ into dedicated function that is able to either call requests.post(), or, if LOCALAPI=True, to use the endpoints directly, without doing unnecessary HTTP requests.

At this point, I consider this PR feature-complete, and ready for review.

Thanks @karpov-sv ! I started to test all the new features -- it looks super nice! I noticed one big thing though: the switch card/table does not work. If I trigger the button, I get an error from the fact that the endpoint /api/v1/columns does not have the 'POST' method. But even if I manually fix this, the switch does not operate (no error though -- just doing nothing).

Thanks @JulienPeloton for reporting, early morning coding has its downsides :-) I fixed it, should now be working.


Auto deploy testing branches, test and report

This is related to #65.
Features

Add a setup for local e2e tests with cypress.
Deploy, with travis-ci, the last commit at branches named tests-* at surge.sh.
Run tests in this url.
Send from travis a comment with a brief result report of the tests at the commit page. See an example at my fork.

New scripts:

npm run e2e - run tests locally (already running) localhost:3000 (cli mode).
npm run e2e:open- run tests locally to (already running) localhost:3000 in browser mode
npm run dev:e2e - start localhost:3000 and tests (browser mode). This is the default command to tdd.

obs:

The deployment pattern of urls at surge is: http://postwoman-:commithash.surge.sh
I added my surge user data at travis (login and token). It would be better to replace it for @liyasthomas account (to keep things centralized).
The github token env at travis should be able to write comment issues with curl (probably it is already).
I tried to do this routine at Pull Requests but I failed. I think it would be better. But could not find a workaround to pull requests restrictions.

Hey @yubathom,
Your changes look good to me!
View build log
TravisBuddy Request Identifier: 3529c0b0-d984-11e9-bca9-bbe9a5d172b3

@yubathom What are the required env tokens..?

To run the tests in public urls and check the changes ourselves and discuss it in  common url at PR's. In testdeploy.sh  is required SURGE_LOGIN and SURGE_TOKEN.
The github token are needed to send the report messages.
Would you prefer to keep the tests runs just locally? That won't require any of theese envs..

To create the to token need to install surge globally :
npm i -g surge
# create an account and run
surge token

I think tests on CI is important too. Let me work on this and will will let you know.

Ok

@NBTX @AndrewBastin test when you're free

Since a docker image (#164 / #165)  is available I guess that'd be better to run these tests with docker componse at ci . Need to update this.

In that case, I'll test when you accomodate #164 / #165

Revisited this and I think this got unnecessarily complicated. I will create another PR with simpler setup and more tests.


Why are we still getting these errors on a fresh new peer?

The user will get confused thinking something is wrong.

Reviewed log messages.

Issue can be closed.


Add "toBase64" and "fromBase64" functions

steps:
  - decoded = fromBase64: $encoded
  - encodedAgain = toBase64: $decoded
  - asAscii = fromBase64:
      value: $encoded
      as: ascii

Could be part of an "encoding" module. Assume "from" or "to" is "utf-8" unless specified.

Added to v2.2.0


Would be useful to have a getRootComponents method

This method returns components that are not subComponents of any other component.

The following method has been added to org.sbolstandard.core3.api.SBOLAPI:
Set getRootComponents(SBOLDocument doc, ComponentType type)


Weird rendering glitches

We have found some canvas glitches when using flutter_svg and charts libraries together in the same screen. Look like the canvas buffer overflows.
Here you can see when the screen renders well:

And here you can see examples of the glitches:

When
As far as we have seen this happens when:

Both libraries are used together in the same screen
Release apk
Android

When not
We have never seen this when:

Only one of the libraries is used in a given screen
Debug apk
iOS

How to reproduce

Clone the demo repo located at: https://github.com/priezz/flutter_canvas_glitch
Create a signed apk and install in a physical device.
Navigate back and forth until you notice the error in the main screen

Logs
This happens only in release build, independently of the computer used to create the build. including the CI. It also happens using different keys to sign the release apk. Flutter doctor is from my machine.
flutter doctor -v
[✓] Flutter (Channel dev, v1.2.1, on Mac OS X 10.13.6 17G5019, locale en-GB)
    • Flutter version 1.2.1 at /Users/jose/Development/sdks/flutter
    • Framework revision 8661d8aecd (6 days ago), 2019-02-14 19:19:53 -0800
    • Engine revision 3757390fa4
    • Dart version 2.1.2 (build 2.1.2-dev.0.0 0a7dcf17eb)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/jose/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
    • All Android licenses accepted.

[✗] iOS toolchain - develop for iOS devices
    ✗ Xcode installation is incomplete; a full installation is necessary for iOS development.
      Download at: https://developer.apple.com/xcode/download/
      Or install Xcode via the App Store.
      Once installed, run:
        sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
    ✗ libimobiledevice and ideviceinstaller are not installed. To install with Brew, run:
        brew update
        brew install --HEAD usbmuxd
        brew link usbmuxd
        brew install --HEAD libimobiledevice
        brew install ideviceinstaller
    ✗ ios-deploy not installed. To install:
        brew install ios-deploy
    ✗ CocoaPods not installed.
        CocoaPods is used to retrieve the iOS platform side's plugin code that responds to your plugin usage on the Dart side.
        Without resolving iOS dependencies with CocoaPods, plugins will not work on iOS.
        For more info, see https://flutter.io/platform-plugins
      To install:
        brew install cocoapods
        pod setup

[✓] Android Studio (version 3.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 33.0.1
    • Dart plugin version 182.5215
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[✓] Connected device (1 available)
    • Android SDK built for x86 • emulator-5554 • android-x86 • Android 9 (API 28) (emulator)

! Doctor found issues in 1 category.

@liyuqian

Did you find any solution to this?

@quetool I opened this issue less than 24h ago. Waiting for the flutter team to have a look at it.
Have you found this same issue?

@dnbit sorry, did't see the datetime... Yes, I do have the same issue in some olders devices...

@quetool only on old devices?

@Okanochiwa I tested on Galaxy S8 Edge and LG Q6 Alpha and only in LG happens... Neither on S8 or any simulator I created happens this...
Wrong (LG Q6 Alpha)

Correct (Others devices I tested: Galaxy S8, many Android simulators, all iPhone)

I would not say that those devices are old. LG Q6 was released in July 2017 with Nougat upgradable to Oreo.
I think that S8 Edge does not exist. Probably you mean s7 Edge? That's 3 years old but still it was a flagship and running Oreo.
I have seen this happening in my Nexus 5x, which is older but still running Oreo.
Also in a Moto G5 and Sony XA1, which are from early 2017, both running Oreo as well.
I will try to find more, but to sum up we have seen this in a variety of reasonable recent models from different manufactures.

Ok, maybe this does not have to do with the device but still... That's my experience...
Anyway, S8 Edge does exists, at least in Argentina https://www.cnet.com/g00/es/analisis/samsung-galaxy-s8-opiniones/

@dnbit Ok, maybe this does not have to do with the device but still... That's my experience...
Anyway, S8 Edge does exists, at least in Argentina https://www.cnet.com/g00/es/analisis/samsung-galaxy-s8-opiniones/

Same  issue  on XiaoMi Note3,  flutter sdk version is 1.2.1.
I try flutter sdk 1.0.0,   is OK.
flutter_svg picture  and  CustomPaint  Widget  in the same page.

Sorry I'm a little busy recently so I don't have time to debug this in details.
Can you please take an SKP screenshot of the glitch using flutter screenshot --type=skia --observatory-port=<PORT_LOGGED_BY_FLUTTER_RUN>, and post it here?
Skia team may be able to directly work on this using the SKP.
CC bsalomon@, @csmartdalton from Skia team.

@bsalomon

broken  page:

skp file:
flutter_02.skp.zip
compared to normal page:

@liyuqian, @bsalomon, @csmartdalton

The most helpful next step here would be an even more reduced test case. If someone could take https://github.com/priezz/flutter_canvas_glitch, and slowly delete code until they are left with just a few lines of code showing the problem (all in one file), which they can then attach here, that would be fantastic. Super bonus points if you manage to do this in a way that removes the dependency on package:flutter_svg and package:flutter as well (leaving only a dependency on dart:ui), though obviously that's a lot of work!

Removing flutter_svg library is difficult because I don't know what's inside it.
But I use https://debugger.skia.org/ to open my skp file and the display is normal.
My guess is that you can debug directly using the skp file.

@Hixie I have minimized the demo here https://github.com/priezz/flutter_canvas_glitch, however it still refers to flutter, charts_flutter and flutter_svg.
I was not able to create the skia snapshot as I can reproduce the issue only when I do the release build with flutter build apk and install the app-release.apk into the device.
To reproduce the bug I use the ListView widget (scroll up and down a few times to get the glitch), however it could not be considered as the source of the issue, because in our app the bug appears w/o the list as well.
The screenshots:
good

glitch

Source code:
import 'package:charts_flutter/flutter.dart' as charts;
import 'package:flutter/material.dart';
import 'package:flutter_svg/flutter_svg.dart';

void main() => runApp(App());

class App extends StatelessWidget {
  @override
  build(BuildContext context) => MaterialApp(
        home: ListView.builder(
          itemCount: 20,
          itemBuilder: (context, i) => i == 0
              ? Container(
                  height: 120,
                  child: charts.PieChart([
                    charts.Series<int, int>(
                      id: 'donut',
                      domainFn: (_, i) => i,
                      measureFn: (_, i) => i + 1,
                      colorFn: (_, i) => charts.Color.fromHex(code: '#00ff00'),
                      data: [0, 1, 2],
                    )
                  ]),
                )
              : SvgPicture.asset(
                  'assets/icons/${icons[i % icons.length]}.svg',
                  allowDrawingOutsideViewBox: false,
                  color: Colors.white,
                  fit: BoxFit.contain,
                  height: 36,
                  width: 36,
                ),
        ),
      );
}

const List<String> icons = [
  'bank',
  'broadband',
  'car',
  'debt',
  'beer',
  'education',
  'family',
  'fun',
  'going_out',
  'groceries',
];

I tried and failed to reproduce this in standalone Skia from the flutter_02.skp. I'm wondering if this is particular to certain devices. Could someone who can reproduce and builds flutter engine try setting GrContextContextOptions::fDisableCoverageCountingPaths to true?
https://cs.chromium.org/chromium/src/third_party/skia/include/gpu/GrContextOptions.h?q=GrContextOptions+file:^src/third_party/skia/+package:^chromium$&g=0&l=85

@bsalomon to your first question, this bug occurred on all the Android devices we have tested (Samsung, Sony, LG...) under Android Oreo.

It's not a necessary problem, but it's a high probability.

Would you say these issues are the same problem?
https://github.com/flutter/flutter/issues/28484
https://github.com/flutter/flutter/issues/22546
https://github.com/flutter/flutter/issues/28786

@marcglasberg:

#28484 occurs on iOS and we don't have any glish on iOS
-#22546: looks fairly similar
#28786 also has glitches on svg and definitely looks very similar

I tested different versions in detail and found that this problem occurred in fluter version 1.1.8 to 1.1.9. There are very few changes in this period, I hope to help you.
1.1.8 is ok,
1.1.0 is broken.

@Hixie

I tried the flutter_canvas_glitch app with the current Flutter master branch on several devices, but so far I haven't been able to reproduce the error

@songfei We have seen it in more modern versions
@jason-simmons Have you tried with release build?

The Skia commit Reland "ccpr: Rework the path cache to support sporadic flushing" is very suspicious for this to me.  I'll have to see if I can repro this.

Hmm. I'm unfortunately not able to reproduce this on a Samsung S8 on the reported revision.  @jason-simmons have you had any luck?

@dnbit I mean this bug started with version 1.1.9, so there is a big possibility that it was caused by code changes during this period.

@dnfield, @jason-simmons: this glitch screenshot comes from a Samsung Galaxy S8 Edge:
https://www.dropbox.com/s/rqgt9n5dak3430v/glitch-samsung-s8-edge.jpeg?dl=0

@dnfield, @jason-simmons: this glitch screenshot comes from a Samsung Galaxy S8 Edge

I finally managed to reproduce this on a BLU Vivo Go device running Android 9.
On an android_debug_unopt engine build it triggered an assertion failure in Skia:
../../third_party/skia/src/gpu/ccpr/GrCCPerFlushResources.cpp:480: fatal error: "assert(fNextCopyInstanceIdx == baseCopyInstance)"

Stack trace:
third_party/skia/src/gpu/ccpr/GrCCPerFlushResources.cpp:480
third_party/skia/src/gpu/ccpr/GrCoverageCountingPathRenderer.cpp:269
third_party/skia/src/gpu/GrDrawingManager.cpp:253
third_party/skia/src/gpu/GrDrawingManager.cpp:481
third_party/skia/src/gpu/GrRenderTargetContext.cpp:1724
third_party/skia/src/gpu/SkGpuDevice.cpp:1625
third_party/skia/src/gpu/SkGpuDevice.cpp:1615
third_party/skia/src/core/SkCanvas.cpp:673
third_party/skia/src/core/SkCanvas.cpp:667
flutter/shell/gpu/gpu_surface_gl.cc:272

The linked CL is where that got introduced in Skia.

I think I'm running into a similar issue where I'm drawing paths to a Canvas - only on release builds, it crashes the app on 64-bit devices (we release 32-bit armv7 builds to support older devices). If I rebuild for arm64, it runs fine on 64-bit devices. While it's drawing new segments of the path, we see weird glitches occur in segments of the Path (we use Path.lineTo() and then canvas.drawPath() to draw multiple paths over an image). The crash is currently holding up our release - and trying on master channel does not help. This also occurred when not using paths and just using the canvas.lineTo() method to draw between points.

I also started having this issue with iOS (both debug and release mode). I encountered this after i upgraded Flutter. I've also tried master channel with no luck.

only svg,  flutter_svg demo,  on BuBuGao（步步高） Android Pad

flutter doctor -v
[✓] Flutter (Channel master, v1.3.10-pre.37, on Mac OS X 10.14.3 18D109, locale zh-Hans-CN)
• Flutter version 1.3.10-pre.37 at /Users/songfei/flutter
• Framework revision 7ac2486a4c (80 minutes ago), 2019-03-13 19:19:31 -0700
• Engine revision 8b1a299ed5
• Dart version 2.2.1 (build 2.2.1-dev.1.0 674fd0e060)
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
• Android SDK at /Users/songfei/Library/Android/sdk
• Android NDK location not configured (optional; useful for native profiling support)
• Platform android-28, build-tools 28.0.3
• ANDROID_HOME = /Users/songfei/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
• All Android licenses accepted.
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Xcode 10.1, Build version 10B61
• ios-deploy 1.9.4
• CocoaPods version 1.5.3
[✓] Android Studio (version 3.3)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin version 33.3.1
• Dart plugin version 182.5215
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
[✓] VS Code (version 1.31.1)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 2.23.1
[✓] Connected device (1 available)
• S3 Pro • 70S3S9207839W • android-arm64 • Android 7.0 (API 24)
• No issues found!

Thanks for more input on this.
As this point we have identified the cause and are working to figure out the best fix for it.  Should have an update sometime soon.

@dnfield great news! Thanks!

So... what is the cause?

@Hixie issues with CCPR.  see https://github.com/flutter/flutter/issues/28227#issuecomment-472169782  - although I suspect there may be other issues with CCPR as well causing some of these.

@dnfield: would you have an expected bug-fix date? Our app cannot be shipped with such bug

I believe disabling CCPR landed on master, so you should be able to use that. That should roll into a dev build in the next few days, a beta build in the next few weeks, and a stable build in the next few months.

@Hixie We have similar issues with SVGs and I just tried the newest master. For us it didn't solve the problem. Is it necessary to disable CCPR by hand. I searched through the flutter and engine repo and couldn't find anything that controls CCPR

any update?  @dnbit

Is this still reproducing on master?

@priezz Can you please follow up on this?

If you are still seeing this on master, please file a new bug. We believe the issue described above is fixed, but it's possible it is not. What would be helpful is if we had new bugs with steps to reproduce so that we could try specifically to narrow down the issue with cases that are known to still be broken. Thanks!

@Hixie  - actually per @csmartdalton86, turning off CCPR isn't enough to undo this.  There's a fix pending in Skia.  I forgot to update this yesterday after learning that information.

Ah ok. Good to know!

I have tested , it's ok!  good job!

@dnfield : do you mean the path cache version 2 discussed in the chat room? If so, I think it's probably only used by CCPR, so disabling CCPR should have solved the issue.

Ahh I think you're right - following up with them to be sure.

Yes that's correct. Turning off ccpr also turns off the path cache that is
probably causing this.
On Wed, Mar 27, 2019, 4:24 PM Dan Field<EMAIL_ADDRESS>wrote:

Ahh I think you're right - following up with them to be sure.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/flutter/flutter/issues/28227#issuecomment-477370189,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AlEMlPI2ts0Uxy6Wdz9CMbWEUB712Jufks5va--ggaJpZM4bGMXv
.

Thanks!
I'm going to close this as it seems to be fixed on master.

Big thanks to @dnfield and all those who contributed 👍


Added keyboardType & textInputAction props to SearchBar, SearchAnchor & SearchAnchor.bar

Added keyboardType & textInputAction props to SearchBar, SearchAnchor & SearchAnchor.bar
Fixes #138483
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I signed the CLA.
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

I think I'll need one more approval. ✅ ✔


Unable to add custom tags in NextJS app

What happened?
When running a new NextJS app on a Windows machine running Node v14.19.3 using the markdoc example, the app fails to load, throwing a failed to compile error:

After some tinkering - I found that by removing the tags, the app / page would finally render.
To reproduce

run npx create-next-app --example https://github.com/markdoc/markdoc-starter
run npm run dev
App will fail to load and show the error above.

Version
0.1.2
Additional context
I tried setting up the project on a M1 Mac running node v16.13.0 and the app was able to compile and render. So I'm wondering if this is related to the OS or possibly the node version.

@SamuelQuinones can you try installing 0.1.4? Issues with .tsx got resolved in that version.

@mfix-stripe both @markdoc/markdoc and @markdoc/next.js were set to latest in the package.json, I ran npm i @markdoc/markdoc@latest and npm i @markdoc/next.js@latest and ended up with this:
"dependencies": {
    "@markdoc/markdoc": "^0.1.2",
<EMAIL_ADDRESS>"^0.1.4",
    "next": "latest",
    "prismjs": "^1.28.0",
    "react": "latest",
    "react-dom": "latest"
  },

Issue still persists

An update:
I tried creating / running the app on Another windows machine, this one using node v16.13.0.
I get the same error so it looks it might be OS / platform related?

I get the same error so it looks it might be OS / platform related?

It definitely seems to be that way. Marked the issues as os:windows. This actually seems to be unrelated to Markdoc, and is simply an issue with our Next.js app being able to parse the JSX in the .tsx file (error points at the first < of a JSX element).

@mfix-stripe Can we rename this issue to something that identifies this is Windows?

@mfix-stripe I'd be happy to fix the issue here but I don't know which ballpark to look!
Any ideas?

@arranf it definitely has something to do which how files are being resolved here: https://github.com/markdoc/next.js/blob/main/src/loader.js
I thought this issue had been fixed with: https://github.com/markdoc/next.js/commit/a5ecea39c2e44d4d44bfbd58fcf125453e4932d9 and https://github.com/markdoc/next.js/commit/d6f933465f4f0d340c38546de53f3b358b379d5e, but I guess some still remain.

Hi, I was looking into this to see if I could PR a solution (not there yet).
From my investigation this could be an issue with Markdoc but it also could be an issue with Next.js and how it's webpack rules are set up and work on Windows.
The loader calls readDir() which results in output from the loader like this:
// … clipped …
import * as tags from '/Users/myusername/project/markdoc/tags.js'
import * as nodes from '/Users/myusername/project/markdoc/nodes.js'
// … clipped …

Using the absolute paths like this breaks. It's like Next.js doesn't see this as a valid js/jsx file and doesn't process run it through any loaders. You can reproduce this outside of markdoc by creating an index.jsx file in Next.js and writing the same imports.
However, if you use relative paths for the imports (in the index.jsx file) then Next.js is happy:
import * as tags from '../../markdoc/tags.js'
import * as nodes from '../../markdoc/nodes.js'

Not sure if this is helpful yet but it's some info that could help 🤷

This seems like a fix would be to convert it to use non absolute paths on Windows?
On Tue, 28 Jun 2022, at 6:17 PM, Jordan Pittman wrote:

Hi, I was looking into this to see if I could PR a solution (not there yet).
From my investigation this could be an issue with Markdoc but it also could be an issue with Next.js and how it's webpack rules are set up and work on Windows.
The loader calls readDir() which results in output from the loader like this:
// … clipped …
import * as tags from '/Users/myusername/project/markdoc/tags.js'
import * as nodes from '/Users/myusername/project/markdoc/nodes.js'
// … clipped …
Using the absolute paths like this breaks. It's like Next.js doesn't see this as a valid js/jsx file and doesn't process run it through any loaders. You can reproduce this outside of markdoc by creating an index.jsx file in Next.js and writing the same imports.
However, if you use relative paths for the imports (in the index.jsx file) then Next.js is happy:
import * as tags from '../../markdoc/tags.js'
import * as nodes from '../../markdoc/nodes.js'
Not sure if this is helpful yet but it's some info that could help 🤷
—
Reply to this email directly, view it on GitHub https://github.com/markdoc/markdoc/issues/85#issuecomment-1169007693, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCMZ5WM3DEVMUIDKKPAW6LVRMXRFANCNFSM5YLJRE5A.
You are receiving this because you were mentioned.Message ID: @.***>

Thanks @thecrypticace.
@arranf I am not sure we should should convert to non-absolute paths on Windows yet.
However, there might be a fix involved with not using .posix in a place or two here: https://github.com/markdoc/next.js/blob/main/src/loader.js.
Using posix fixed one issue for me in the past (https://github.com/markdoc/next.js/pull/5/files) , but might have introduced another, and it's difficult for me to verify on Windows since I don't have access to a Windows machine.

@mfix-stripe Yep, I think not using POSIX paths would fix it. The Next.js config for js, jsx, ts, tsx, etc… files does not use POSIX paths for the includes array which is the source of the issue:
[
  'C:\\Users\\myusername\\project',
  /next[\\/]dist[\\/]shared[\\/]lib/,
  /next[\\/]dist[\\/]client/,
  /next[\\/]dist[\\/]pages/,
  /[\\/](strip-ansi|ansi-regex)[\\/]/
]

@mfix-stripe Yep, I think not using POSIX paths would fix it. The Next.js config for js, jsx, ts, tsx, etc… files does not use POSIX paths for the includes array which is the source of the issue:
[
  'C:\\Users\\myusername\\project',
  /next[\\/]dist[\\/]shared[\\/]lib/,
  /next[\\/]dist[\\/]client/,
  /next[\\/]dist[\\/]pages/,
  /[\\/](strip-ansi|ansi-regex)[\\/]/
]

Agreed. I think we might be able to just remove all the normalizeAbsolutePath and just update the tests. The tests need to work on both unix and windows, which is a little tricky, but we just need to hack the paths so they are identical.

@thecrypticace @SamuelQuinones @arranf could one of you try installing 0.1.5-0 and seeing if that fixes the issue?

almost — the slashes need to be double escaped because they're going into a string:

almost — the slashes need to be double escaped because they're going into a string:

@thecrypticace I think the error above actually has to do with the imports starting w/ C:, and webpack not understanding is correctly. I believe the escaping issue is just broken on the error overlay, not with the actual imports themselves.

@thecrypticace do you think enabling preferAbsolute could resolve this?

I don't think preferAbsolute would end up changing the result.

Thanks @thecrypticace!  Just released https://github.com/markdoc/next.js/releases/tag/0.1.5-1. @thecrypticace could you install this latest version and double-check that it works for you? Really appreciate your help on this 🙂

@mfix-stripe yep that does indeed work 🎉

🙌 fantastic! Thanks for all your help here @thecrypticace 🙏

https://github.com/markdoc/next.js/releases/tag/0.1.5


Feature Request

Hey @SRvSaha , Would it be possible to pause the request in the middle off session using any keyboard shortcut or something so that when ever we want, we resume the checking again.
Just thought of it.. What do you think..

maybe just do a hotkey thing where one could just pause it by using hotkey and resume back the search

Reason #68


mv:cannot stat'/home/lxc/.zcompdump.zwc' : No such file or directory

zmanage info:
Zim commit ref:   d19c8dd
Zsh version:      zsh 5.7.1 (x86_64-ubuntu-linux-gnu)
System info:      Linux lxc-PC 5.3.0-19-generic #20-Ubuntu SMP Fri Oct 18 09:04:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

.zimrc:
zmodules=(directory environment git git-info history input utility custom \
          prompt completion syntax-highlighting history-substring-search archive)
zprompt_theme='pure'
SPROMPT='zsh: correct %F{red}%R%f to %F{green}%r%f [nyae]? '
zinput_mode='emacs'
zhighlighters=(main brackets cursor)

.zshrc:
#
# User configuration sourced by interactive shells
#

# Define zim location
export ZIM_HOME=${ZDOTDIR:-${HOME}}/.zim

# Start zim
[[ -s ${ZIM_HOME}/init.zsh ]] && source ${ZIM_HOME}/init.zsh
# Set up the prompt

autoload -Uz promptinit
promptinit
prompt liquidprompt

setopt histignorealldups sharehistory

# Use emacs keybindings even if our EDITOR is set to vi
bindkey -e

# Keep 1000 lines of history within the shell and save it to ~/.zsh_history:
HISTSIZE=1000
SAVEHIST=1000
HISTFILE=~/.zsh_history

# Use modern completion system
autoload -Uz compinit
compinit

#z.lua
eval "$(lua //home/lxc/.zim/modules/z.lua/z.lua  --init zsh once enhanced)"    # ZSH 初始化
alias zc='z -c'      # 严格匹配当前路径的子路径
alias zz='z -i'      # 使用交互式选择模式
alias zf='z -I'      # 使用 fzf 对多个结果进行选择
alias zb='z -b'      # 快速回到父目录

#fzf
[ -f ~/.fzf.zsh ] && source ~/.fzf.zsh

#load zplug
source ~/.zplug/init.zsh
# fz work with z.lua
zplug "changyuheng/fz", defer:1
zplug "rupa/z", use:z.sh
FZ_HISTORY_CD_CMD="_zlua"

#tmux plugin manager
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-sensible'
set -g @plugin 'jimeh/tmux-themepack';
set -g @plugin 'tmux-plugins/tmux-resurrect'

Description
I installed tmux today on my Ubuntu 19.10, when I rantmux in my terminal, and the error appeared.
If I use zsh normally, there is no error.
Steps to reproduce
Everytime when I login tmux.
Images or other information

I have this issue too and I see it every time I log into my shell. Not seeing it after that.

This seems to be a problem when running zrecompile.
Please confirm if you're able to reproduce the issue by running:
zstyle -s ':zim:completion' dumpfile 'zdumpfile' || zdumpfile=${ZDOTDIR:-${HOME}}/.zcompdump
autoload -Uz zrecompile && zrecompile -p -q ${zdumpfile}
unset zdumpfile

Also, do you get any output when running compaudit?

@ericbn Just tried both in a newly spawned shell and both output nothing and exit fine. As I noted before though - I'm only seeing this just after I log into my machine and only sometimes. It does not seem to be very consistent...