Module12 larger tests

Slides are mostly where I would like them to be - I think that this does a pretty good job of reflecting the feedback in the sheet from last time, plus added some new content. I reworked the narrative getting to mocks and spies, and I hope that this time it is much more clear what the difference between a mock and a spy is.

Added mock example for testing pushing clock
Added spy example for testing pushing clock
Clarified spying on an object vs spying on a module
Refactored await expect(somePromise).resolves.toEqual to expect(await somePromise).toEqual - much less verbose and confusing mixing resolves with await

I may put together a short activity for testing the todo app, we will see how things go tomorrow.

LGTM
In the past we didn't have an activity for this lecture and I usually would ask students to look thru coveytown and find examples of mocks and spies and share with the class.

I think ss7-8 should immediately follow s4. Once you establish the idea of testing with a custom observer, you can say "remember back in Module , when we talked about test doubles. That's what we did! We build a fake listener to replace the real listeners in our application".

Moved.

s13, maybe add "OTOH: ...." to the title?

Updated title

s14: are we testing the clock or the listener? Up to this point, we know we are testing the clock, because the listener is something we wrote (or, in the case of the mock, caused to be created) ourselves-- it's something we control.

You are testing the ProducerClock. Added more discussion in the presenter notes for this slide.

s14: Should this point out that spyOn is global.? Imagine that instead of invoking clock1.tick (), the test invoked some other piece of code, which in turn invokes clock1, which in turn calls clock1.tick. That was the situation in the Fall 23 slide (M12.1 s24). I always found that really hard to understand, hence my comment "mocks are local; spies are global".

"Mocks are local; spies are global" is also a somewhat muddy explanation, unfortunately. Spies are local to whatever they spy on. In this case, the spy is "local" to clock1. Anyone else who has a reference to clock1 will also see the spy. The same argument applies to listener1 on slide 12. I added a new slide 15 that I think explains this much more clearly.

I think the point is that spies are useful when we control neither the SUT or the system in which the SUT is embedded. Do I understand this correctly?

Yes, exactly. I put these words on the new slide 15.

s18: "The best way to reduce this coupling to build the mocks around the interface you control" What does this mean? Tell us more.

I think the problem is back on slide 15: exactly what property of echo are we trying to test? (I confess that this code, and hence this problem, is mine :-) I think this might be an integration test, not a unit test.

Whoops, I finished the sentence. We are before discussing unit vs integration tests. You are right that to unit test echo, you would need to mock axios. There's not really a better way around it. But, for testing clients that call echo, mocking echo is a better choice.

s21: I can see how the kind of jest test that we wrote might work for this-- you can run the same jest test that you did before, only in a different setup. But how do you even write the bigger tests, like the one on s26? Maybe a forward reference to the CI workflows we do later? Maybe forward reference to capture/replay, as on slide 30?

I added to the presenter notes on these slides. I didn't want to put more on the slides because "how to implement" these tests is not a learning objective for the module, and seemed like a sidebar/distraction.

You wrote:
Refactored await expect(somePromise).resolves.toEqual to expect(await somePromise).toEqual - much less verbose and confusing mixing resolves with await
I like this! Where is it? [BTW, I should update the async lecture to incorporate this]

Slides 17, 18, 19, (numbers correspond to the latest version of the slide deck, not what you reviewed)

Also, on the discord you wrote about including an example from covey.town that shows checking for multiple effects.
What happened to that?

I looked through quite a few of the tests in Covey.Town, and unfortunately I do not think that they would be good examples here. The entire test suite is derived from the autograders - these tests are not really representative of the tests that one might write if they were simply trying to create a good test suite. For example: we check for the absence of side effects. We also have way more "unit" tests (checking a single method) as opposed to "small integration" tests (checking a few methods) because we need to assign points at that level of granularity. These unit tests make use of some pretty gross mocks that I would are anti-patterns: in some cases, there is more implementation to support the mocks than there is for the production code. This is an interesting side discussion, but the tldr is that in the future, maybe we should set up time specifically for creating those new tests to use as examples.

Looking forward to the next draft.

Thanks for all of the great suggestions and feedback! However, just to manage expectations: it is unlikely that I will have bandwidth for another round of revisions this semester (but will save the notes for next term).

s1: update copyright date.

"Mocks are local; spies are global" is also a somewhat muddy explanation, unfortunately. Spies are local to whatever they spy on. In this case, the spy is "local" to clock1. Anyone else who has a reference to clock1 will also see the spy. The same argument applies to listener1 on slide 12. I added a new slide 15 that I think explains this much more clearly.

I like the new s15.
I think the point is that spies are useful when we control neither the SUT or the system in which the SUT is embedded. Do I understand this correctly?

Yes, exactly. I put these words on the new slide 15.

We have to have some control over the system in which the SUT is embedded, because we have to be able to name the place where the spy is to be applied.   And the comment on the third code line is misleading: it should be "Spy on calls to notify on this client".    And we need to say somewhere that the clockClient in the second line is just some arbitrary listener that we happen to know the name of.

s18: "The best way to reduce this coupling to build the mocks around the interface you control" What does this mean? Tell us more.

Whoops, I finished the sentence. We are before discussing unit vs integration tests. You are right that to unit test echo, you would need to mock axios. There's not really a better way around it. But, for testing clients that call echo, mocking echo is a better choice.

Got it.

s21: I can see how the kind of jest test that we wrote might work for this-- you can run the same jest test that you did before, only in a different setup. But how do you even write the bigger tests, like the one on s26? Maybe a forward reference to the CI workflows we do later? Maybe forward reference to capture/replay, as on slide 30?

I added to the presenter notes on these slides. I didn't want to put more on the slides because "how to implement" these tests is not a learning objective for the module, and seemed like a sidebar/distraction.

Got it.
You wrote:

Refactored await expect(somePromise).resolves.toEqual to expect(await somePromise).toEqual - much less verbose and confusing mixing resolves with await
I like this! Where is it? [BTW, I should update the async lecture to incorporate this]

Slides 17, 18, 19, (numbers correspond to the latest version of the slide deck, not what you reviewed)

Got it.  It would be useful to point out that this is different from what we wrote in the async lecture.

Thanks for all of the great suggestions and feedback! However, just to manage expectations: it is unlikely that I will have bandwidth for another round of revisions this semester (but will save the notes for next term).

If you would like, I can make the tweaks I proposed (s1, s15, s17notes, s19notes).  I can make it a separate branch, or just do it in module12-larger-tests.  Let me know.

Feel free to make changes directly and push. I may be able to skim quickly tomorrow morning, but otherwise am on other work items between now and my lecture on Thursday. On Feb 13, 2024, at 2:45 PM, Mitchell Wand @.***> wrote:
s1: update copyright date.
"Mocks are local; spies are global" is also a somewhat muddy explanation, unfortunately. Spies are local to whatever they spy on. In this case, the spy is "local" to clock1. Anyone else who has a reference to clock1 will also see the spy. The same argument applies to listener1 on slide 12. I added a new slide 15 that I think explains this much more clearly.
I like the new s15.
I think the point is that spies are useful when we control neither the SUT or the system in which the SUT is embedded. Do I understand this correctly?
Yes, exactly. I put these words on the new slide 15.
We have to have some control over the system in which the SUT is embedded, because we have to be able to name the place where the spy is to be applied.   And the comment on the third code line is misleading: it should be "Spy on calls to notify on this client".    And we need to say somewhere that the clockClient in the second line is just some arbitrary listener that we happen to know the name of.
s18: "The best way to reduce this coupling to build the mocks around the interface you control" What does this mean? Tell us more.
Whoops, I finished the sentence. We are before discussing unit vs integration tests. You are right that to unit test echo, you would need to mock axios. There's not really a better way around it. But, for testing clients that call echo, mocking echo is a better choice.
Got it.
s21: I can see how the kind of jest test that we wrote might work for this-- you can run the same jest test that you did before, only in a different setup. But how do you even write the bigger tests, like the one on s26? Maybe a forward reference to the CI workflows we do later? Maybe forward reference to capture/replay, as on slide 30?
I added to the presenter notes on these slides. I didn't want to put more on the slides because "how to implement" these tests is not a learning objective for the module, and seemed like a sidebar/distraction.
Got it.
You wrote:
Refactored await expect(somePromise).resolves.toEqual to expect(await somePromise).toEqual - much less verbose and confusing mixing resolves with await
I like this! Where is it? [BTW, I should update the async lecture to incorporate this]
Slides 17, 18, 19, (numbers correspond to the latest version of the slide deck, not what you reviewed)
Got it.  It would be useful to point out that this is different from what we wrote in the async lecture.
Thanks for all of the great suggestions and feedback! However, just to manage expectations: it is unlikely that I will have bandwidth for another round of revisions this semester (but will save the notes for next term).
If you would like, I can make the tweaks I proposed (s1, s15, s17notes, s19notes).  I can make it a separate branch, or just do it in module12-larger-tests.  Let me know.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

added bullet per Adeel's comment above

merging.   I may do some more rewriting later tonite, but I'd rather have this all nicely in main.


Freeze on Windows 10

(I see similar, recent issues posted about crashes on windows, so please ignore/close if this bug report is redundant :-) )
When using the latest 1.29.1, interacting with the application quickly starts to spike CPU and then causes Windows to hang, resulting in a forced reboot. Interactions I tried that ultimately caused this are: typing code in an existing file, unfolding a folder in the folder tree, jumping to a file by typing the start of the file name. The application is usable for a short while, with high CPU before the system/OS becomes unresponsive.
I was able to disable the extensions manually, within the application. With disabled extensions the issue remains.
I guess this is not a lot of information, but please feel free to ask certain details.
Thanks in advance for looking into this :-)

VSCode Version: 1.29.1
OS Version: Windows 10 Enterprise

Steps to Reproduce:

open VSCode, last opened folder opens automatically
interact with the code base, eg:
2a. type code in a file
2b. unfold a folded folder
2c. goto a file with Ctrl+P
2d. possibly other interactions too
continue to do this for a brief period, CPU spikes and within 1 minute OS hangs

Does this issue occur when all extensions are disabled?: Yes

Can you start a profile recording then reproduce? ctrl + shift + P -> Toggle Developer Tools -> Performance -> Record.
In task manager, can you see if CPU is spiking on a process?

After filing the issue, my colleagues and I found out the virus scanner active on the systems is interfering with pretty much anything related to the file system in VSCode. Unfortunately, this did not show up clearly in the task manager. I'll close this issue.
My apologies for the confusion, and thanks for getting back to me! :-)


Add mac arm64 wheels

use SSC self-hosted mac arm64 runner to build arm64 wheels
only build x86_64 wheels on x86_64 macos-11 runner
use python 3.9 as 3.8 requires rosetta to run on arm64 mac
remove CMAKE_OSX_ARCHITECTURES logic as it is no longer required

Main benefit is that we can now test the arm64 mac wheel in CI (also much faster, CI job took 3min)
I pip installed https://test.pypi.org/project/clang-format/17.0.3/ on the m2 mac mini & seemed to work fine.

Thanks a lot! Great to finally get this problem solved!


option to disable keypress listener on linux.

Exception in thread Thread-4 (keypress_listener_thread):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/crypt/ssd1/chess-auto-bot/src/gui.py", line 349, in keypress_listener_thread
    if keyboard.is_pressed("1"):
  File "/home/crypt/ssd1/chess-auto-bot/venv/lib/python3.10/site-packages/keyboard/__init__.py", line 410, in is_pressed
    _listener.start_if_necessary()
  File "/home/crypt/ssd1/chess-auto-bot/venv/lib/python3.10/site-packages/keyboard/_generic.py", line 35, in start_if_necessary
    self.init()
  File "/home/crypt/ssd1/chess-auto-bot/venv/lib/python3.10/site-packages/keyboard/__init__.py", line 196, in init
    _os_keyboard.init()
  File "/home/crypt/ssd1/chess-auto-bot/venv/lib/python3.10/site-packages/keyboard/_nixkeyboard.py", line 113, in init
    build_device()
  File "/home/crypt/ssd1/chess-auto-bot/venv/lib/python3.10/site-packages/keyboard/_nixkeyboard.py", line 109, in build_device
    ensure_root()
  File "/home/crypt/ssd1/chess-auto-bot/venv/lib/python3.10/site-packages/keyboard/_nixcommon.py", line 174, in ensure_root
    raise ImportError('You must be root to use this library on linux.')
ImportError: You must be root to use this library on linux.
You must be root to use this library on linux.
<class 'ImportError'> stockfish_bot.py 195

Linux users can't just run it with sudo, this will cause issues with graphical apps that use X (chrome, for example).

index 200e4c7..96588c9 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -5,5 +5,4 @@ PyAutoGUI~=0.9.53
 chess~=1.9.3
 stockfish~=3.28.0
 packaging~=21.3
-keyboard~=0.13.5
-PyQt5~=5.15.7
\ No newline at end of file
+PyQt5~=5.15.7
diff --git a/src/gui.py b/src/gui.py
index 3b1fab5..c91a1c1 100644
--- a/src/gui.py
+++ b/src/gui.py
@@ -12,7 +12,6 @@ from webdriver_manager.chrome import ChromeDriverManager
 from overlay import run
 from stockfish_bot import StockfishBot
 from selenium.common import WebDriverException
-import keyboard

 class GUI:
@@ -346,11 +345,6 @@ class GUI:
             if not self.opened_browser:
                 continue

-            if keyboard.is_pressed("1"):
-                self.on_start_button_listener()
-            elif keyboard.is_pressed("2"):
-                self.on_stop_button_listener()
-
     def on_open_browser_button_listener(self):
         # Set Opening Browser button state to opening
         self.opening_browser = True
diff --git a/src/stockfish_bot.py b/src/stockfish_bot.py
index 415c599..db08110 100644
--- a/src/stockfish_bot.py
+++ b/src/stockfish_bot.py
@@ -11,7 +11,6 @@ import re
 from grabbers.chesscom_grabber import ChesscomGrabber
 from grabbers.lichess_grabber import LichessGrabber
 from utilities import char_to_num
-import keyboard

 class StockfishBot(multiprocess.Process):
@@ -192,9 +191,6 @@ class StockfishBot(multiprocess.Process):
                             ((int(move_start_pos[0]), int(move_start_pos[1])), (int(move_end_pos[0]), int(move_end_pos[1]))),
                         ])
                         while True:
-                            if keyboard.is_pressed("3"):
-                                break
-
                             if len(move_list) != len(self.grabber.get_move_list()):
                                 self_moved = True
                                 move_list = self.grabber.get_move_list()

quick and dirty attempt to get rid of keyboard. Bot is now starting but only detecting a game against bots. If I run it in a game against other players if i logged in to chess.com. Playing as a guest against other players work just fine.


OCAT SystemInfo: macserial using serial of active system rather than serial of Config.plist? Check serial online?

Inquiry
What is the purpose of macserial using the serial of the active system on which OCAT is running rather than using the serial of the loaded Config.plist?
In OCAT -> PlatformInfo -> SystemInfo i see the output of the macserial utility.
As a user I expected it to use the serial I set in PlatformInfo -> Generic, as this would give me some information about the serial I am using in my Config.plist. I suppose you had a reason for using the system serial, but I never had a use-case for that yet.
Feature Request
On a related matter, it would be nice to have a button somewhere in the PlatformInfo section of OCAT linking to the Apple Check Your Service and Support Coverage page where we can make sure that the generated serial has not been used on a real Mac. I think the link format for submitting the serial rarely ever changes:
Example: https://checkcoverage.apple.com/?sn=C02C30JEPN5T

System information is a real thing, and the serial number inside the configuration file you opened, if it is not running on a real machine, then please ask how to generate system information?
Querying serial number is a good idea, I will add this feature, thanks.

By the way, there is a question that I need to clarify.
Is it considered a violation to explicitly direct a user in an app to go to Apple's website to look up a serial number that may not exist? Or is there any legal risk? Thanks.
If there is any legal risk or violation, I will remove the serial number lookup feature.
I personally feel that adding this feature seems a bit inappropriate.

Is it considered a violation to explicitly direct a user in an app to go to Apple's website to look up a serial number that may not exist? Or is there any legal risk? Thanks. If there is any legal risk or violation, I will remove the serial number lookup feature.

Personally, I do not think there is a legal violation for using their website to look up a serial number from an app, but IANAL ;-) - I have seen such functionality in other similar GUI apps with a 'Check Coverage' button, which forwards the serial using the link syntax I used above. Then the user only needs to solve the Captcha. I don't think that Apple ever complained, as these apps have been around for years.
I believe, that this functionality actually protects Apple, the owners of the real serial and the hackintosh community. If OCAT users fail to check their serial and accidentally use an active serial number your app created, it could actually cause more problems for Apple and the hackintosh community, as some people's Apple IDs may get blocked. The url opens in a user's browser, so the actual request from their database is done by the user not done by your app.

Rename the SystemInfo Tab to Serialnfo Tab (because it basically calls macserial), and use the serial from the Config.plist in the PlatformInfo section.

You are a very meticulous person and the first to question this detail, thanks.
In fact, I basically didn't think hard about this module when I was working on it before.
I am very much in favor of changing the system information to serial number information.
Feel free to find more questions or suggestions.

Feature added.


Action bar not showing up

Hi ,
I've added the sliding panel to start from the top but I'd like to have the action bar as well. I've been having issues with style theme to get this to work. Any recommendation as to how this should work?

Use Toolbar like in the demo.


Search in all files doesn't show results when interupted by new search

VSCode Version: 1.30.0 (user setup)
OS Version: Windows 10 1803

Steps to Reproduce:

Open a large folder in vs code so the search takes a while
Click the magnifying glass
Search anything
Modify the search term of files to include or exlcude
Press enter before de previous search is done.

The message 'search was canceled before any results could be found' is shown'.
Clicking 'Search again' or attempting to search by pressing enter or the refresh button does not search again.
After a while the message will change to a number of results, which is te same number as a successful search of the current query, but no results will be displayed. Searching again from this point will work.

Does this issue occur when all extensions are disabled?: Yes

https://github.com/Microsoft/vscode/issues/65084 Fixed in the new 1.30.1 release


⚠ Samsung has degraded performance

In a28fc62, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 1840 ms

Resolved: Samsung performance has improved in 3623d0a.


Release/8.0.4

.

Hello @creatorsoft, is this submitted by mistake. I am not sure what we are trying to fix here. Pls read submit guidelines here: https://github.com/dnnsoftware/Dnn.Platform/blob/master/CONTRIBUTING.md


DirectionsStep doesn't have path prop (array of coordinates)

DirectionsStep should have a path prop that contains an array of coordinates for the step in the leg of a route.
https://developers.google.com/maps/documentation/javascript/directions
gm.directions(
  {
    origin: 'Madison , Wi, USA',
    destination: 'Tampa, FL, USA',
    mode: 'driving'
  }, (err, result) => {
    //result.routes[0].legs[0].steps[0].path is undefined
  }
)

It does return the encoded polyline but from my understanding, this is smoother (less accurate) than the actual path coordinates.
encode_polylines:   false .. not sure if that matters

Nevermind. I was confusing the Google Maps Directions API (used by this library) with Google Maps JavaScript API,

@jlc467 can you provide me with a complete example. Thanks.


nixos/geoip-updater: new service

The GeoIP Legacy databases from MaxMind have no stable URLs and change
every month (or so). Our current method of packaging these database in
Nix and playing catch-up with the ever-changing file hashes is a bad
idea. For instance, it makes it impossible to realize old NixOS
configurations.
This patch adds a NixOS service that periodically updates the GeoIP
databases in /var/lib/geoip-databases. Future patches will move users
over.

I guess there is little point in this service without any of the geolite-legacy users (packages) moving over to it. Hence this is WIP for now; it needs testing, community feedback and moving some packages over to use this runtime dependency, instead of using the current "geolite-legacy" Nix attribute (which breaks every month).

My suggestions:

make it possible geoip = pkgs.geoip.override { geoipDatabase = config.services.geoip-databases.databaseDir; }
fetch&unpack geoip2 dbs too, GeoLite2-Country.mmdb.gz and GeoLite2-City.mmdb.gz. I am using the v2 now in my own update script.
maybe package some geoip2 tooling? libmaxminddb, mmdblookup. The former is a better library when you want to create new applications using geoip API. I have a working expression for it:

  libmaxminddb = stdenv.mkDerivation rec {
    name = "libmaxminddb-${version}";
    version = "1.2.0";
    src = fetchurl {
      url = "https://github.com/maxmind/libmaxminddb/releases/download/${version}/libmaxminddb-${version}.tar.gz";
      sha256 = "0dxdyw6sxxmpzk2a96qp323r5kdmw7vm6m0l5a8gr52gf7nmks0z";
    };
  };

What about nix outside nixos?

I guess we can expose the updater script to the user (make a package), so it can be run manually.

@bjornfor Is missing adoption the only argument holding this back from merging? What about the use cases where the geolite DB is referenced in extraConfiguration by the user, not by the package itself. I would really like to see this merged and I think adoption will follow afterwards.

If people find this useful as is, I'm fine with merging (of course).

@bjornfor You have my +1 for usefulness.

I noticed that the service would fail started at boot. It seems to be running before the network is up. I looked at man systemd.special and tried to fix it with
-      after = [ "network.target" ];
+      after = [ "network-online.target" "nss-lookup.target" ];
+      wants = [ "network-online.target" ];

It doesn't make a difference, it still fails. Also had a look at https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ (suggested by the man page). But not liking the options.
Any ideas?
I guess either I'll add a "try a few times" loop or dig into the systemd special targets and how they are implemented on NixOS. (Perhaps we have a network-online.target dependency bug?)
(I added some error handling so that we'll know when the service fails.)

@bjornfor I think network-online.target will become only useful, if you use systemd-networkd:
$ systemctl cat systemd-networkd-wait-online.service

dhcpcd does fit into this.

Hm, looks like I introduced a regression when adding syslog priority levels with "logger". systemd doesn't understand which unit the messages come from. So "systemctl status geoip-updater" doesn't show any messages. But the journal contains all messages (with proper error levels).
I'll see if I can make it work with "logger", or else find another solution. I really want to have "journalctl -b -perr".

I've rebased and cleaned up. I consider it done.

@danbst: You added one GeoIP2 database to the list (GeoLite2-Country.mmdb.gz). Where did you find it (it's url/name)? Should we add more? A quick look at maxmind.com/geoip indicates a subscription fee for updated databases. Do you have any commens about what we can or cannot add to NixOS?
Because of the one GeoIP2 db in the current module, I removed the word "Legacy" from the service description.

I've found it at http://dev.maxmind.com/geoip/geoip2/geolite2/ . Yes, we can add city database too
As for payments - they are required for so called "developer package". As long as we don't redistribute databases copies, we are free from royalty fee. When downloading from maxmind directly, they are free.

@danbst: Thanks. I added the "city" database to the list.
A couple of issues that are still nagging me:

In addition to running when the timer triggers, it also runs on every boot. This was intended to deal with the initial setup, but as is it runs more often than desired. I looked at systemd timer config and found nothing appropriate. I did manage to get something different (not necessarily better) by adding a 2nd service with ConditionPathExists on the databaseDir, so that we have a initial setup run unless that dir exists. But then the service isn't run when the database list changes...

If entries are changed/removed from the database list, stale database files will be left in /var/lib/geoip-databases.

I'll have a look in the next days and see if I come up with something to address this.

Updated again. From commit message:
   Changes v5 -> v6:
      - Update database files atomically (per DB)
      - If a database is removed from the configuration, it'll be removed
        from /var/lib/geoip-databases too (on next run).
      - Add NixOS module assertion so that if user inputs non- .gz or .xz
        file there will be a build time error instead of runtime.
      - Run updater as user "nobody" instead of "root".
      - Rename NixOS service from "geoip-databases" to "geoip-updater".
      - Drop RemainAfterExit, or else the timer won't trigger the unit.
      - Bring back "curl --fail", or else we won't catch and log curl
        failures.

Perhaps we should bring back proxy service, that has wantedBy = multi-user.target and performs update-script with parameter "--update-if-not-exists". The parameter "--update-if-not-exists" won't download DB if file is already downloaded.

Another update:
Changes v6 -> v7:
      - Add --skip-existing flag to geoip-updater, which skips updating
        existing database files. Pass that flag when we run the service on
        boot (and on any NixOS configuration change).
        (IMHO, this is somewhat a workaround for systemd persistent timers
        not being triggered immediately when a timer has never expired
        before. But it does have the nice side effect of ensuring that the
        installed databases always correspond to the configured ones, since
        the service is now always run after configuration changes.)

Hm, I'm doing some testing and get this on rebuild
$ nixops deploy
building all machine configurations...
jolly.roger> copying closure...
fleet> closures copied successfully
jolly.roger> updating GRUB 2 menu...
jolly.roger> activating the configuration...
jolly.roger> setting up /etc...
jolly.roger> the following new units were started: geoip-updater-setup.service
jolly.roger> activation finished successfully
fleet> deployment finished successfully

This line the following new units were started: geoip-updater-setup.service is printed on every redeploy, even if nothing changed.

RemainAfterExit is required in setup service, so it won't be started on every activation, only when service changed.
Btw, really nice log output!
-- Logs begin at Thu 2016-09-01 10:06:58 UTC, end at Thu 2017-01-12 15:33:20 UTC. --
Jan 12 15:32:50 jolly.roger systemd[1]: Starting GeoIP Updater Setup...
Jan 12 15:32:50 jolly.roger systemd[1]: Started GeoIP Updater Setup.
Jan 12 15:32:50 jolly.roger geoip-updater[11112]: Option --skip-existing is set: not updating existing databases
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Server is reachable (try 1)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Starting update of GeoIP databases in /var/lib/geoip-databases
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Skipping existing file: /var/lib/geoip-databases/GeoLite2-Country.mmdb
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Removed GeoIPASNum.dat (not listed in services.geoip-updater.databases)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Removed GeoIPASNumv6.dat (not listed in services.geoip-updater.databases)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Removed GeoIP.dat (not listed in services.geoip-updater.databases)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Removed GeoIPv6.dat (not listed in services.geoip-updater.databases)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Removed GeoLite2-City.mmdb (not listed in services.geoip-updater.databases)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Removed GeoLiteCity.dat (not listed in services.geoip-updater.databases)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Removed GeoLiteCityv6.dat (not listed in services.geoip-updater.databases)
Jan 12 15:32:51 jolly.roger geoip-updater[11112]: Completed GeoIP database update in /var/lib/geoip-databases

RemainAfterExit works really strange. On first activation geoip-updater-setup.service is started. On timer trigger it is stopped (because it conflicts with geoip-updater.service), so it will be started again on next activation. I hope this is minor issue.
Another strange consequence of setting RandomizedDelaySec=3600 is it affects interval in a non-obvious way when you want to test this service. For example, if I set interval to minutely, I don't get update every minute (every random 0-3600 seconds). So automated test (which is to be written) should unset RandomizedDelaySec to make test not time out.

I noticed some timer weirdness too, with short interval.
Just pushed an update with RemainAfterExit=true. I guess it's better that way. (Still, the design with the two services and the "conflict" feels a bit odd...)
Is having a test for this service really needed? I was under the impression that tests couldn't use (external) networking, so either I'm wrong and we can write a test (if wanted) or else we have to write the test so that it creates a fake maxmind.com server.

The only thing we can do with timers and short intervals is to specify in interval option description, that it adds randomized delay to that interval, so it will be inprecise for short intervals.
As for test, I think you're right, it will be hard to write such test. ping @kampfschlaefer for his opinion on using fake maxmind server.

if RandomizedDelaySec is not a good fit, I would suggest instead (this has an even better distribution as a it depends on the boot of

(accident)

@Mic92:

if RandomizedDelaySec is not a good fit, I would suggest instead (this has an even better distribution as a it depends on the boot of

I personally would prefer to do the update at a fixed time, preferably at night.

MaxMind say the (open source) databases are updated on the first tuesday of the month. So we actually only need to download once a month.
Because I don't like that the current service run on every boot, I've been thinking of this:

One service is triggered by timer, once a month.
One service is triggered by the /var/lib/geoip-database directory not existing (for initial setup).

Pro:

No needless running the service when there is nothing to do

Con:

A service configuration change will not cause a (re)run. I.e. if you change the list of databases, it won't be reflected in /var/lib/geoip-databases until next timer interval. (Or you trigger it yourself.)

I really like /var/lib/geoip-databases to be as accurate to the configuration as possible. But even with the current scheme (run every week, run on every boot), there is no guarantee that things are OK. You still have to check if the service ran successfully. It's doing impure IO, after all :-)  Hence, I'm thinking it might be OK to just run the initial setup service once. If it fails that time, then oh well, look at "systemctl --failed" and restart it.
Thoughts?

Can this be merged as-is, and tunings/thoughts separated to another issue? I find this service useful in it's current form (and I use it on-top of 16.09).

Applied to master: 824d82fa0f6fdd1b2213020283d8bd7aa77738b1
I see no problem with cherry-picking this to release-16.09. I'll do that later if nobody objects.

Awesome! btw, much thanks @bjornfor for your initiative.


pass parameter in SendAsync

Hi Team
I want to pass additional parameter to MakeJsonRootDialog function
string strPar = "SomeText";
await Conversation.SendAsync(activity, ControllerHandler.MakeJsonRootDialog(strPar));
public static IDialog MakeJsonRootDialog(string strParam)
{
}
I got compile error :
Error	CS1503	Argument 2: cannot convert from 'Microsoft.Bot.Builder.Dialogs.IDialog<Newtonsoft.Json.Linq.JObject>' to 'System.Func<Microsoft.Bot.Builder.Dialogs.IDialog>'
Thanks

You can use a closure to capture that extra parameter.  You can't change the interface of that dialog root factory method because we wouldn't know how to pass those extra parameters to it anyway.

Can you give example - is it mean that I need to put all MakeJsonRootDialog code inside of closure?

You can capture the closure in a lambda expression like this:
string strPar = "SomeText";
await Conversation.SendAsync(activity, () => ControllerHandler.MakeJsonRootDialog(strPar));

public static IDialog MakeJsonRootDialog(string strParam)
{
}

thanks a lot


[new-rule-option] Introduce new typedef rule option (variable-declaration-ignore-function)

PR checklist

[x] New feature #2654

[x] Includes tests

[x] Documentation update

Overview of change:
Add new option for typedef rule.
Ignores typedef requirement for vars for arrow and non arrow functions.

CHANGELOG.md entry:
[new-rule-option] typedef new option: variable-declaration-ignore-function

Thanks for your interest in palantir/tslint, @michaelw85! Before we can accept your pull request, you need to sign our contributor license agreement - just visit https://cla.palantir.com/ and follow the instructions. Once you sign, I'll automatically update this pull request.

Hi yes, sorry for the delay! I will soon 🙂.
For context, both adidahiya and I are maintaining TSLint in roughly part time amounts. There's also a lot of ongoing work for migrating to ESLint. So this might take a few days more.

@JoshuaKGoldberg Thanks for the reply. Don't worry there is no rush (for me) although I would definitely be happy to have this feature. I wasn't sure if anyone noticed the PR so I thought I would tag you since you categorized the linked feature request.

@JoshuaKGoldberg PR is updated! Thanks for reviewing.

testNext failures are unrelated; will file a separate issue.


chore: use uppercase in Dockerfile

The AS instruction is used to assign a stage name in a multi-stage build. While using uppercase for AS is not strictly enforced, it is a common convention that makes it consistent with other instructions and improves readability.

Thank you for the fix and merge - I didn't have access to my laptop to amend the commit earlier :/


this.$store is undefined in components in migration build

Version
3.1.0-beta.7
Reproduction link
https://github.com/RyanClementsHax/vue-compat-vuex-bug
Steps to reproduce
Follow the steps outlined here to upgrade a vue 2.x app using vuex to vue 3, and use @vue/compat v3.1.0-beta.7
What is expected?
The store is accessible in components via this.$store after migration build is installed and compiler errors are dealt with in the console.
What is actually happening?
this.$store is undefined in components after the migration build is installed and compiler errors are dealt with.

I could be reading the @vue/compat instructions incorrectly, but it seems from them that this shouldn't be happening. Am I wrong? I very well could be.

You need to upgrade to Vuex 4 (point 8)

Remember to use the forum or the Discord chat to ask questions!

Oops thanks!

Vuex 3 should also work with the migration build in theory.

Hello Guys! Con you please confirm that compat build not working with vuex 3. Thx a lot!


Vulnerability roundup 85: qemu-5.0.0: 1 advisory [6.5]

search, files

[ ] CVE-2020-10717 CVSSv3=6.5 (nixos-unstable)

Scanned versions: nixos-unstable: 22c98819ccd. May contain false positives.
Cc @edolstra

Version 5.0.x is nowhere in nixpkgs
Closing.


Replace isPanScrollEnabled with shouldRespond(to ..

Replace isPanScrollEnabled with func shouldRespond(to panModalGestureRecognizer: UIPanGestureRecognizer) -> Bool, this would allow for specific views to ignore touches to the outer modal e.g. in the case of UIPickerView

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Add more Plotter features

Introduction
Current Plotter interface is in Proof of Concept state, enough to test initial GUI approach.

DoD — Plotter interface allows complex visualizations of mathematical problems; users can draw shapes (lines, rectangles, circles, ..) and plot functions.

Tasks

[x] Add more subprograms and parameters

Suggestion: overload subprograms to add minor parameters: color, width, style .. instead of duplicating HTML-Canvas API

Add more subprograms and parameters
Do not copy Canvas API
Overload subprograms to add minor parameters: color, width, style ..

Very basic functionality added.


Honor new variable IDE_OPTIONS in ide command wrapper

As a IDEasy user, I want to have the flexibility to configure specific aspects like HTTP proxy servers, proxy excuses, truststore with passphrase, etc. so that I can make IDEasy work also in my environment limited by corporate VPN.
See #245 and #774
The idea of this story is to add a environment variable IDE_OPTIONS and adapt our bash and CMD code calling ideasy binary to pass whatever options are defined in this variable.
To make it crystal clear:
For bash we change this line:
https://github.com/devonfw/IDEasy/blob/6b41eefddd9a396ee5f1bf6d7509e21c78c6d4cb/cli/src/main/package/bin/ide#L3
to
_IDEASY="$(dirname "${BASH_SOURCE}")/ideasy ${IDE_OPTIONS}"

And for CMD we do the same here:
https://github.com/devonfw/IDEasy/blob/6b41eefddd9a396ee5f1bf6d7509e21c78c6d4cb/cli/src/main/package/bin/ide.bat#L17
https://github.com/devonfw/IDEasy/blob/6b41eefddd9a396ee5f1bf6d7509e21c78c6d4cb/cli/src/main/package/bin/ide.bat#L9
by adding %IDE_OPTIONS%.
Further a PR implementing this story shall extend the documentation by adding the new variable here:
https://github.com/devonfw/IDEasy/blob/main/documentation/variables.adoc
Explaining the new variable and also linking to this page:
https://github.com/devonfw/IDEasy/blob/main/documentation/proxy-support.adoc
Then also the proxy support page needs to be updated by adding details about according JVM options and linking
https://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html
https://docs.oracle.com/cd/E29585_01/PlatformServices.61x/security/src/csec_ssl_jsp_start_server.html
(maybe we find more recent or accurate links).
Also we should give examples for common situations.
A further improvement would be to automatically honor http_proxy, https_proxy, all_proxy, and no_proxy and trying to convert that to according JVM args if not already present in IDE_OPTIONS and adding them automagically so it works in most cases out of the box.

This issue was implemented with PR #811.
Hence, closing.


DiffuSeq-v2 checkpoint release

Hi, are there any plans to release the DiffuSeq-v2 QQP model?

Yeah, I will update this soon.

Hi, great work, where is the checkpoint

Hi, another ping here. Will there be a plan to release the checkpoint anytime soon ?


[FEATURE] Add "Apply policy" action on data streams page

Is your feature request related to a problem?
There is no easy way to apply a policy to an already created data stream.
What solution would you like?
While on data streams page, be able to select my data stream, click on "Actions" and have a new "Apply policy" item there. The expectation is that it applies it on all the internal indexes of the data stream.
What alternatives have you considered?
Go to Indexes page, toggle "Show data stream indexes", click on "Data streams", select my data stream, select all the internal indexes of my data stream, click on "Actions" and "Apply policy".
Do you have any additional context?

[Catch All Triage - 1, 2, 3, 4, 5]


fix(deps): update dependency antd to v4.24.16

This PR contains the following updates:

Package
Type
Update
Change

antd (source)
dependencies
patch
4.24.15 -> 4.24.16

antd (source)
dependencies
patch
4.24.14 -> 4.24.16

Release Notes

ant-design/ant-design (antd)
v4.24.16
Compare Source

🐞 Fix Select that scrollbar is not displayed when there are few options. #47050
🐞 Fix Transfer cannot invert current page correctly. #47134 @linxianxi
🐞 Fix that clicking Form tooltip icon should not trigger Switch. #46159 @Wxh16144
🐞 Fix Segmented options className being overrided by className. rc-segmented#175 @stoil-terziev

🐞 修复 Select 组件在选项较少时不显示滚动条的问题。#47050
🐞 修复 Transfer 反选当页错误的问题。#47134 @linxianxi
🐞 修复点击 Form tooltip 图标会触发 Switch 切换的问题。#46159 @Wxh16144
🐞 修复 Segmented options 内 className 会覆盖组件自带 className 的问题。rc-segmented#175 @stoil-terziev

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about these updates again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Renovate Ignore Notification
Because you closed this PR without merging, Renovate will ignore this update (4.24.16). You will get a PR once a newer version is released. To ignore this dependency forever, add it to the ignoreDeps array of your Renovate config.
If you accidentally closed this PR, or if you changed your mind: rename this PR to get a fresh replacement PR.


fix: unhandled exception in Fody on netstandard

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Bug fix
What is the current behavior? (You can also link to an open issue here)
https://github.com/kswoll/ReactiveUI.Fody/issues/26
What is the new behavior (if this is a feature change)?
ReactiveUI.Fody will actually work on netstandard
What might this PR break?
Nothing
Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

Other information:
I noticed this fix isn't included in #1503 so I just wanted to make sure it got in there. I included it in my recent netstandard PR to ReactiveUI.Fody.

Thank-you gtbuchanan!


reflect: add Value.Clear

I propose adding a new method MapClear to reflect.Value, which efficiently deletes all elements from a map.
I am working on a decoder for a serialization format. If I want decoding into a non-nil map to leave it with only the serialized elements, I currently have two choices: 1. Allocate a new map, or 2. Clear all keys.
Clearing the keys has the advantage that it re-uses existing storage, if any, so it is lighter on allocations. But it takes more time to iterate over the map and clear keys. It also isn't correct, strictly speaking, as it doesn't behave right with NaN-keys.
It would be preferable to have a way to efficiently clear the map, preserving the already allocated buckets. This isn't an operation available in the Go language, so arguably it does not belong in reflect either. But we have #20138 to at least make the iteration-and-deletion idiom transparently efficient in plain Go.

cc @dsnet

Interesting observation about NaN. It does mean that the documentation on maps.Clear is somewhat lying: https://pkg.go.dev/golang.org/x/exp/maps#Clear

FWIW that observation was made by @randall77 here, I'm not taking credit.

Suddenly I like #20660.

It's definitely a little odd to add something to reflect that you can't do in the language proper. Seems like we should fix that somehow first.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

BTW, now we can modify slice length and capacity separately in reflect, but which is unable to do in the language.

You can do it in the language, it's just weird.
    s = s[:len(s):100]
    s = s[:100:cap(s)]

@ianlancetaylor we can't modify slice length and capacity separately in reflect. In reflection, we can.

@ianlancetaylor
In you example code, length and capacity are modify at the same time, though one of them is the same as the old value.

@go101 That seems a very flimsy argument. With s = s[:len(s):5] you do exactly the same as with SetCap(5). Claiming it's something the language doesn't allow you to do "because one of them is the same as the old value" is splitting hairs. Either way, it's off-topic here.

The we may also think language also allow us to do map clear. :)
Off topic.

@rsc For my purposes, I'd also be okay with having MapClear() not remove NaN keys. That is, for my purposes, I'd be fine having to do
v.MapClear()
if v.Len() > 0 { // there where some NaNs in that map
    v.Set(reflect.New(v.Type()).Elem())
}

That way, MapClear wouldn't allow anything the language doesn't allow. But I agree that's kind of a strange API. I'd prefer to find a way to address this NaN-key thing in the language proper as well.

There are never any good answers involving NaNs.
Perhaps we should make maps.Clear do an actual clear, including NaNs (with runtime help as needed)
and then reflect.MapClear can also do an actual clear, including NaNs.
Any objections to changing both?

Personally,

would

Personally, I would like something like that. However, this does open the door to other functionality in maps that might hook into the runtime. Where do we draw the line?
For example, #54454 proposes maps.Shrink which would need runtime support to work.

Where do we draw the line?

An arbitrary suggestion: maps shouldn't do anything that user code couldn't already do using reflect, but it can maybe use runtime hooks to cheat at optimization.
Then adding reflect.MapClear would pave the way for maps.Clear, because it would provide a stable, supported way for user code to provide the same semantics (if a little slower).

An arbitrary suggestion: maps shouldn't do anything that user code couldn't already do using reflect,...

That's kind of just kicking the can down the road one step. I would argue that reflect shouldn't be able to do anything that can't be done in the language proper, which argues against reflect.MapClear being able to clear NaN keys.
I'm ok with maps.Clear and reflect.MapClear existing but not clearing NaN keys. We just need to document that and explain why (basically, because delete(m, k) doesn't do anything for NaN keys).

One possible halfway point would be to have maps.Clear panic if it is unable to actually clear the map. That wouldn't go quite as far as preventing them from being put in the map in the first place, but would at least fail in a way that is easy to fuzz and diagnose (as opposed to silently leaving entries in an allegedly-cleared map).

A bit semantically strange, the existence of map.Clear that wouldn't actually clear the map.
For now that operation doesn't really exist (in the std lib) so everything is still semantically consistent if perhaps confusing.
If there was such an operation, I would expect it to work on clearable maps, i.e. where map keys are proven to have reflexive comparison.
Otheriwse that could easily leak into Set types for example, especially when computing cardinality. (sets of floats or floats composites).
But then if it requires modifying map.Clear from an x package, how many programs it would break is another question too perhaps.

@bcmills to me, that feels like the kind of perfect compromise in that it leaves everyone unhappy. Especially as it doesn't seem possible to check if maps.Clear would panic, without iterating over it and finding a NaN key (and avoiding the O(N) iteration cost is why I want to use it).

Me neither, but if the issue appears, it might for any type which is a composite of floats values. Not just directly float32 or float64.
What if a pointer type is stored for a struct where one field is a NaN float64 for instance?
While interfacing with javascript via wasm, I hadn't thought about that but that might occur more often. I think I need to write NaN checks when comparing objects.

@Merovius, while it's true that one couldn't always check for the panic locally immediately before making the call to Clear, it is still straightforward to avoid adding irreflexive keys to the map in the first place by checking k == k at the insertion site.

@bcmills I don't like that. As a library author using maps.Clear/reflect.(*Value).MapClear that would require me to document that maps passed to me can't contain NaN keys. But that limitation is not inherent to my use-case - it would be less efficient, but semantically completely fine to just allocate a new map. I guess I can use recover - but I don't think "you have to think about recover and fall back, if clearing panics" makes for safer code than "you have to check if len > 0 after clearing and fall back, if there are NaNs".
Not to mention that checking if an inserted value has this problem isn't always easy.
@mvdan I tend to agree. I feel that right now, this issue - at least as far as NaN is concerned - is simply blocked on #20660 - i.e. "somehow decide what to do about NaN in map keys":

If we could convince ourselves to disallow inserting them, it would no longer be in doubt how maps.Clear or reflect.(*Value).MapClear would handle the situation - it can't happen.
If we decided that NaN-keys are allowed but (counter to the spec) compare as equal in that context, both maps.Clear and reflect.(*Value).Clear would have an obvious behavior.
If we decide that the current behavior of NaN-keys is what we want, well, then maps.Clear and reflect.(*Value).Clear might just not have a good definition in the presence of NaN-keys

@Merovius

Not to mention that checking if an inserted value has this problem isn't always easy.

It is always easy: if k != k.

But that limitation is not inherent to my use-case - it would be less efficient, but semantically completely fine to just allocate a new map.

That may be true, but if a caller puts an irreflexive key in a map they're almost certainly going to have trouble with way more than just your library. (How many other Go libraries assume that they can range over a map and look up / delete its values by key? I would bet it's a significant fraction of the ecosystem.)
So I don't think users of your library would lose much, if at all, by having the additional “keys must be reflexive” restriction.

@atdiar

What happens if a pointer value is stored for a struct key where one field is a NaN float64 for instance? Isn't it now dangling?

No, it can still be accessed using range.
Maybe "dangling" was a poor choice of a word.
What I meant is the below
https://go.dev/play/p/rJ3UucF1Zlx

Yes, that's what the discussion is about, that it's not possible to delete a NaN key from a map. Otherwise we wouldn't have a problem with maps.Clear and (*Value).MapClear.

I get that of course.
The point is that any type that is a composite of something that "might" hold NaN has te same issue.
Not just NaN.

Correct.

@bcmills wrote:

An arbitrary suggestion: maps shouldn't do anything that user code couldn't already do using reflect, but it can maybe use runtime hooks to cheat at optimization.

@randall77 wrote:

That's kind of just kicking the can down the road one step. I would argue that reflect shouldn't be able to do anything that can't be done in the language proper, which argues against reflect.MapClear being able to clear NaN keys.

It seems like there are three possibilities here:

Allow maps.Clear and reflect.Value.MapClear to clear all entries even though the language can't.
Make maps.Clear and reflect.Value.MapClear preserve NaN-containing keys to match the language.
Add a builtin form - maybe delete(m), as a unary function - to the language, and then reflect.Value.MapDeleteAll can provide the reflect form of delete(m).

Bryan and Keith dislike (1) but would be OK with (2) and presumably also (3), which keeps the language as powerful as the reflect library.
What do people think about (3), adding delete(m) to the language?

Ok by me. It does seem like a big hammer for a weird corner case.

Not just essentially, exactly that mechanism with exactly that syntax was suggested. Interesting.
Obviously, delete(m) would fit my use-case and I'd be fine with it, personally.

I think it's fine given that the NaN keys and wanting a reflect API are both factors which weren't properly considered in the previous proposal.
@rsc in your option 3, would we end up with one or two new reflect APIs? My intuition is that "clear the map except NaN keys" is not worthwhile.

If NaN containing keys are always a mistake, then perhaps that (2) is sufficient and the actual issue is to prevent these NaN containing keys in the first place.
If there are legitimate cases for these NaN containing keys, then (3) should be fine.
I am more inclined toward option 2.

If NaN containing keys are always a mistake, then perhaps that (2) is sufficient and the actual issue is to prevent these NaN containing keys in the first place.

That is pretty much exactly #20660, and I think it's also a good option. (But I may be biased because I filed it. 😅)

Make maps.Clear and reflect.Value.MapClear preserve NaN-containing keys to match the language.
…
Bryan and Keith dislike (1) but would be OK with (2)

Just to be clear, I have a strong preference against that particular variation on option (2).
Instead, I would prefer:
2a. Make maps.Clear and reflect.Value.MapClear panic if they are unable to clear the map.

https://github.com/golang/go/issues/45328 was declined in part because we said the loop was just as good. But now we've realized that the loop does not handle everything, which is why we're back to a language change.

Created #56351 for the language change. If that goes through, we'd add reflect.(*Value).MapClear to do the same thing (clear the map entirely, even NaNs). We wouldn't need maps.Clear(m) because delete(m) already does it.

#56351 is accepted (soon), so we should revive this. Because clear(x) applies to slices and maps, we should call this Value.Clear and then it applies to slices or maps and otherwise panics.

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

Ah, this proposal is for reflect package, so should be closed  after https://go.dev/cl/457895 merged.


Bus stops

Blocked by #387
As busways (#308) and guided busways (#413) have been brought up, it seems appropriate to open a separate bus stop issue as well. (In line with this, bus lanes would probably end up being a different topic too, since these each would require different implementations.)
Based on some conversations on Slack, I think it would be helpful to clarify the role of bus features within a street map. Many of the first reference points that come to mind for print maps of bus transit in America are schematic transit maps, which typically show some degree of simplification of a route or network, and may take on an abstract form or show an overlay on a geographically accurate street grid. However, these types of maps often benefit from de-emphasizing other features in some way, and can be considered specialized. It's not the case that bus passengers are a niche audience, but that these maps only serve a particular use case even for bus passengers. Most people who rely on public transport also use some form of generalized street map, because it contains the details they need to make connections to and from specific stops, context for wayfinding, and the shape of the street network. These features get omitted from schematic transit maps, because that including that information density is incompatible with the goal of highlighting the general shape and connections in a network at a glance. To that end, fulfilling the role of a good street map useful for public transit users seems like a fitting role for Americana.
The better examples of transit maps do make some important and interesting design decisions that I still think would be helpful to draw influence from even if it doesn't make sense to replicate their role directly. They are also some of the easier to find reference points for print maps of bus-related features.  Maps which distinguish the frequent network from the coverage network can make a big difference in how usable they are for example. Unfortunately bus transit is historically neglected despite quietly constituting a very substantial portion of local travel in American cities, so there are not as many examples of good street maps for bus passengers, or even good general guide maps featuring bus stops. It is common to find maps which make some really unhelpful or questionable decisions in how they show bus stops and/or routes.
For symbology, I quite like the idea of a silhouette of the MUTCD standard bus icon. Many transit agencies simply use the icon at stops in place of a custom logo, and many custom logos are just minimal variants to it, so it has a broad recognizability. It is also common for signposts for park and rides along roadways to feature this icon, so it seems like it would integrate well with the existing route sign aesthetic. A previous older PR took a similar approach at #17 that got me thinking about this.

The simplest first step would just be to render bus stops where they exist, before getting into more details on what else we can do with them. At the lowest zoom levels, all of them should be visible. The zoom level bus stops appear at should be z12 or lower if I understand the order correctly - when service roads appear, because they very often cluster around service roads at bus bays or drop off zones or even parking lots.
Once they are on the map in some capacity, it would be neat if we were able to collapse bus stops at intermediate zoom levels to make the locations of the significant ones more clear. Showing one stop for each "stop area group" relation, or omitting stops which aren't part of a frequent route as tagged by interval could be some ways to do that.

I think we need to come to a general agreement about whether we prefer shielded POI icons like this, or ones with no background, as appear in osm-carto.  This decision will be the foundation for #387 and open the door to general POI contribution.

Our shield guidelines have backgrounds simplified to a 1px stroke. I don't see why POI icons shouldn't be the same.

Once they are on the map in some capacity, it would be neat if we were able to collapse bus stops at intermediate zoom levels to make the locations of the significant ones more clear.

The Transit tile layer attempts to group stop areas together. I’m not sure how much processing is involved, but it would probably require some postprocessing for the vector tiles, because GL JS doesn’t support clustering features from vector tiles as it does for GeoJSON sources.

To be clear the standard sign I posted is just one variant, there are many signs which don't feature an outline, or which have a circle border, or other small modifications. The green square seems to be common for road-facing park and ride signs in particular. I'll find some other examples to share here for reference.
I'm relatively agnostic as to whether or not the symbol has an outline, and it's possible somewhere down the line we could have some variations in the symbology based on different types of bus related features. Carto actually uses both a symbol without an outline, and one with an outline, with the outline being reserved for bus stations specifically:

You can see it in the top right there at this bus station I've been working on mapping. I think it looks a bit weird to have it shown as one big bus symbol with lots of little bus stop children, so I was thinking stations would be where we might use agency logos.
I don't feel too strongly about it, but I am slightly leaning towards the idea of a 1px outline for bus stops under the logic that they're often representing the position of a sign, and it seems intuitive for them to have a sign-like symbol. I'm not sure if a decision has been made about whether or not it's OK to have a mix of outlined and outline-less symbols, but in my experience there are some icons it makes sense for and others it doesn't. The box with a flag symbol commonly used for schools on paper maps is usually without an outline, but restroom symbols almost always have an outline. (I think that makes sense, especially since a restroom is a "contained" feature there's some logic in making it look contained.)
Are we sticking to 16x16 for POI symbols, or is there a size threshold? That will have some effect on choosing a symbol that looks legible.

Re: stop group areas, I've seen the Transit layer does it, but unfortunately it appears to be closed source. I was planning on looking at the code which reads road route relations for shields as a reference point, as I was thinking a process for displaying a symbol for a stop group could look similar.
I think it would be an interesting form of mapper feedback as well, since they're ostensibly useful but hard to see the results of. As far as I can tell, the Transit layer just shows bubbles around them, which is a form of feedback, but it makes the map more crowded when one of the main advantages of grouping stop features together is to simplify them without losing information about unrelated stops in conjunction.

Rejuvenating this discussion now that some icons are beginning to be rendered (though we still lack many that are probably more important than bus stops). One possibility I've seen that I like is some kind of symbology of a stop, like a sign, or a sign next to a bus. See for inspiration, perhaps, the second and third icons suggested for maki here: https://github.com/mapbox/maki/issues/401

Another enticing possibility could be to render attributes of bus stops, like shelters and benches. These attributes can be useful for route planning and wayfinding, and it would provide nice mapper feedback. In general, rendering attributes as variations on a main theme is a nice way to streamline icons, as nicely outlined by Andy Townsend in this carto comment. One project that does this in a nice way is Röntgen. See for example how that project combines shelter, bench, and bus stop iconology: https://github.com/enzet/map-machine/blob/main/doc/bus_stop.png.

I think having a shielded/MUTCD-style sign for bus stations, similar to how Carto distinguishes them, would be a good approach.


[Pydantic V2] Type checking in validators fails when reserializing models across threads

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
I'm serializing models to disk using msgpack and getting the following contradictory error:
10 validation errors for RoleGroup
tasks.0
  Input should be a valid dictionary or instance of RoleTask [type=model_type, input_value=RoleTask(role_name='Comme...s: 'joined_paragraphs'>), input_type=RoleTask]
    For further information visit https://errors.pydantic.dev/2.4/v/model_type
tasks.1
  Input should be a valid dictionary or instance of RoleTask [type=model_type, input_value=RoleTask(role_name='Comme...s: 'joined_paragraphs'>), input_type=RoleTask]
    For further information visit https://errors.pydantic.dev/2.4/v/model_type

This is the models:
class RoleTask(BaseModel):
    role_name: str
    task_name: str

class Role(BaseModel):
    role_name: str

class RoleGroup(Role):
    tasks: list[RoleTask]

This is how it's serialized and deserialized

def decoder(obj: T | dict, force_type: str | None = None) -> T:
    if "__serialized_type__" in obj:
        obj_type = obj["__serialized_type__"]

        if obj_type == "RoleGroup" or force_type == "RoleGroup":
            obj = RoleGroup(**obj["class_dict"])
        elif obj_type == "RoleTask" or force_type == "RoleTask":
            obj = RoleTask(**obj["class_dict"])
    return obj

def type_check(
    obj: T, class_ref: type[T], class_str: str, force_type: bool = None
) -> bool:
    """
    isinstance and is class are failing across threads when they shouldn't.
    Possibly to do with PYTHONHASHSEED or import hierarchy.

    obj.__class__.__name__ seems to work best
    """
    try:
        is_instance = isinstance(obj, class_ref)
    except TypeError:
        is_instance = False

    return (
        is_instance
        or type(obj) is class_ref
        or (hasattr(obj, "__class__") and obj.__class__ is class_ref)
        or (
            hasattr(obj, "__class__")
            and hasattr(obj.__class__, "__name__")
            and obj.__class__.__name__ == class_str
        )
        or force_type == class_str
    )

def encoder(obj: T, force_type: str | None = None) -> dict | T:
    serialized = None

    if type_check(
        obj, RoleGroup, "RoleGroup", force_type
    ):
        obj: RoleGroup
        attr_dict = obj.__dict__.copy()
        # ERRORS HERE when constructing new_class
        new_class = RoleGroup(**attr_dict)
        serialized = {
            "__serialized_type__": "RoleGroup",
            "class_dict": new_class.__dict__,
        }
    elif type_check(obj, RoleTask, "RoleTask", force_type):
        obj: RoleTask
        attr_dict = obj.__dict__.copy()
        new_class = RoleTask(**attr_dict)
        serialized = {
            "__serialized_type__": "RoleTask",
            "class_dict": new_class.__dict__,
        }

    return serialized if serialized else obj

task = RoleTask(role_name="test", task_name="task")
my_role_group = RoleGroup(role_name="role", tasks=[task])

import msgpack
binary_encoded: bytes = msgpack.packb(my_role_group, default=encoder)

reseriazlied_role_group: RoleGroup = msgpack.unpackb(binary_encoded, object_hook=decoder)

Note it only mostly fails when it is serialized in one thread, and then reserialized in another.
I had a similar problem with isinstance() builtin returning False even though it was showing it as an instance when I printed it's type(), not sure if it's related though, as it's failing in the encoder.
Example Code
No response
Python, Pydantic & OS Version
pydantic version: 2.4.2
        pydantic-core version: 2.10.1
          pydantic-core build: profile=release pgo=true
                 install path: /home/user/.local/lib/python3.11/site-packages/pydantic
               python version: 3.11.5 (main, Sep  2 2023, 14:16:33) [GCC 13.2.1 20230801]
                     platform: Linux-6.4.16-5-MANJARO-x86_64-with-glibc2.38
             related packages: mypy-1.5.1 pydantic-settings-2.0.3 email-validator-2.0.0.post2 fastapi-0.88.0 typing_extensions-4.8.0

More specifically, I think pydantic might be having a bit of trouble with loading data that has been deserialized from some non-JSON format.

To expand on this — we currently have different strictness rules when validating data directly from JSON vs. from data that has been loaded into python. I'm not sure if it's relevant here, but I'm thinking that might be the source of the error you are hitting with the msgpack decoder.
If Sydney's suggestion to use RoleGroup.model_validate_json(json.dumps(obj['class_dict'])) doesn't resolve the issue, it would help a lot if you could produce a self-contained example that we could run to hit the same error, which we could then use to try to better debug what's happening and how to resolve it. Even if it does resolve the issue, having the self-contained example would probably help us to work toward a solution that works for your use case without the extra serialize/deserialize pass.

Yeah I tried to create an isolated example, but it doesn't seem to happen there 🤷
It's got nothing to do with JSON, I'm using the dict from my model to initialize a new model, it's actually in the encoder where it fails before it's even been serialized, but that is running in a different thread from where the original object was constructed - still strange if you look at the error it says it requires a RoleTask but I've provded a RoleTask.
I think it's because I'm writing this app in streamlit, and it has it's own script runner which wraps and execs() imports, I think they're doing something wrong there.
Anyway, I just removed that line and serialized the dict from the original object and it works fine, can't remember why I was reconstructing a new object but V1 required it for some reason.
I will close this as it's a strange issue and not really effecting me anymore.


Issues running first crawl

I seem to be having some trouble getting my first index crawled.  I've installed meilisearch in an azure cloud deployment.  In my local laravel app, I've required this package and meilisearch-php sdk package.
I created the index without issue and updated the table row to include {"meilisearch":{"url":"https:\/\/[SUBDOMAIN INSERTED].azurewebsites.net","apiKey":"[ADMIN KEY INSERTED]"}} where the two variables are populated with the correct information for my configuration.  I have tested the Admin API key successfully in meilisearch on the web interface.
Initially when I ran php artisan site-search:crawl I received an error about casting (an array was expected for array_map but one of the functions was returning an array as contract model).
Doing some quick work to see if I could temporarily solve the problem with a simple cast, the issues just kept cascading down into other lines of code, leading me to believe that my installed versions were not meeting requirements.
I had installed the latest version of meilisearch, but I noticed a suggested version much lower.  So I reverted back to meilisearch-php v. 0.17.2 and the casting issues went away.
However, now the crawl command is still throwing an error, saying that the Authorization header is missing.  I'm at a loss at this point.  I believe I've implemented all as instructed. Do you have any recommendations for me on where things might be going wrong?
UPDATE:
Poking at this a little further and I can see that this is still due to the meilisearch-php compatibility.  v0.17.2 uses the old authorization header which is deprecated in the docker instance I deployed.  If I edit the meilisearch-php vendor files directly to update the header then the error pushes on to another compatibility issue.
So I guess what I need to know is for laravel-site-search v.1.2.0, what version of meilisearch-php is required?  Because the requirements spec'd on the documentation site don't give any information on versioning, but there are clearly some compatibility issues that need to be resolved.  Or perhaps you can update to allow compatibility with the newest meilisearch-php release?

I currently have no time to dedicate on this, but I'd accept a PR that fixes this.

Also have a problem with the first crawl.
array_map(): Argument #2 ($array) must be of type array, MeiliSearch\Contracts\IndexesResults given.
119▕         return array_map(
120▕             fn (Indexes $index) => $index->getUid(),
➜ 121▕             $this->meilisearch->getAllIndexes(),
122▕         );
Any solution for this?

Also have a problem with the first crawl.
array_map(): Argument #2 ($array) must be of type array, MeiliSearch\Contracts\IndexesResults given.
119▕ return array_map( 120▕ fn (Indexes $index) => $index->getUid(), ➜ 121▕ $this->meilisearch->getAllIndexes(), 122▕ );
Any solution for this?

Same issue here!!

I had the same bug, I have not analyzed the problem, but I updated MeiliSearch to version v.0.27.2 with the docker and relaunching the crawl was successful.

To solve this problem just edit the line with $ this->meilisearch->getAllIndexes()->getResults () This should also fix meilisearch-php and meilisearch update to version .28
This solves the error!
Thanxs, mikzero.

Glad you found a solution.
If there's something the needs to change in the package to make it work with the latest version of Meilisearch, send a PR 👍  Closing for now!


Translating text throwing a 500 error.

I'm using the demo code from this repo combined with demo code from the speech to text on the same page, and no matter what I do, even passing in the parameters manually,  the server is giving me a 500 error on POST when it tries to hit the api/translate endpoint.
http://oddestthingintheuniverse.mybluemix.net/
My code setup in app.js is the following:
If anyone could help point me in the right direction it'd be greatly appreciated. I can also link to a full source of the repo if needed.
// For local development, put username and password in config
// or store in your environment
var config = {
  version: 'v1',
  url: 'https://stream.watsonplatform.net/speech-to-text/api',
// removed username and password for code example
  username: username,
  password: password
};

// if bluemix credentials exists, then override local
var speechCredentials = extend(config, bluemix.getServiceCreds('speech_to_text'));
var authorization = watson.authorization(speechCredentials);

// if bluemix credentials exists, then override local
var languageCredentials =  extend({
// removed username and password for code example
  username: username,
  password: password,
  version: 'v2'
}, bluemix.getServiceCreds('language-translation')); // VCAP_SERVICES

var language_translation = watson.language_translation(languageCredentials);

// Setup static public directory
app.use(express.static(path.join(__dirname , './public')));
// render index page
app.get('/', function(req, res) {
  res.render('index');
});

// Get token from Watson for the speech recognition using your credentials
app.get('/token', function(req, res) {
  authorization.getToken({url: speechCredentials.url}, function(err, token) {
    if (err) {
      console.log('error:', err);
      res.status(err.code);
    }

    res.send(token);
  });
});

// Gets the model data for the language translation

app.get('/api/models', function(req, res, next) {
  console.log('/v2/models');
  language_translation.getModels({}, function(err, models) {
    if (err) {
      return next(err);
    }
    else {
      res.json(models);
    }
  });
});

// Post request for the input box to detect language you're typing in.

app.post('/api/identify', function(req, res, next) {
  console.log('/v2/identify');
  var params = {
    text: req.body.textData,
    'X-WDC-PL-OPT-OUT': req.header('X-WDC-PL-OPT-OUT')
  };
  language_translation.identify(params, function(err, models) {
      if (err) {
        return next(err);
      }
      else {
        res.json(models);
      }
  });
});

app.get('/api/identifiable_languages', function(req, res, next) {
  console.log('/v2/identifiable_languages');
  language_translation.getIdentifiableLanguages({}, function(err, models) {
    if (err) {
      return next(err);
    }
    else {
      res.json(models);
    }
  });
});

app.post('/api/translate', function(req, res, next) {
  console.log('/v2/translate');
  var params = extend({ 'X-WDC-PL-OPT-OUT': req.header('X-WDC-PL-OPT-OUT')}, req.body);
  language_translation.translate(params, function(err, models) {
    if (err) {
      return next(err);
    }
    else
      res.json(models);
  });
});

The error I'm getting keeps on being
"Error: Missing required parameters: model_id or source and target
    at LanguageTranslation.translate (/home/vcap/app/node_modules/watson-developer-cloud/services/language_translation/v2.js:65:14)
    at port (/home/vcap/app/app.js:120:24)
    at Layer.handle [as handle_request] (/home/vcap/app/node_modules/express/lib/router/layer.js:82:5)
    at next (/home/vcap/app/node_modules/express/lib/router/route.js:100:13)
    at Route.dispatch (/home/vcap/app/node_modules/express/lib/router/route.js:81:3)
    at Layer.handle [as handle_request] (/home/vcap/app/node_modules/express/lib/router/layer.js:82:5)
    at /home/vcap/app/node_modules/express/lib/router/index.js:233:24
    at Function.proto.process_params (/home/vcap/app/node_modules/express/lib/router/index.js:311:12)
    at /home/vcap/app/node_modules/express/lib/router/index.js:227:12
    at Function.match_layer (/home/vcap/app/node_modules/express/lib/router/index.js:294:3)"

The error you are getting seems to be a missing parameter. Check the call to translate you need to send source,target and text or model_id and text

@germanattanasio Yes, that's what the error said.
However, I tried passing in the parameters manually according to what it said it was missing, and I ended up getting a 502 Error.

can you paste the parameters you are sending?

@germanattanasio Sure I have to wait until I get back into the office tomorrow and I'm on my work machine but I'll get back to you then.

@germanattanasio I actually figured out the issue!
I had forgotten to add in the require('./config/express')(app); in when I was merging the two services app.js files.
Closing issue :)


remove unused sampling code

[ ] Remove unused functions in sampling.R

[x] case_control_func() https://github.com/CAM-Roche/RandomisedTrialsEmulation/blob/c39d96bf0c0f1fe4612b7904b091804f89d797c2/R/sampling.R#L16

[x] case_control_sampling() https://github.com/CAM-Roche/RandomisedTrialsEmulation/blob/c39d96bf0c0f1fe4612b7904b091804f89d797c2/R/sampling.R#L114

[ ] Rename the remaining functions with these shorter names

Started in #78.
Will be resolved with #83


Add support for incoming & outgoing messages

Pull Request Test Coverage Report for Build 969199579

64 of 64 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 960575089:
0.0%

Covered Lines:
1442

Relevant Lines:
1442

💛  - Coveralls

Pull Request Test Coverage Report for Build 979690042
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

76 of 76 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 960575089:
0.0%

Covered Lines:
1453

Relevant Lines:
1453

💛  - Coveralls


Test Slask

you slacker

in progress; see the tests branch

Added tests of most of slask.py in https://github.com/llimllib/slask/commit/933e902e826982696c365913223d566d6445659a .
Still need to test the server-communicatign code and the plugins.

Test the command line options as well

Added in 946ec65343d97f1d3147aea9baa1b5a33562a377
(Still more tests necessary; open issues for specific ones)