[11.x] Fix console prompt docblock

Because of changes from https://github.com/laravel/framework/pull/51055, Illuminate\Console\View\Components\Ask::render now accepts $multiline (boolean) as the third parameter. So I add a missing @param block for it and also update Illuminate\Console\View\Components\Factory methods comments.

Thanks!


Update cTiVo to 2.4.4

Thank you for the contribution. It was merged directly as 722a6be8cb243d4a272807366a865fd685d6fd53 to keep commit history cleaner. Your contribution is still credited to you.


Speed up Position Encodings

Detailed Description
Position encoding creation is quite slow, taking about 4 seconds for a batch of 32. To speed up our training we need to get this down.
Context
A recent bit of profiling shows that the position encoding and all the concatenations in it take up about 4 seconds of the creation of each batch.
Profile stats for: get_train_batch
         199664 function calls (194382 primitive calls) in 11.075 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     23/1    0.000    0.000   11.075   11.075 {built-in method builtins.next}
        1    0.000    0.000   11.075   11.075 supporters.py:658(prefetch_iterator)
        2    0.000    0.000   11.075    5.538 supporters.py:582(__next__)
        3    0.000    0.000   11.075    3.692 apply_func.py:69(apply_to_collection)
        2    0.000    0.000   11.075    5.537 supporters.py:591(request_next_batch)
        2    0.000    0.000   11.075    5.537 supporters.py:603(next_fn)
        2    0.000    0.000   11.075    5.537 dataloader.py:517(__next__)
        2    0.000    0.000   11.074    5.537 dataloader.py:559(_next_data)
        2    0.454    0.227   11.074    5.537 fetch.py:47(fetch)
        2    0.001    0.000   10.619    5.310 datasets.py:250(__getitem__)
        2    0.001    0.001   10.239    5.119 datasets.py:134(__getitem__)
        2    0.061    0.031    8.311    4.156 position_encoding.py:62(generate_position_encodings_for_batch)
       12    1.197    0.100    8.244    0.687 position_encoding.py:193(encode_absolute_position)
      416    0.005    0.000    4.530    0.011 einops.py:327(reduce)
      416    0.006    0.000    4.521    0.011 einops.py:202(apply)
       32    0.000    0.000    4.501    0.141 einops.py:455(repeat)
       32    0.000    0.000    4.494    0.140 _backends.py:98(add_axes)
       32    0.000    0.000    4.493    0.140 _backends.py:336(tile)
       32    4.493    0.140    4.493    0.140 {method 'repeat' of 'torch._C._TensorBase' objects}
      826    2.113    0.003    2.113    0.003 {built-in method cat}
        2    0.001    0.001    1.623    0.812 batch.py:144(load_netcdf)
       10    0.000    0.000    1.437    0.144 position_encoding.py:251(combine_space_and_time_features)
      126    1.390    0.011    1.390    0.011 {method 'acquire' of '_thread.lock' objects}
        2    0.000    0.000    1.380    0.690 _base.py:635(__exit__)
        2    0.000    0.000    1.380    0.690 thread.py:210(shutdown)
       14    0.000    0.000    1.380    0.099 threading.py:1021(join)
       14    0.000    0.000    1.379    0.099 threading.py:1059(_wait_for_tstate_lock)
       12    0.020    0.002    1.207    0.101 position_encoding.py:279(normalize_geospatial_coordinates)
      404    0.192    0.000    0.956    0.002 position_encoding.py:399(fourier_encode)
        6    0.000    0.000    0.241    0.040 datasets.py:340(add_encodings)
       14    0.000    0.000    0.207    0.015 xr_utils.py:84(validate)
        2    0.000    0.000    0.187    0.093 batch.py:157(normalize)
      404    0.187    0.000    0.187    0.000 {method 'sin' of 'torch._C._TensorBase' objects}
      396    0.182    0.000    0.182    0.000 {built-in method stack}
        4    0.000    0.000    0.162    0.041 satellite_model.py:18(model_validation)

Possible Implementation
It seems like reducing the number of concatentations is the key, as generating the Fourier features is quite fast. But concatentation takes 2 of the 4 seconds it takes to create the encodings.
The easiest way would be to just run the spatial and feature encodings over all the batch at once, not one at a time like it currently does.
An issue with that is finding some suitable replacements, for the spatial encoding, for example, torch.meshgrid only takes 1D tensors, and so can't be done over the whole batch at once, I think?
For the datetime features, doing it over the whole batch should also speed it up a bit, just need to extract the hour, minute, and day of year of a set of timestamps all at once.

Well done for spotting this issue and diagnosing the problem!
One random question (which might not be relevant):  Is it possible to do all the encoding stuff in "numpy land"?  (i.e. before Torch gets involved)?  I wonder if numpy might have better-optimised routines for doing this stuff on the CPU?

Oh, quite possibly! Yeah, all the torch functions used have numpy equivalents, I'll try benchmarking those next!

Replacing all the the torch things with Numpy resulted in a huge slowdown on my laptop after running each version of normalize_geospatial_coordinates 100 times:
Torch: 1.322833046913147 sec Numpy: 4.325204303264618 sec

oops, sorry for sending you down that deadend!


Flexible Pricing: Automatically approve if the Learner is elligible when they request it

Pre-Flight checklist

[ ] Screenshots and design review for any changes that affect layout or styling

[ ] Desktop screenshots
[ ] Mobile width screenshots

[ ] Migrations

[ ] Migration is backwards-compatible with current production code

[ ] Testing

[ ] Code is tested
[ ] Changes have been manually tested

[ ] Settings

[ ] New settings are documented and present in app.json
[ ] New settings have reasonable development defaults, if applicable
[ ] Opened issue for DevOps regarding necessary configuration changes to deployed environments

What are the relevant tickets?
fixes #542
What's this PR do?
(Required)
How should this be manually tested?
(Required)
Where should the reviewer start?
(Optional)
Any background context you want to provide?
(Optional)
Screenshots (if appropriate)
(Optional)

Codecov Report

Merging #580 (ce63b1d) into main (bd19e4f) will decrease coverage by 0.97%.
The diff coverage is 38.31%.

@@            Coverage Diff             @@
##             main     #580      +/-   ##
==========================================
- Coverage   85.77%   84.79%   -0.98%
==========================================
  Files         119      119
  Lines        5175     5276     +101
  Branches      838      851      +13
==========================================
+ Hits         4439     4474      +35
- Misses        604      670      +66
  Partials      132      132

Impacted Files
Coverage Δ

cms/models.py
72.92% <ø> (ø)

flexiblepricing/api.py
43.15% <16.92%> (-56.85%)
:arrow_down:

flexiblepricing/serializers.py
64.70% <33.33%> (-35.30%)
:arrow_down:

flexiblepricing/admin.py
95.83% <100.00%> (+0.59%)
:arrow_up:

flexiblepricing/constants.py
100.00% <100.00%> (ø)

flexiblepricing/exceptions.py
100.00% <100.00%> (ø)

flexiblepricing/factories.py
100.00% <100.00%> (ø)

flexiblepricing/models.py
91.80% <100.00%> (+2.44%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd19e4f...ce63b1d. Read the comment docs.


Enable default select element in interactive dialogs

Summary

Resolves the issue with default select elements not selected by default.
Ticket Link

Resolves https://github.com/mattermost/mattermost-server/issues/11053
Screenshots

model.Dialog{
	...
	}, {
		DisplayName: "Option Selector",
		Name:        "someoptionselector",
		Type:        "select",
		Placeholder: "Select an option...",
		HelpText:    "Choose an option from the list.",
		Default:     "opt3",
		Optional:    false,
		Options: []*model.PostActionOptions{{
				Text:  "Option1",
				Value: "opt1",
			}, {
				Text:  "Option2",
				Value: "opt2",
			}, {
				Text:  "Option3",
				Value: "opt3",
			}},
	},

/update-branch

@marianunez I requested the review to soon, please ignore

This issue has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!
/cc @jasonblais @hanzei

@prapti I know this is probably on your queue but a quick reminder to help with QA review after holidays :)

I'm trying to upload the latest demo plugin from master, but as soon as it gets uploaded I get a JS error shown in the screenshot below. Has anyone else seen it?

I'm trying to upload the latest demo plugin from master, but as soon as it gets uploaded I get a JS error shown in the screenshot below. Has anyone else seen it?

@prapti make sure you are on master (5.20) for the webapp. You should not get that error then.

Creating a new SpinWick test server using Mattermost Cloud.

Mattermost test server created! :tada:
Access here: https://mattermost-webapp-pr-4327.test.mattermost.cloud

Account Type
Username
Password

Admin
sysadmin
Sys@dmin123

User
user-1
User-1@123

/update-branch

Will try to auto merge this PR once all tests and checks are passing. This might take up to an hour.

New commit detected. SpinWick will upgrade if the updated docker image is available.

Mattermost test server updated with git commit 4ae7440db67c1959c344a7561459a93e72748839.
Access here: https://mattermost-webapp-pr-4327.test.mattermost.cloud

@jwilander Heads up that the PR didn't automerge

Test server destroyed


Possible bug in phase 2 -> table 7

We were going to the code, and it seems table 7 is also using a buffered disk, but in the end, the last cache is not flushed.
All flushing is done here:
if (table_index != 7) {
            sort_manager->FlushCache();
            sort_timer.PrintElapsed("sort time = ");

            // clear disk caches
            disk.FreeMemory();
            sort_manager->FreeMemory();

            output_files[table_index - 2] = std::move(sort_manager);
            new_table_sizes[table_index] = write_counter;
        }

I checked everywhere if I can see anything that would flush the remaining content of a write buffer, couldn't find anything.

Not a C++ developer but aren't stream flushed on close, closed in the destructor and the destructor is called when the variable runs out of context (unless its a pointer)?

well they wrote their own buffered disk, and keep it in memory, so they are not giving it to any real file stream at that point.


Exec in container with custom user

Hey folks,
first of all many thanks for this great library!
We are about the change the default user used in our docker image on https://github.com/camunda/zeebe from root to a less privileged user to better comply with docker security best practices.
Unfortunately we have some test setup code where we'd like to make use of root privileges still though 🙃 . We thus made use of the docker-java-api directly bypassing the ContainerState.execInContainer API offered by you to run commands, which we used previously.
Thus we figured it would be nice to expose the ExecCreateCmd.withUser API provided by the docker-java-api on testcontainer's ContainerState.
Curious on your feedback on this proposed change!

Thanks for your contribution, @megglos ! This will be part of the next release


fix timing issue in component updates due to consecutive dispatches

This PR fixes #1262 and should hopefully fix the problems described in #1226 as well.

LGTM

Well, that was all kinds of fun.
@MrWolfZ : thanks for digging in to this one!

Whoops, looks like this one went into master instead of v7-hooks-alpha.
I'll revert, figure out how to pull it into the alpha branch, then publish this as 7.1.0-alpha.2.

@markerikson I don't understand. This fix has nothing to do with the hooks alpha. It's a fix for the existing connect API and I expected this to be released as 7.0.3.

Drat.  Y'know what, you're right.  I misread this.  Sorry, my bad.
I'll get this back in and put out 7.0.3 shortly.

Reset master to remove that reversion and force-pushed.  (Wheeee!)

And 7.0.3 is out on NPM.  Sorry about that, and thanks for catching my mistake!

Guys, I hope, you know what u r doing over there.

Thanks! This resolved our issues with 7.0.2


Segmentation fault in ShowCache::handleCacheEvent because cache_vc->first_buf is NULL

gdb debug
# gdb traffic_server core.11704
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7_4.1
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /mmy/app/ats/bin/traffic_server...done.

warning: core file may not match specified executable file.

Core was generated by `/mmy/app/ats/bin/traffic_server -M --httpport 19101:fd=7'.
Program terminated with signal 11, Segmentation fault.
#0  data (this=0x0) at ../../iocore/eventsystem/I_IOBuffer.h:215
215	../../iocore/eventsystem/I_IOBuffer.h: No such file or directory.
Missing separate debuginfos, use: debuginfo-install trafficserver-5.3.2-57.el7.centos.x86_64
(gdb) bt
#0  data (this=0x0) at ../../iocore/eventsystem/I_IOBuffer.h:215
#1  ShowCache::handleCacheEvent (this=0x2afeb0c57000, event=<optimized out>, e=0x2afebb520ae0) at CachePages.cc:332
#2  0x00002afe9c774a0e in handleEvent (data=0x2afebb520ae0, event=<optimized out>, this=<optimized out>)
    at ../../iocore/eventsystem/I_Continuation.h:145
#3  CacheVC::callcont (this=0x2afebb520ae0, event=<optimized out>) at ../../iocore/cache/P_CacheInternal.h:663
#4  0x00002afe9c7d555d in CacheVC::openReadStartEarliest (this=0x2afebb520ae0) at CacheRead.cc:947
#5  0x00002afe9c7b518e in handleEvent (data=0x0, event=3900, this=0x2afebb520ae0)
    at ../../iocore/eventsystem/I_Continuation.h:145
#6  CacheVC::handleReadDone (this=<optimized out>, event=<optimized out>, e=<optimized out>) at Cache.cc:2671
#7  0x00002afe9c7735b3 in handleEvent (data=<optimized out>, event=3900, this=<optimized out>)
    at ../../iocore/eventsystem/I_Continuation.h:145
#8  AIOCallbackInternal::io_complete (this=<optimized out>, event=<optimized out>, data=<optimized out>)
    at ../../iocore/aio/P_AIO.h:117
#9  0x00002afe9c82d360 in handleEvent (data=0x2afeb791aca0, event=1, this=<optimized out>) at I_Continuation.h:145
#10 EThread::process_event (this=this@entry=0x2afea400d2c0, e=e@entry=0x2afeb791aca0, calling_code=1)
    at UnixEThread.cc:128
#11 0x00002afe9c82dc56 in EThread::execute (this=0x2afea400d2c0) at UnixEThread.cc:179
#12 0x00002afe9c82ce3a in spawn_thread_internal (a=0x2afea19247d0) at Thread.cc:85
#13 0x00002afe9ecaedc5 in start_thread () from /lib64/libpthread.so.0
#14 0x00002afe9fd4621d in clone () from /lib64/libc.so.6
(gdb) p *(CacheVC *)0x2afebb520ae0
$4 = {<CacheVConnection> = {<VConnection> = {<Continuation> = {<force_VFPT_to_top> = {
          _vptr.force_VFPT_to_top = 0x2afe9cb00190 <vtable for CacheVC+16>},
        handler = (int (Continuation::*)(Continuation * const, int,
    void *)) 0x2afe9c7d0ae0 <CacheVC::openReadMain(int, Event*)>, mutex = {m_ptr = 0x2afeb41ddc70},
        link = {<SLink<Continuation>> = {next = 0x0}, prev = 0x0}}, lerrno = 0}, <No data fields>},
  static size_to_init = 344, key = {b = {1077575731632374735,<PHONE_NUMBER>7261088732}, u64 = {1077575731632374735,
<PHONE_NUMBER>7261088732}, u32 = {780633039, 250892651,<PHONE_NUMBER>,<PHONE_NUMBER>},
    u8 = "σ\207.kQ\364\016\334\037\312\343\004\341\373", <incomplete sequence \355>}, first_key = {b = {
<PHONE_NUMBER>6047557196,<PHONE_NUMBER>729738088}, u64 = {15767100406047557196,<PHONE_NUMBER>729738088}, u32 = {409060940,
<PHONE_NUMBER>,<PHONE_NUMBER>,<PHONE_NUMBER>}, u8 = "L\306a\030H\376\317\332h?\260a\230\060\026o"}, earliest_key = {b = {
<PHONE_NUMBER>380720586,<PHONE_NUMBER>896848827}, u64 = {3335720549380720586,<PHONE_NUMBER>896848827}, u32 = {2412709834,
      776657962, 181646267,<PHONE_NUMBER>}, u8 = "\312\aϏ*\334J.\273\263\323\n\273o\247F"}, update_key = {b = {0, 0},
    u64 = {0, 0}, u32 = {0, 0, 0, 0}, u8 = '\000' <repeats 15 times>}, dir = {w = {37593, 32484, 9147, 0, 663}},
  earliest_dir = {w = {37593, 32484, 9147, 0, 663}}, overwrite_dir = {w = {0, 0, 0, 0, 0}}, first_dir = {w = {0, 0, 0,
      0, 0}}, _action = {_vptr.Action = 0x2afe9cafae50 <vtable for Action+16>, continuation = 0x2afeb0c57000, mutex = {
      m_ptr = 0x2afeb41ddc70}, cancelled = 0}, request = {<MIMEHdr> = {<HdrHeapSDKHandle> = {m_heap = 0x0},
      m_mime = 0x0}, m_http = 0x0, m_url_cached = {<HdrHeapSDKHandle> = {m_heap = 0x0}, m_url_impl = 0x0},
    m_host_mime = 0x0, m_host_length = 0, m_port = 0, m_target_cached = false, m_target_in_url = false,
    m_port_in_header = false, static USE_HDR_HEAP_MAGIC = 0x1}, vector = {magic = 0x0, data = {data = 0x2afebb520bf0,
      fast_data = {{alternate = {m_alt = 0x2afeb4170000}}, {alternate = {m_alt = 0x0}}, {alternate = {m_alt = 0x0}}, {
          alternate = {m_alt = 0x0}}}, default_val = 0x2afe9d08e730 <default_vec_info>, size = 4, pos = 0}, xcount = 1,
    vector_buf = {m_ptr = 0x0}}, alternate = {m_alt = 0x2afeb4170000}, buf = {m_ptr = 0x2afeb4dc3e30}, first_buf = {
    m_ptr = 0x0}, blocks = {m_ptr = 0x0}, writer_buf = {m_ptr = 0x0}, od = 0x0,
  io = {<AIOCallback> = {<Continuation> = {<force_VFPT_to_top> = {
---Type <return> to continue, or q <return> to quit---
          _vptr.force_VFPT_to_top = 0x2afe9cb000f0 <vtable for AIOCallbackInternal+16>},
        handler = (int (Continuation::*)(Continuation * const, int,
    void *)) 0x2afe9c773580 <AIOCallbackInternal::io_complete(int, void*)>, mutex = {m_ptr = 0x2afeb41ddc70},
        link = {<SLink<Continuation>> = {next = 0x0}, prev = 0x0}}, aiocb = {aio_fildes = 0, aio_buf = 0x2b058c152000,
        aio_nbytes = 1048576, aio_offset =<PHONE_NUMBER>416, aio_reqprio = 0, aio_lio_opcode = 1, aio_state = 0,
        aio__pad = {0}}, action = {_vptr.Action = 0x0, continuation = 0x2afebb520ae0, mutex = {m_ptr = 0x2afeb41ddc70},
        cancelled = 0}, thread = 0x2afea400d2c0, then = 0x0, aio_result = 1048576}, first = 0x0,
    aio_req = 0x2afea95c6a00, sleep_time = 0}, alternate_index = 0, opendir_link = {<SLink<CacheVC>> = {next = 0x0},
    prev = 0x0}, vio = {_cont = 0x0, nbytes = 0, ndone = 0, op = 1, buffer = {mbuf = 0x0, entry = 0x0},
    vc_server = 0x0, mutex = {m_ptr = 0x0}}, initial_thread = 0x2afea400d2c0, frag_type = CACHE_FRAG_TYPE_HTTP,
  info = 0x0, write_vector = 0x2afeb0f10bf0, params = 0x0, header_len = 0, frag_len = 0, write_len = 0, agg_len = 0,
  write_serial = 0, vol = 0x2afea33fc800, last_collision = 0x2b03b225ffb0, trigger = 0x0, read_key = 0x2afebb520b18,
  save_handler = (int (Continuation::*)(Continuation * const, int,
    void *)) 0x2afe9c7d4e00 <CacheVC::openReadStartEarliest(int, Event*)>, pin_in_cache = 0,
  start_time =<PHONE_NUMBER>944897282, base_stat = 13, recursive = 1, closed = 0, seek_to = 0, offset = 0,
  writer_offset = 0, length = 0, doc_pos = 72, write_pos = 0, total_len = 0, doc_len = 275795641, update_len = 0,
  fragment = 0, scan_msec_delay = 0, write_vc = 0x2afeaf7f7560, hostname = 0x0, host_len = 0, header_to_write_len = 0,
  header_to_write = 0x0, writer_lock_retry = 0, {flags = 24576, f = {use_first_key = 0, overwrite = 0,
      close_complete = 0, sync = 0, evacuator = 0, single_fragment = 0, evac_vector = 0, lookup = 0, update = 0,
      remove = 0, remove_aborted_writers = 0, open_read_timeout = 0, data_done = 0, read_from_writer_called = 1,
      not_from_ram_cache = 1, rewrite_resident_alt = 0, readers = 0, doc_from_ram_cache = 0, hit_evacuate = 0,
      allow_empty_doc = 0}}, scan_vol_map = 0x0, scan_fix_buffer_offset = 0}
(gdb)
(gdb) f 4
#4  0x00002afe9c7d555d in CacheVC::openReadStartEarliest (this=0x2afebb520ae0) at CacheRead.cc:947
947	CacheRead.cc: No such file or directory.
(gdb) p first_buf
$6 = {m_ptr = 0x0}
(gdb)

This problem occurs in both 5.3.2 and 7.1.4

traffic.out:
traffic_server: Segmentation fault (Address not mapped to object [0x20])
traffic_server - STACK TRACE:
/mmy/app/ats/bin/traffic_server(crash_logger_invoke(int, siginfo_t*, void*)+0xa7)[0x2acbda4c22f7]
/lib64/libpthread.so.0(+0xf100)[0x2acbdcbcb100]
/mmy/app/ats/bin/traffic_server(ShowCache::handleCacheEvent(int, Event*)+0x5d6)[0x2acbda6e07f6]
/mmy/app/ats/bin/traffic_server(CacheVC::callcont(int)+0x5e)[0x2acbda689a0e]
/mmy/app/ats/bin/traffic_server(CacheVC::openReadStartEarliest(int, Event*)+0x75d)[0x2acbda6ea55d]
/mmy/app/ats/bin/traffic_server(CacheVC::handleReadDone(int, Event*)+0x1ce)[0x2acbda6ca18e]
/mmy/app/ats/bin/traffic_server(AIOCallbackInternal::io_complete(int, void*)+0x33)[0x2acbda6885b3]
/mmy/app/ats/bin/traffic_server(EThread::process_event(Event*, int)+0x130)[0x2acbda742360]
/mmy/app/ats/bin/traffic_server(EThread::execute()+0x616)[0x2acbda742c56]
/mmy/app/ats/bin/traffic_server(+0x34ee3a)[0x2acbda741e3a]
/lib64/libpthread.so.0(+0x7dc5)[0x2acbdcbc3dc5]
/lib64/libc.so.6(clone+0x6d)[0x2acbddc5b21d]

/var/log/messages
Sep 26 15:58:31  traffic_server[11704]: NOTE: --- traffic_server Starting ---
Sep 26 15:58:31  traffic_server[11704]: NOTE: traffic_server Version: Apache Traffic Server - traffi_server - 5.3.2 - (build # 031515 on Mar 15 2018 at 15:23:54)
Sep 26 19:28:31 kernel: [ET_NET 14][12308]: segfault at 20 ip 00002afe9c7cb7f6 sp 00002afea6c06950 error 4 in traffic_server[2afe9c4de000+40b000]
Sep 26 19:31:31  traffic_manager[11447]: {0x7feab81798c0} FATAL: [LocalManager::pollMgmtProcessServer] Error in read (errno: 104)
Sep 26 19:31:31 traffic_manager[11447]: {0x7feab81798c0} ERROR: [LocalManager::sendMgmtMsgToProcesses] Error writing message
Sep 26 19:31:31  traffic_manager[11447]: {0x7feab81798c0} ERROR:  (last system error 32: Broken pipe)
Sep 26 19:31:31  traffic_cop[11441]: (test) read failed [104 'Connection reset by peer']
Sep 26 19:31:31  traffic_cop[11441]: server heartbeat failed [1]
Sep 26 19:31:31  traffic_cop[11441]: cop received child status signal [11447 256]
Sep 26 19:31:31  traffic_cop[11441]: traffic_manager not running, making sure traffic_server is dead
Sep 26 19:31:31  traffic_cop[11441]: spawning traffic_manager
Sep 26 19:31:31  traffic_manager[29549]: NOTE: --- Manager Starting ---
Sep 26 19:31:31  traffic_manager[29549]: NOTE: Manager Version: Apache Traffic Server - traffic_manager - 5.3.2 - (build # 031515 on Mar 15 2018 at 15:24:07)
Sep 26 19:31:31  traffic_manager[29549]: NOTE: RLIMIT_NOFILE(7):cur(65535),max(65535)
Sep 26 19:31:35  traffic_server[29628]: NOTE: --- traffic_server Starting ---
Sep 26 19:31:35 traffic_server[29628]: NOTE: traffic_server Version: Apache Traffic Server - traffi_server - 5.3.2 - (build # 031515 on Mar 15 2018 at 15:23:54)

Interesting. Overall look, I think it is only lookup case while reading from writer.  The first_buf may be unset when someone wants to update the object that you are currently reading.
But still need more information, Can you please set config :
CONFIG proxy.config.diags.debug.tags string cache*

For upper state machines, case CACHE_EVENT_OPEN_READ, doesn't mean first_buf has header Doc.
In read form write process, the cache_vc->alternate save head info, the cache_vc->first_buf is NULL.
I think ShowCache::lookup_url should call Cache::lookup function, In case,  cache_vc callback upstream ECACHE_DOC_BUSY.

recurrent question：
create big file 1.html, cache 1.html, modify  os file 1.html,  update cache 1.html (os response 200).
In the process, using http_ui lookup_url 1.html (ensure open_read while in update write)

recurrent question：
create big file 1.html, cache 1.html, modify os file 1.html, update cache 1.html (os response 200).
In the process, using http_ui lookup_url 1.html (ensure open_read while in update write)

Thank you for solving our doubts. We really use the http_ui lookup_url in our script.


Checkboxes in property pages do not display properly in v0.17.2

Checkbox text starts behind the checkbox (spacing between checkbox and text is wrong) in v0.17.2. I have tried both Chrome and Safari on a Mac (latest version of macOS - v10.12.5) and the issue is the same in both. See screen shots attached

Look fine here:

Could you use the Chrome inspector to see if there's any css being applied to them from somewhere else? I wonder if a node you have installed is setting some generic CSS style to checkboxes.

Hey Nick, I have uninstalled all nodes that are not part of base and restarted Node-RED. I have checked with inspector and can't see anything odd.
I did not uninstall previous Node-RED, just did an NPM install over the top. I will remove the entire Node-RED install and start again to check
Chris

OK, removing my previous install of 0.16.x and re-installing v0.17.2 has fixed it. I will go back and install the nodes (including my own) that I had installed and see if I can recreate.

Can't recreate it. All working fine now

Great 👍


[release-4.16] backport IBGU

/hold

/cc @fontivan @rauhersu can we use golang 1.22 for talm 4.16?

/cc @fontivan @rauhersu can we use golang 1.22 for talm 4.16?

@sudomakeinstall2 I believe we can but we will have to align this with the midstream build. I will discuss with @rauhersu

/cc @fontivan @rauhersu can we use golang 1.22 for talm 4.16?

@sudomakeinstall2 I believe we can but we will have to align this with the midstream build. I will discuss with @rauhersu

@sudomakeinstall2 Midstream should be ready for this now

/override ci/prow/integration

/unhold


Adding advanced usage docs

What do these changes do?

Related issue number

[x] passes git diff upstream/master -u -- "*.py" | flake8 --diff
[x] passes black --check modin/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-Performance-Testing/44/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-PRB/55/
Test PASSed.

Merged build finished. Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-Performance-PRB/2/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-PRB/65/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-Performance-PRB/3/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-PRB/66/
Test PASSed.


MOBILE ONLY - Uncaught TypeError: Cannot read property 'query' of null

I am having issue with the search function only on mobile, "Uncaught TypeError: Cannot read property 'query' of null".
Note: this works in desktop :(

I already have this code :(


[BUG] node logs in support bundle is empty

Describe the bug
The node logs of a support bundle are empty.
➜  pwd
/Users/kiefer/Downloads/supportbundle_427b5109-18dc-4364-85b8-5fa19079367c_2022-01-04T07-22-00Z/nodes
➜  ls -alh .
total 24
drwxr-xr-x@ 5 kiefer  staff   160B Jan  4 15:40 .
drwxr-xr-x@ 8 kiefer  staff   256B Jan  4 15:38 ..
-rw-r--r--@ 1 kiefer  staff   162B Jan  4 15:22 node1.zip
-rw-r--r--@ 1 kiefer  staff   162B Jan  4 15:22 node2.zip
-rw-r--r--@ 1 kiefer  staff   162B Jan  4 15:22 node3.zip
➜ unzip node1.zip
Archive:  node1.zip
   creating: node1/
➜ ls -alh node1
total 0
drwxr-xr-x@ 2 kiefer  staff    64B Jan  4 15:22 .
drwxr-xr-x@ 6 kiefer  staff   192B Jan  4 15:40 ..

To Reproduce
Steps to reproduce the behavior:

Create a support bundle from GUI

Expected behavior

Node logs should be collected.
Support bundle

Environment:

Harvester ISO version: v1.0.0
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630):

Additional context
Add any other context about the problem here.

Note:
in bundleGenerationError.log
Failed to get /apis/authorization.k8s.io/v1/selfsubjectrulesreviews: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/subjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/selfsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/default/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/default/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/kube-system/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/kube-system/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/cattle-system/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/cattle-system/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/cattle-dashboards/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/cattle-dashboards/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/cattle-fleet-clusters-system/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/cattle-fleet-clusters-system/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/cattle-monitoring-system/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/cattle-monitoring-system/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/fleet-local/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/fleet-local/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/harvester-system/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/harvester-system/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/local/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/local/localsubjectaccessreviews: the server does not allow this method on the requested resource
Failed to get /api/v1/namespaces/longhorn-system/bindings: the server does not allow this method on the requested resource
Failed to get /apis/authorization.k8s.io/v1/namespaces/longhorn-system/localsubjectaccessreviews: the server does not allow this method on the requested resource
BUG: Support bundle: cannot get log for pod rancher-monitoring-grafana-7f54b7d8bc-pnsjl container grafana-sc-dashboard: container "grafana-sc-dashboard" in pod "rancher-monitoring-grafana-7f54b7d8bc-pnsjl" is waiting to start: PodInitializing
BUG: Support bundle: cannot get log for pod rancher-monitoring-grafana-7f54b7d8bc-pnsjl container grafana: container "grafana" in pod "rancher-monitoring-grafana-7f54b7d8bc-pnsjl" is waiting to start: PodInitializing
BUG: Support bundle: cannot get log for pod rancher-monitoring-grafana-7f54b7d8bc-pnsjl container grafana-proxy: container "grafana-proxy" in pod "rancher-monitoring-grafana-7f54b7d8bc-pnsjl" is waiting to start: PodInitializing

node1:~ # kubectl get pods -A
NAMESPACE                   NAME                                                     READY   STATUS      RESTARTS   AGE
cattle-fleet-local-system   fleet-agent-78f694664b-bscbz                             1/1     Running     0          3h42m
cattle-fleet-system         fleet-controller-6666887949-vvnlc                        1/1     Running     0          3h42m
cattle-fleet-system         gitjob-7b97c9c7fd-mt9t2                                  1/1     Running     0          3h42m
cattle-monitoring-system    prometheus-rancher-monitoring-prometheus-0               3/3     Running     1          3h39m
cattle-monitoring-system    rancher-monitoring-grafana-7f54b7d8bc-pnsjl              0/3     Init:0/2    0          3h41m
cattle-monitoring-system    rancher-monitoring-kube-state-metrics-744b9448f4-zlvpb   1/1     Running     0          3h41m
cattle-monitoring-system    rancher-monitoring-operator-754bcd8cb4-p7dc7             1/1     Running     0          3h41m
cattle-monitoring-system    rancher-monitoring-prometheus-adapter-77568b975-wtf6k    1/1     Running     0          3h41m
cattle-monitoring-system    rancher-monitoring-prometheus-node-exporter-cdjcm        1/1     Running     0          3h41m

env:
harvester 1.0.0 in libvirt VM

Agent logs
node1:~ # kubectl logs -f -n harvester-system supportbundle-agent-bundle-xrffz-whmjn
+ exec tini -- /usr/bin/support-bundle-collector.sh
+ HOST_PATH=/host
+ OUTPUT_DIR=/tmp/support-bundle
+ '[' -z http://<IP_ADDRESS>:8080 ']'
+ '[' '!' -e /tmp/support-bundle ']'
+ mkdir -p /tmp/support-bundle
+ NODE_NAME=node2
+ BUNDLE_DIR=/tmp/support-bundle/node2
+ mkdir -p /tmp/support-bundle/node2
++ bash -c 'source /host/etc/os-release && echo $ID'
+ OS_ID=harvester
+ '[' -z harvester ']'
+ OS_COLLECTOR=collector-harvester
++ which collector-harvester
+ '[' -x '' ']'
+ echo 'No OS collector found'
No OS collector found
+ cd /tmp/support-bundle
++ basename /tmp/support-bundle/node2
+ zip -r node_bundle.zip node2
  adding: node2/ (stored 0%)
+ rm -rf bundle
+ set -o errexit
+ curl -v -i -H 'Content-Type: application/zip' --data-binary @node_bundle.zip http://<IP_ADDRESS>:8080/nodes/node2
*   Trying <IP_ADDRESS>:8080...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8080 (#0)
> POST /nodes/node2 HTTP/1.1
> Host: <IP_ADDRESS>:8080
> User-Agent: curl/7.80.0
> Accept: */*
> Content-Type: application/zip
> Content-Length: 162
>
} [162 bytes data]
* Mark bundle as not supporting multiuse
HTTP/1.1 201 Created
< HTTP/1.1 201 Created
< Date: Thu, 06 Jan 2022 04:20:34 GMT
< Content-Length: 0
<
100   162    0     0  100   162      0   315k --:--:-- --:--:-- --:--:--  158k
* Connection #0 to host <IP_ADDRESS> left intact
Date: Thu, 06 Jan 2022 04:20:34 GMT
Content-Length: 0

+ sleep infinity
rpc error: code = NotFound desc = an error occurred when try to find container "274342171114b5ee0e5c5b02f4f6efafbf6956b6c12dc767ca939f63324c9faf": not foundnode1:~ #

Root cause:
harvester v1.0.0 will put ID=harvester in /etc/os-release
but rancher/support-bundle-kit didn't have collector-harvester, only has collector-harvesteros
need to rename or duplicate collector

Please help rename it, we want to avoid the harvesteros term. Thanks.

Fix PR in https://github.com/rancher/support-bundle-kit/pull/30

@bk201, @tjjh89017 node logs also include some sensitive file, should we exclude it?

@bk201, @tjjh89017 node logs also include some sensitive file, should we exclude it?

It's a todo item https://github.com/rancher/support-bundle-kit/issues/28

Verified the bug has been fixed.

Test Information

Environment: qemu/KVM 3 nodes
Harvester Version: master-ebde990d-head
ui-source Option: Auto

Verify Steps:

Install Harvester with any nodes
Login to Dashboard then navigate to Support page
Click Generate Support Bundle
Generated zip file of support bundle should contain nodes/<node_name>.zip/logs/*.log


Reconnect SSH after disconnect

This PR implements feature request #64 .
When in _poll_slurm, we now do two things:

When get_status is called, any conn.run() commands are run through a new method run_remote_command_with_reconnect.

A new method get_ssh_conn_status is run in async with get_status. The new method polls the ssh connection with a test command and reconnects on the test command's failure.

I have checked this rather manually on my laptop (2021 Mac M1 Pro, running  MacOS Ventura 13.3.1 [22E261] ) by (i) running a trivial sample workflow which is disptached to Perlmutter and (ii) turning off my wifi connection once in poll slurm and (iii) Turning it back on again once the debug level logger message:
app_log.debug( f"SSH connection error {e}, reconnecting, attempt {self.ssh_connect_retries}...")

appears. I can confirm that the ssh connection becomes reestablished and the workflow runs to completion without error.
Some things to note are

This code only checks for connection drops in _poll_slurm while connection can drop and cause hangs at any call to conn.run(). If we wanted to be super tight, we should use the new method run_remote_command_with_reconnect in place of any conn.run() command

Does this code also check out with the plain ssh executor? Let's discuss.

[ ] I have added the tests to cover my changes.
[ ] I have updated the documentation, VERSION, and CHANGELOG accordingly.
[x] I have read the CONTRIBUTING document.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


IRC client no longer joining the IRC channel

With the latest SNAPSHOT (8), the plugin no longer joins the IRC channel

Seems like the client is not joining the channel but is still sending messages to the channel... probably an issue with the join statement.
To be investigated

Fixed in commit 6f9c64f5768a5dbc41d95109933faa0ab8ecb6ef


Fix pagination on new public pages

Fixes #2623
The problem is not really on the public pages but in how the pagination view was changed to work for the create-map dialog, so refactored the code to allow the click-behaviour to work as default. Instead, only block stop event it if a url resolver (url_to) is not provided (which is the desired behavior for the create map dialog).

@xavijam tested locally and works as expected now, but code review
cc @saleiva for reporting the issue

:+1:

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)


Feature request: Start Evaluating Pydantic v2

Use case
Pydantic is already included in aws-lamdba-powertools as the provider for deep data validation and parsing as an optional dependency. Pydantic V2 is now in beta evaluation, when it becomes a stable version, it should be introduced as an option into powertools without major breaking changes. We need a way to test the integration with the rest of the library and benchmark performance within Lambda.
Solution/User Experience
Today the parser is included as an extra using Pydantic. With the launch of PydanticV2 we could include a second extra, add max version, or identify if we can do this transparently to the user with an optional switch.
To keep powertools lean, having both dependencies on a single extra seems less than ideal.
What is the best way to evaluate an impact like this without falling out of sync with the enhancements to powertools?
Alternative solutions
- Have two extras tagged to Pydantic versions
- Freeze the parser to Pydantic v1
- Identify if there are any breaking implementations within parser for breaking API changes of the Pydantic upgrade (not personally recommended)
- Identify how to handle the version difference in the full install layer as users will choose one version, not both in their function.

Acknowledgment

[X] This feature request meets Powertools for AWS Lambda (Python) Tenets
[ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

### Tasks

Pydantic does not provide a Java or .NET library, nor am I aware of one for TypeScript.

Hey Daniel, good idea!! Have they settled on major breaking changes for
Pydantic V2? If not, any major ones that stands out already?
We knew v2 was in the works back then so we kept our usage to the bare
minimum hoping the transition would be seamless. I suspect the only one
surprise might be the coercion mode they changed now (strict vs coerce).
Happy with a new optional dep for the time being until we figure this out.
Layer will be tricky, it’d require a major version unless the transition is
painless — we’d need plenty of guidance like our upgrade guide either way.
Thanks for kicking the tires ;)
On Thu, 8 Jun 2023 at 20:59, Daniel Furman @.***> wrote:

Pydantic does not provide a Java or .NET library, nor am I aware of one
for TypeScript.
—
Reply to this email directly, view it on GitHub
https://github.com/awslabs/aws-lambda-powertools-python/issues/2427#issuecomment-1583173294,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZPQBFUP75VIHVMDNFPXCLXKIOIJANCNFSM6AAAAAAY7V5Q2M
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
com>

Pydantic 2 has now been released

Hey everyone! We will be creating an RFC to discuss migrating from Pydantic v1 to v2 by the end of this month. In addition to the breaking changes this migration will cause, we are trying to understand if there is a way to keep both versions of Pydantic in Powertools and can let customers choose which one they want to use.
Once we have the RFC, I will update this issue.

@leandrodamascena I use both powertools and pydantic in my lambda. Is there some way for me to use pydantic 2 while also letting powertools continue to use v1? If I try to update it, it complains that powertools is pinning the older version

@leandrodamascena I use both powertools and pydantic in my lambda. Is there some way for me to use pydantic 2 while also letting powertools continue to use v1? If I try to update it, it complains that powertools is pinning the older version

I don't think you can @half2me! Because we still don't support Pydantic v2 and if you try to use Powertools with this version you will get errors. We are prioritizing efforts to support Pydantic v2 as soon as possible.

Hey everyone! I started working with Pydantic v2 and Powertools and I could use SqsModel with Pydantic v2. I need to dig deeper and investigate this migration further, especially as we want to support v1 and v2 at the same time to avoid breaking changes.
Things I've discovered so far:
1 - All optional fields must have a default value of 'None' otherwise Pydantic will fail.
Pydantic v1
class SqsAttributesModel(BaseModel):
    ApproximateReceiveCount: str
    ApproximateFirstReceiveTimestamp: datetime
    MessageDeduplicationId: Optional[str]
    MessageGroupId: Optional[str]
    SenderId: str
    SentTimestamp: datetime
    SequenceNumber: Optional[str]
    AWSTraceHeader: Optional[str]

Pydantic v2
class SqsAttributesModel(BaseModel):
    ApproximateReceiveCount: str
    ApproximateFirstReceiveTimestamp: datetime
    MessageDeduplicationId: Optional[str] = None
    MessageGroupId: Optional[str] = None
    SenderId: str
    SentTimestamp: datetime
    SequenceNumber: Optional[str] = None
    AWSTraceHeader: Optional[str] = None

2 - The @root_validator decorator is deprecated and will be removed in v3. Even though Pydantic suggests using the new @model_validator decorator, we can still use the @root_validator decorator to avoid breaking changes and plan this removal for Powertools v3.
Pydantic v1
@root_validator(allow_reuse=True)
def check_message_id(cls, values):
    message_id, event_type = values.get("messageId"), values.get("eventType")
    if message_id is not None and event_type != "MESSAGE":
        raise TypeError("messageId is available only when the `eventType` is `MESSAGE`")
    return values

Pydantic v2
@root_validator(allow_reuse=True, skip_on_failure=True)
def check_message_id(cls, values):
    message_id, event_type = values.get("messageId"), values.get("eventType")
    if message_id is not None and event_type != "MESSAGE":
        raise TypeError("messageId is available only when the `eventType` is `MESSAGE`")
    return values

3 - Pydantic package size dropped from ~9MB to ~5.5MB
4 - Let's take this opportunity to check if we can leverage this new version and make some changes to improve the performance in the use of Pydantic.

Exampled used:
from aws_lambda_powertools.utilities.parser import event_parser, BaseModel, envelopes
from aws_lambda_powertools.utilities.parser.models import (
    SqsModel,
)

from aws_lambda_powertools import Logger
import pydantic

logger = Logger()

@event_parser(model=SqsModel)
def lambda_handler(event: SqsModel, context):
    logger.info(f"Pydantic version -> {pydantic.__version__}")
    for record in event.Records:
        logger.info(f"Event body -> {record.body}")

Output
❯ sam local invoke --event events/sqs.json
Invoking app.lambda_handler (python3.10)
Local image is up-to-date
Using local image: public.ecr.aws/lambda/python:3.10-rapid-x86_64.

Mounting /home/leandro/DEVEL-PYTHON/tmp/pydantic2/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated, inside runtime container
START RequestId: 8965caed-f0f8-4122-aff0-2d2c69acc631 Version: $LATEST
START RequestId: 8965caed-f0f8-4122-aff0-2d2c69acc631 Version: $LATEST
{"level":"INFO","location":"lambda_handler:16","message":"Pydantic version -> 2.0","timestamp":"2023-07-04 10:31:21,050+0000","service":"service_undefined"}
{"level":"INFO","location":"lambda_handler:18","message":"Event body -> {\"message\": \"foo1\"}","timestamp":"2023-07-04 10:31:21,051+0000","service":"service_undefined"}
{"level":"INFO","location":"lambda_handler:18","message":"Event body -> {\"message\": \"foo1\"}","timestamp":"2023-07-04 10:31:21,051+0000","service":"service_undefined"}
END RequestId: 8965caed-f0f8-4122-aff0-2d2c69acc631
REPORT RequestId: 8965caed-f0f8-4122-aff0-2d2c69acc631	Init Duration: 0.08 ms	Duration: 362.40 ms	Billed Duration: 363 ms	Memory Size: 128 MB	Max Memory Used: 128 MB

Thank you.

I forgot to mention that the refactored model code works in Pydantic v1 and v2. It's still too early to come to any conclusions and so I'm going to write the RFC. But that's what we intend to allow a seamless experience for customers who choose to use Pydantic v1 or v2.
❯ sam local invoke --event events/sqs.json
Invoking app.lambda_handler (python3.10)
Local image is up-to-date
Using local image: public.ecr.aws/lambda/python:3.10-rapid-x86_64.

Mounting /home/leandro/DEVEL-PYTHON/tmp/pydantic2/.aws-sam/build/HelloWorldFunction as /var/task:ro,delegated, inside runtime container
START RequestId: 7b6d3090-e61f-4a36-8478-d17a3b16ed4e Version: $LATEST
START RequestId: 7b6d3090-e61f-4a36-8478-d17a3b16ed4e Version: $LATEST
{"level":"INFO","location":"lambda_handler:13","message":"Pydantic version -> 1.10.10","timestamp":"2023-07-04 10:39:45,260+0000","service":"service_undefined"}
{"level":"INFO","location":"lambda_handler:15","message":"Event body -> {\"message\": \"foo1\"}","timestamp":"2023-07-04 10:39:45,261+0000","service":"service_undefined"}
{"level":"INFO","location":"lambda_handler:15","message":"Event body -> {\"message\": \"foo1\"}","timestamp":"2023-07-04 10:39:45,261+0000","service":"service_undefined"}
END RequestId: 7b6d3090-e61f-4a36-8478-d17a3b16ed4e
REPORT RequestId: 7b6d3090-e61f-4a36-8478-d17a3b16ed4e	Init Duration: 0.14 ms	Duration: 211.51 ms	Billed Duration: 212 ms	Memory Size: 128 MB	Max Memory Used: 128 MB

Thanks a lot! I'm happy to test this it once its available and provide feedback

Please keep in mind, that some old feature still work but are deprecated.
e.g. root_validator

Pydantic V1 style @root_validator validators are deprecated. You should migrate to Pydantic V2 style @model_validator validators, see the migration guide for more details

Please keep in mind, that some old feature still work but are deprecated. e.g. root_validator

Thanks for bringing this up @Wurstnase. I am considering this note in the RFC.

Initial RFC is out by @leandrodamascena: https://github.com/aws-powertools/powertools-lambda-python/issues/2672

Thanks @leandrodamascena  for this awesome summary!
I've noticed some extra issues in the pydantic migration guide that require our attention:

the parse_raw/ parse_obj funcs are now deprecated and need to renamed
Types: Empty Dicts/List now fail validation, they didnt before so this might alter people's validations
.dict() and .json() are also removed, we use in several use cases, one of them is also idempotency and even the parser docs.
@validators However, in Pydantic V2, when a TypeError is raised in a validator, it is no longer converted into a ValidationError, so that means people need to change their code to catch TypeError (breaking change) OR we change the validator to raise ValidationError instead of TypeError.
#4 also applied to code such as json.loads that raises TypeError, it's best to catch all exceptions and perhas re-raise them as validationerrors

@ran-isenberg do you mind moving this conversation to the RFC issue? So we can address all comments and update the RFC.
RFC: https://github.com/aws-powertools/powertools-lambda-python/issues/2672

Lê - could you add a table to the RFC at the bottom of what’s deprecated vs
removed? And behaviour changes affecting us like validation error Ran
brought up?
It’d give us more confidence that we’re not missing anything, and don’t
overengineer for something that isn’t removed in V2 (yet).
On Wed, 5 Jul 2023 at 19:39, Ran Isenberg @.***> wrote:

Thanks @leandrodamascena https://github.com/leandrodamascena for this
awesome summary!
I've noticed some extra issues in the pydantic migration guide that
require our attention:

the parse_raw/ parse_obj funcs are now deprecated and need to
renamed
Types: Empty Dicts/List now fail validation, they didnt before so
this might alter people's validations
.dict() and .json() are also removed, we use in several use cases,
one of them is also idempotency and even the parser docs.
@Validators https://github.com/Validators However, in Pydantic
V2, when a TypeError is raised in a validator, it is no longer converted
into a ValidationError, so that means people need to change their code to
catch TypeError (breaking change) OR we change the validator to raise
ValidationError instead of TypeError.
#4
https://github.com/aws-powertools/powertools-lambda-python/issues/4
also applied to code such as json.loads that raises TypeError, it's best to
catch all exceptions and perhas re-raise them as validationerrors

—
Reply to this email directly, view it on GitHub
https://github.com/aws-powertools/powertools-lambda-python/issues/2427#issuecomment-1622201400,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZPQBF63U2YLU5XHERX5N3XOWRERANCNFSM6AAAAAAY7V5Q2M
.
You are receiving this because you commented.Message ID:
@.***
com>

@heitorlessa yes, great idea Heitor! :rocket:


Errors in running GloveExample in DL4J-examples

I didn't change anything and just run the default example, but received this error message. What might be the problem?
1.1.7.jar" org.deeplearning4j.examples.nlp.glove.GloVeExample
o.n.l.f.Nd4jBackend - Loaded [CpuBackend] backend
o.n.n.NativeOpsHolder - Number of threads used for NativeOps: 2
o.n.n.Nd4jBlas - Number of threads used for BLAS: 2
o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Windows 10]
o.n.l.a.o.e.DefaultOpExecutioner - Cores: [4]; Memory: [3.5GB];
o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
o.d.m.s.SequenceVectors - Starting vocabulary building...
o.d.m.w.w.VocabConstructor - Sequences checked: [71], Current vocabulary size: [28]; Sequences/sec: [33.05];
o.d.m.e.l.WordVectorSerializer - Projected memory use for model: [0.02 MB]
o.d.m.e.i.InMemoryLookupTable - Initializing syn1...
o.d.m.s.SequenceVectors - Building learning algorithms:
o.d.m.s.SequenceVectors -           building ElementsLearningAlgorithm: [GloVe]
o.d.m.e.l.i.e.GloVe - GloVe params: {Max Memory: [0], Learning rate: [0.1], Alpha: [0.75], xMax: [100.0], Symmetric: [true], Shuffle: [true]}
o.d.m.g.AbstractCoOccurrences - Actual memory limit: [3780640768]
o.d.m.g.AbstractCoOccurrences - Number of word pairs saved so far: [454]
o.d.m.g.AbstractCoOccurrences - CoOccurrences map was built.
o.d.m.s.SequenceVectors - Starting learning process...
Exception in thread "GloVe ELA t.2" java.lang.NullPointerException
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:160)
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:147)
	at org.deeplearning4j.models.embeddings.learning.impl.elements.GloVe$GloveCalculationsThread.run(GloVe.java:262)
o.d.m.e.l.i.e.GloVe - Processed [421] pairs, Error was [8.77615388324181]
Exception in thread "GloVe ELA t.2" java.lang.NullPointerException
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:160)
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:147)
	at org.deeplearning4j.models.embeddings.learning.impl.elements.GloVe$GloveCalculationsThread.run(GloVe.java:262)
o.d.m.e.l.i.e.GloVe - Processed [736] pairs, Error was [6.50891944035564]
o.d.m.e.l.i.e.GloVe - Processed [1190] pairs, Error was [8.694534749871718]
o.d.m.e.l.i.e.GloVe - Processed [1644] pairs, Error was [8.390842903821625]
o.d.m.e.l.i.e.GloVe - Processed [2098] pairs, Error was [8.126683610228756]
o.d.m.e.l.i.e.GloVe - Processed [2552] pairs, Error was [7.8741337047486075]
o.d.m.e.l.i.e.GloVe - Processed [3006] pairs, Error was [7.638759851907549]
o.d.m.e.l.i.e.GloVe - Processed [3460] pairs, Error was [7.421269145066776]
o.d.m.e.l.i.e.GloVe - Processed [3914] pairs, Error was [7.219817799753098]
Exception in thread "GloVe ELA t.0" java.lang.NullPointerException
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:160)
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:147)
	at org.deeplearning4j.models.embeddings.learning.impl.elements.GloVe$GloveCalculationsThread.run(GloVe.java:262)
o.d.m.e.l.i.e.GloVe - Processed [4358] pairs, Error was [6.340276097827357]
o.d.m.e.l.i.e.GloVe - Processed [4812] pairs, Error was [6.880693995054875]
o.d.m.e.l.i.e.GloVe - Processed [5266] pairs, Error was [6.717555712048161]
o.d.m.e.l.i.e.GloVe - Processed [5720] pairs, Error was [6.566312666567867]
Exception in thread "GloVe ELA t.0" java.lang.NullPointerException
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:160)
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:147)
	at org.deeplearning4j.models.embeddings.learning.impl.elements.GloVe$GloveCalculationsThread.run(GloVe.java:262)
o.d.m.e.l.i.e.GloVe - Processed [6126] pairs, Error was [5.739417753865672]
o.d.m.e.l.i.e.GloVe - Processed [6580] pairs, Error was [6.308963946293283]
o.d.m.e.l.i.e.GloVe - Processed [7034] pairs, Error was [6.187631401991596]
Exception in thread "GloVe ELA t.2" java.lang.NullPointerException
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:160)
	at org.deeplearning4j.models.glove.AbstractCoOccurrences$1.next(AbstractCoOccurrences.java:147)
	at org.deeplearning4j.models.embeddings.learning.impl.elements.GloVe$GloveCalculationsThread.run(GloVe.java:262)
o.d.m.e.l.i.e.GloVe - Processed [7381] pairs, Error was [4.856713125768342]
Exception in thread "GloVe ELA t.0" java.lang.NullPointerException
o.d.m.e.l.i.e.GloVe - Processed [7763] pairs, Error was [5.236226830378817]
o.d.m.e.l.i.e.GloVe - Processed [8217] pairs, Error was [5.898072603096695]
Exception in thread "GloVe ELA t.3" java.lang.NullPointerException
o.d.m.e.l.i.e.GloVe - Processed [8344] pairs, Error was [1.6073951146509666]
Exception in thread "GloVe ELA t.0" java.lang.NullPointerException
o.d.m.e.l.i.e.GloVe - Processed [8744] pairs, Error was [5.038342630863234]
o.d.m.e.l.i.e.GloVe - Processed [9198] pairs, Error was [5.711398310246838]
o.d.m.e.l.i.e.GloVe - Processed [9652] pairs, Error was [5.630855539552693]
Exception in thread "GloVe ELA t.1" java.lang.NullPointerException
o.d.m.e.l.i.e.GloVe - Processed [10039] pairs, Error was [4.710978363096721]
Exception in thread "GloVe ELA t.3" java.lang.NullPointerException
o.d.m.e.l.i.e.GloVe - Processed [10421] pairs, Error was [4.606325618589391]
o.d.m.e.l.i.e.GloVe - GloVe finalizer...
o.d.m.e.l.i.e.GloVe - GloVe finalizer...
o.d.m.e.l.i.e.GloVe - GloVe finalizer...
o.d.m.e.l.i.e.GloVe - GloVe finalizer...
o.d.m.s.SequenceVectors - Time spent on training: 22367 ms
o.d.e.n.g.GloVeExample - Day/night similarity: NaN
o.d.e.n.g.GloVeExample - Nearest words to 'day': []

Process finished with exit code 0

I've found that reducing the learning rate to about 0.01 can get rid of the NaNs but not the NullPointerExceptions.

Closing due to GloVe feature removal. See https://github.com/eclipse/deeplearning4j/issues/7855#issuecomment-624616313


Breakpoints are not detected

I have a symfony project, in console I get:

When I do the call where it should hit the breakpoint, at first it asks me for the path and I've set it correctly, but the breakpoint never gets triggered, i keep getting session initialized and session terminated, in VSCode and PHPStorm  the breakpoints work fine

Can you enable protocol debugging and then provide a copy of the log messages from the Atom Debugger?

I have had the same problem.
Disabling "Allow for multiple debug sessions at once" helped.

I also use "PHP Server" extension to open project, with port any, but to 80, because in 80 port it doesn't open it browser...
And while using the real Apache (some of it's russian bundle - https://ospanel.io/) - i have no debugging at all, while apache open http in 80 port, as recommended for debugging...

The Great Creator of this extension - why doest it work so strange??

I just figured out there are indeed multiple breakpoint systems apparently available in parallel.
Alt-F9 sets the "right" breakpoint in gutter right hand side, while I guess atom-debug-ui has reserved gutter left hand side.
Breakpoints set w/ Alt-F9 work correctly.

In the meanwhile, I migrated to IntelliJ PhpStorm, never looked back.

Its Jan 2022, and this issue still exists. I have the gutters on the right and enabled breakpoints using Alt-F9; removed the other debugger but still the php-debugger won't stop at the expected breakpoint.


random seed reproducibility of treesim.birth_death_tree when using numpy.save

Hi, I've discovered an obscure issue.
birth_death_tree results are changing if numpy.save was used in the code.
You need two mostly identical files 1.py and 2.py:
# 1.py
import random

import numpy as np

import dendropy
from dendropy.simulate import treesim

np.save('file', 'aaa')

rng=random.Random(1)

t = treesim.birth_death_tree(
    birth_rate=10.,
    death_rate=5.,
    birth_rate_sd=0.5,
    death_rate_sd=0.5,
    num_extant_tips=10,
    rng=rng)
print(t)
# this last line only needed in 2.py
print('bye')

Now two scripts produce slightly different results:
$ diff 1.py 2.py
20a21
> print('bye')
$ python 1.py
(((T5:0.161869436373317,T3:0.16186943637331702):0.023498203571967177,(T2:0.06328384937399689,((T7:0.0033653976337396055,(T10:0.0,T8:0.0):0.0033653976337396055):0.02687413460381695,T6:0.030239532237556554):0.033044317136440335):0.12208379057128729):0.11452667890031788,((T9:0.012543727576614564,T1:0.012543727576614564):0.034777908390860215,T4:0.04732163596747478):0.2525726828781273):0.1867278924742215
$ python 2.py
(((T10:0.05117261119826575,(((T5:0.0,T8:0.0):0.01079898660762326,T9:0.01079898660762326):0.02286191361221651,T7:0.03366090021983977):0.017511710978425975):0.2613333470907571,(T2:0.003592458452326968,T6:0.003592458452326968):0.30891349983669586):0.017361623925104708,((T3:0.10902504017366038,T1:0.10902504017366038):0.2026460498516629,T4:0.31167109002532334):0.01819649218880426):0.10581733263041147

You can clone files from here.
Behaviour is slightly different between python2 and python3. In python2, 2.py generates different results every run, while 1.py is returning the same tree. In python3 both are reproducible, but inconsistent with each other.
This was done in a clean virtualenv environment with dendropy 4.3.0 and numpy from pypi 1.13.3. I tried two different linux machines. The bug reproduces on both.
In dendropy 4.2.0 I could not reproduce the issue.

Confirmed this has been fixed in newer versions of DendroPy.


alexnet parameter amount inconsistent

I tried to utilize the example code in ./example/image-classification/symbol_alexnet.py.
However, when I checked the shape to compute the parameter amount of the network, I found that it was not consist to the number presented in the original paper (~60M).
shape checking:
(96, 3, 11, 11)     param: 34848
(96, )              param: 96
(256, 96, 5, 5)     param: 614400
(256, )             param: 256
(384, 256, 3, 3)    param: 884736
(384, )             param: 384
(384, 384, 3, 3)    param: 1327104
(384, )             param: 384
(256, 384, 3, 3)    param: 884736
(256, )             param: 256
(4096, 6400)        param: 26214400
(4096, )            param: 4096
(4096, 4096)        param: 16777216
(4096, )            param: 4096
(1000, 4096)        param: 4096000
(1000, )            param: 1000

Total parameter: 50844008 (~50M)

I guessed the problem occurs between the pooling layer and the fully-connected layer, but I had no idea how to solve it.
Does anyone know how to fix it to make it correspond to what the original paper mentions?
Thanks.

Same issue.
The shape between poll_3 and fc_1 should be (4096, 9216).
I try to use flatten = mx.symbol.Reshape(data=pool3, target=[0, 256 * 6 * 6]), but it don't work.

fixed. add pad=(2,2) at first conv layer.
https://github.com/guolinke/deep-learning-benchmarks/blob/master/mxnet/models/alexnet.py#L15

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!


Method search has an interface problem

Hi, I face a problem while using the node.search method.

This works but it's not great:

node.search(SearchType.Tag, "...")

It works but mypy is warning me about the fact that SearchType doesn't have a Tag attribute, which is tru as everything is lowercased in the python Enum.

This fails but should work according to the search method type hints:

node.search("tag", "...")
# OR
node.search("Tag", "...")

This fails as the string can't be converted to a SearchType (TypeError: argument 'by': 'str' object cannot be converted to 'SearchType')
I think the problem comes from the way the SearchType Enum is defined in the rxml.pyi file.
According to this Single Enum section of the pyo3 documentation the python SerachType class should be like this:
class SearchType:
    Tag: 'SearchType'  # These are class attributes
    Attr: 'SearchType'
    Text: 'SearchType'

    def __eq__(self, other: object) -> bool: ...
    def __int__(self) -> int: ...

And the Node search method should look like this:
def search(self, by: SearchType, value: str, depth: int | None = None) -> list[Node]: ...

sry, i've really written the pyi wrongly, the implementation does use uppercase on the first letter(as you can see here), i will fix it

same for the search method definition, i've written it wrong, fixed on the 2.1.2 release

actually, by default python(and pyo3) enum are integers, so, i'm changing the def to int on the pyi, releasing as 2.1.3


Drop Boost dependency

Some potential users of faker library asks for deleting boost dependency because they cannot afford to have this dependency. We could use date lib for replacing date operations or come up with other solution.

Utilizing the standard chrono library would remove the need for a dependency altogether: https://en.cppreference.com/w/cpp/chrono

Yes, it would be an important change to make in faker library. Good idea. I will try it. Would you like to work on doxygen? I actually created some config file for doxygen but I dont have any experience with that, could you take a look at this? (and this autodeploying).
https://github.com/cieslarmichal/faker-cxx/pull/12

Fixed in https://github.com/cieslarmichal/faker-cxx/pull/21


Unintentional Nuking of Instruction Keyframe for All Blockbench Formats

Whilst using the newest version of the Entity Plugin, I have noticed that there has been a nuking of the Instruction Keyframes within the Effects timeline in the Animation section. This removal extends past the itemsadder format to the entirety of Blockbench and effects all formats, which is a detriment if other models utilize instruction keyframes for certain actions.
Steps to Reproduce:

Have Blockbench and the latest version of the Itemsadder entity format installed.
Open a Generic format model.
Go to Animate.
Click the "Animate Effects" wand near the timeline and place an Instructions Keyframe.
Notice that under Keyframe (Instructions) there is nowhere to input text when there should be an input box.

Expected Outcome: The Instruction Keyframes should not be nuked for non-Itemsadder entity format projects.

Whilst using the newest version of the Entity Plugin

Are you sure this doesn't happen on previous versions? for example 1.2.0 and 1.1.2.
Thanks let me know

Anyway here is an updated version.
Let me know if everything works as expected.
iaentitymodel_test_1.2.2.zip

Anyway here is an updated version. Let me know if everything works as expected. iaentitymodel_test_1.2.2.zip

Tried this out and the issue seems to be fixed for generic format! The input box for the instruction script now shows up as intended. :)

Amazing!
Thanks for the test


feat: cbc flag when total project budget does not add up

Implements NDT-315

[ ] Check to trigger automatic release process

depends on quick edit


Script to build docker multi arch images based on config

There are some docker images can build multiarch directly without having separate tags(amd64, arm64). Create a config file which container target tags and arch

https://github.com/BretFisher/multi-platform-docker-build


Discussion: handle recursion?

Currently this library does not handle self-referencing models and models with cyclical references. For example:
from pydantic import BaseModel

class A(BaseModel):
    bs: List["B"]

class B(BaseModel):
    a: A

In the above there are two models with cyclical dependencies - A that has a list of B and B that has A. If we simply resolve the forward ref and then generate values for one of these, we will hit a recursion exception. While my original thinking was that we should not handle this case and instead let users customize factories to handle it manually - this creates problems downstream, see: https://github.com/starlite-api/starlite/issues/196. I would like to discuss how to resolve this.

If this was
class A(BaseModel):
    b: "B"

class B(BaseModel):
    a: A

It would be downright impossible do instantiate this anyway... so "true" recursion is really possible unless there's an "out"
.
In the example you have, the "out" is the "empty list" in the example from the startlite bug, the "out" is a None on the subentity.
My recommendation would be that anything that can be empty is empty unless specified otherwise, and everything Optional[] is None, unless specified otherwise.
I know this would break the current behaviour and people might rely on this behaviour that sometimes Optional stuff is None and dict/list have 1 element and sets have a lot of letters.
For the record, i personally so make my ModelFactories have a Use(list) or similar for any "emptyable" field.  My philosophy is that worst case should be handled, and if you NEED a value in there it's because you're testing what happens with it, so might as well define it clearly in your test :)
A backward compatible option could be way to turn this ModelFactory into a "pessimist" mode that would do what I do making stuff empty or None, which could resolve these recursive references.

We should aim for backward compatible solutions.

So how should we handle the provided example then?

So how should we handle the provided example then?

I think to add a max recursion value to a factory with a default. Upon identifying more than the max recursion, switch the value to none, if it's optional, otherwise raise a verbose error with user instructions


Fix code signing on Xcode 8 for device builds

Need to update our project file for Xcode 8 in the same way as Himotoki did:
https://github.com/ikesyo/Himotoki/pull/130

Fixed by https://github.com/garnett/Swinject/pull/1


implement command sandboxing

Current dependencies on/for this PR:

main

PR #2   👈

This comment was auto-generated by Graphite.


Hosting a MemGator instance on FOSSHOST

Contact FOSSHOST to see if they will be willing to support an instance if MemGator for public use.

FOSSHOST appears to have shutdown per this Dec. 2022 HN post.

If you wait long enough, the issues close themselves ;-(

@phonedude Sadly, yes. PRs welcome. ;-)

ahem.  https://github.com/oduwsdl/MemGator/commit/aa956b6a4ae8bacda4d935caba906c3fbfc2ab6e

@phonedude


Several rs combined be

I noticed that in vcf file, ExAC stores different variations with different rsIDs by one rsID.
Example:
quote from DBsnp vcf file (37 genome version):
1	32128283	rs745892055	C	CA,CCCCCG	.	.	RS=745892055;RSPOS=32128283;dbSNPBuildID=144;SSR=0;SAO=0;VP=0x050000080005000002000200;GENEINFO=COL16A1:1307;WGT=1;VC=DIV;INT;ASP
1	32128283	rs1568396	C	A,G	.	.	RS=1568396;RSPOS=32128283;dbSNPBuildID=88;SSR=0;SAO=0;VP=0x05010008000517013e000100;GENEINFO=COL16A1:1307;WGT=1;VC=SNV;SLO;INT;ASP;VLD;G5A;G5;GNO;KGPhase1;KGPhase3;CAF=0.4902,.,0.5098;COMMON=1
quote from ExAC:
1    32128283    rs1568396    C    G,CCCCCG,A,CA    2768259.72    PASS    AC=15561,603,185,4;AC_AFR=358,0,18,0;AC_AMR=51,0,1,0;AC_Adj=3531,0,25,1;AC_EAS=181,0,0,0;AC_FIN=4,0,0,0;AC_Het=1603,0,6,0,0,17,1,0,0,0;AC_Hom=955,0,1,0;AC_NFE=1391,0,2,1;AC_OTH=43,0,1,0;AC_SAS=1503,0,3,0;AF=0.458,0.018,5.447e-03,1.178e-04;AN=33964;AN_AFR=532;AN_AMR=114;AN_Adj=7246;AN_EAS=244;AN_FIN=8;AN_NFE=2472;AN_OTH=84;AN_SAS=3792 ...
So basically one position is associated with several different types of variations simple SNP and Insertions. Each type of variation has it's own rsID (rs745892055,rs1568396) and in ExAC you mix together both type in one variation and use only one rs1568396 without mention of rs745892055.


Add "engines" field to package.json

Especially since this project uses experimental flags, it may be best to include an "engines" field to package.json, so we know what versions of Node are required to run WebRun.

At the moment webrun actually bundles Node 10, but I'm not sure if newer versions support the necessary flags.
A PR with an engines field would be welcome. 😁

Ah, I overlooked that it bundled Node. I'll look into this over this next week and report back.

Another thing that I hadn't considered until I was working on the pull request (#31), is that the other dependencies in this project may have other Node needs. So, even though you may be safe with Node v10+, I haven't done the research to know how safe any of the other dependencies are with that choice.


Store preferences in OSM profile with fallback to localStorage

Currently every time I want to edit on OSM.org, this modal asking me if I’m interested in a tutorial (or just want to start editing) pops up every time. Yes, the new editor is awesome and no, I don’t need a tutorial. ;) At some point in the past I’m pretty sure I did the tutorial and it probably helped me. But now the popup is annoying – every time.
TL;DR: Only show this popup once for a user. When I click »Edit« in the top bar, I want to directly edit. (I assume the tutorial can be accessed otherwise anyway if it’s needed later.)
As @tomhughes said in https://github.com/openstreetmap/openstreetmap-website/issues/1162#issuecomment-189285230, the mechanism for remembering if to show the popup or not is briwser local storage. It would be more appropriate to permanently store it with the user profile because otherwise it will pop up again when switching browsers or devices, clearing profiles, reinstalling etc – everything which has nothing to do with the editor.

The store will definitely need to stay as the browser's localStorage.  There is no capability to store editor preferences in the user's OSM profile.
Have you disabled localStorage, or are running in incognito mode?

BTW, I was wrong before and there actually is a way to store editor preferences with the user's OSM profile.
http://wiki.openstreetmap.org/wiki/API_v0.6#Preferences_of_the_logged-in_user

Hello @bhousel , @jancborchardt
I would like to work on a fix for this. Has anyone already started looking into this? Or has this been fixed?
What I plan to do this aunthenticate with the logged in user and send a PUT request with a key and a value which will enable the user to remember his/her choice.

I'm not sure whether we want to do this.  My initial feeling was to just close it as WONTFIX, but I decided to leave it open for discussion.  Do we really want to store user preferences in OSM?
We do store a bunch of things in localStorage that could just as well be stored in OSM, but we'd need to add the necessary methods to iD.Connection to PUT/GET to these endpoints and serialize/deserialize the XML.  It just seems like a lot of work for very little benefit.
For this to work the user needs to be already authenticated.  This is the case when iD is embedded in OSM.org, but not necessarily when iD is used on other sites.

There's not much precedent for what to store in the user preferences. Anything that is stored should be valid across multiple computers and editor instances, which the iD walkthrough remembering is. If you've gone through the iD walkthrough on osm.org, you don't need to on another site.

I'm in favor of storing preferences using the API. Maybe just store them all in one JSON blob?

Since it stores key=values, I think it makes more sense to store individual values for most items.

@jfirebaugh how about history too?  I have been regularly pushing my edits up into the many hundreds and overflowing the localStorage limit.

Yes I'd also prefer separate key=values for flexibility in changing things.

History isn't a preference, and storing it on the server means that you'll loose if you can't upload, which is when you need it the most.

Hello,
Even I think storing on the API Side would be a nice idea. I would like to look into this. So would need some guidance on this. I guess this will require a new pair of key-value? So how do I add this pair to the user preference list?
By the way on this site : http://www.openstreetmap.org/edit?editor=id that popup is visible but it isnt visible on www.openstreetmap.org. Is that the expected behaviour?

@DarkPrince304 This is not a very good task for a new contributor, but if you really are interested my advice is to look in iD.Connection at how iD communicates with the OSM API.
There is an authenticated() method that would tell you if the user is already logged in.
Code in the global iD context object manages localStorage, so that is the place where you'd need to check whether the user is authenticated and call the (not yet written) code in iD.connection to pull values from server side storage.  It would need to fall back to localStorage if the user is not authenticated, and ideally push local storage to server storage once authentication happens.

See https://github.com/tyrasd/overpass-turbo/commit/d1933261c9d8bebb1f96b676871736454479a3d7 for an example of implementation of http://wiki.openstreetmap.org/wiki/API_v0.6#Preferences_of_the_logged-in_user.

This relates to #3239 / #4418 and would land as an iD v3 feature..
There are currently assumptions in the code that a user's settings are available synchronously at startup (e.g. to show the splash screen and restore history), and introducing an asynchronous dependency would be a breaking change.

I was still able to trigger the "Welcome to the iD OpenStreetMap editor" by directly clicking "edit" from the osm.org homepage when not logged in.
I think the assumption is the user will first click "Log in", then after logging in, then click "Edit".
However if he instead first clicks "Edit", (then fills in his password,) he will trigger the "Welcome to the iD OpenStreetMap editor" pop-up (despite having used iD lots before.)

@jidanni: I actually got it will being logged in and clicking on Edit, so I think it's because of the new privacy policy (#7040).

The preference to show third party icons, which the UI implies will be stored, is currently not. I'm assuming fixing that would be part of this bug?

@jtracey Third-party icons aren't intended to be stored between sessions. The privacy option is about whether to load them at all.

@quincylvania I know, I'm saying the setting to not do that isn't stored (i.e., I have to disable it every time I load iD).

I'm saying the setting to not do that isn't stored (i.e., I have to disable it every time I load iD).

The preference should be stored, unless you are using something like incognito mode, or maybe a privacy-focused web browser that throws out the storage.

I delete local storage when I close the browser, yes. Which is exactly what this issue is about ameliorating, no?

I delete local storage when I close the browser, yes. Which is exactly what this issue is about ameliorating, no?

Yes.


Total number showing references or definitions count during Go to or Find all references is misleadig

VSCode Version: Alpha
OS Version: Windows 8

Steps to Reproduce:

In a Typescript (or any project that supports Go to and Find all references), make sure you have a symbol (eg: class) referred more than in one file
Go to a file where this symbol is referred and do Find all references
A peek dialog is shown with all references and total count.

I see that in the right side of the peek window we have count per files and on the top we have total count. Since on the top, we show file name / path with total count, it is misleading that count is for file.
Probably we should show the total count also on the right or update the label

sort of by design


[R20-708] Image loading performance

Issue: https://digital-vic.atlassian.net/browse/R20-708
What I did

Added ability to implement image provider in RplImage
Implemented srcset calculations based on asset dimensions and design treatment

How to test

Checklist

For all PR's

[ ] I've added relevant changes to the project Readme if needed.
[ ] I've updated the documentation site as needed.
[ ] I have added unit tests to cover my changes (if not applicable, please state why in a comment)

For new components only

[ ] I have added a story covering all variants
[ ] I have checked a11y tab in storybook passes
[ ] Any events are emitted on the event bus

Fixtures will probably need updating for cypress to pass

Next steps: rework this to output <picture> element blocks instead to make the aspect/treatment versions a bit clearer - at the moment it's (ab)using srcset to do this

Getting too stale, will need to recreate this PR


New version: StochDynamicProgramming v0.6.0

Registering package: StochDynamicProgramming
Repository: https://github.com/JuliaStochOpt/StochDynamicProgramming.jl
Created by: @frapac
Version: v0.6.0
Commit: 4fb9694d5f2272105b5dfa99053143d4d0c3e8d0
Reviewed by: @frapac
Reference: https://github.com/JuliaStochOpt/StochDynamicProgramming.jl/commit/4fb9694d5f2272105b5dfa99053143d4d0c3e8d0#commitcomment-38312331

[noblock]
I have added the bounds on master. Would you like me to reopen a new PR?

[noblock]
Re-trigger Registrator on the latest commit. This will update this existing PR.


Vram requirements

Hi, could you pls post vram requirements for the models pls?
Thanks.

Sure, we will show the VRAM requirements for the models after sorting them out.

Thanks!

Hey @wuhy68 could you point me to the vram requirements I can't find them.

We still work hard to we will show the VRAM requirements a little later

We are working hard to sort out some experiment results, and we will show them (including VRAM requirements) together a little later. Thank you for your attention.

Thank you! Looking forward


CLOUDSTACK-9104: VM naming convention in case vmware is used

I have reverted all the changes. Now functionality is same as it was  in earlier version.
ACS Link ==>
https://issues.apache.org/jira/browse/CLOUDSTACK-9104

@priyankparihar I do not know any vmware users
@bhaisaab Do you have any insight on this change? or do you know any candidate reviewer for this?

@DaanHoogland yes I can do that, I'm personally dealt with a lot of vmware/cloudstack naming issues. The syntax/pattern has changes several times in the past between 4.0, 4.2, 4.3 causing VM sync issues when users upgrade.

@bhaisaab @priyankparihar what is the verdict on this change?

@bhaisaab
User with different account cannot create VMs with the same name, which was possible earlier (I am not sure in which CCP version). That time naming convention used was like this “I--”
Currently if vm.instancename.flag is set to true the VM name will be exactly as display name given.

@priyankparihar Fair enough, please see a code review comment. Once you fix the null check, I think your PR is good to go. Also, write a unit test to enforce the changes; i.e. the unit test should break in future if someone tries to change the VM naming convention in future.

@bhaisaab I think null check is not required. I have verified( because in case of null if automatically picks  some random value). But if you still think it may break something then please notify.

@priyankparihar I won't be able to give a LGTM without the null check, split string related code and a unit test that breaks if someone tries to change the convention in future


i26

PR Overview

Cleaned up README, was getting a bit long and repetative in places
Increased check-in survey modal size to "large". Extra-large seemed too big, lots of whitespace on a 1920x1080 monitor:

...versus large:

Not quite "full-screen" but an improvement.

Thoughts on large vs xl appreciated

@Murkantilism I prefer the XL modal with the survey centered horizontally & vertically. It feels a little less claustrophobic that way.

@Murkantilism I prefer the XL modal with the survey centered horizontally & vertically. It feels a little less claustrophobic that way.

@BrennenHodge We can merge this. I will pull and sort this out.


Audio docs

Can I please know the Package u used ?
https://pub.dev/packages/just_audio ?

Ah ok I saw it.
https://github.com/flet-dev/flet/issues/237


Unknown characteristic

Hardware:
Board: ESP-WROOM-32
Core Installation/update date: 01/MAY/2021
IDE name: Arduino IDE
I am able to get a discription as text with 2901 and a characteristics as text on the client by calling the format in the 2904 but can I set the name of the characteristic to "my characteristic" in stead of "unknown characteristic" by the server to the client?

Oeps, forgot to respond. I diden't continue on creating an app, because the app need different premission for ios bluetooth, than an android device.

No worries 😃, just cleaning up a bit. Feel free to re-open this if you need.


Update climate data pull to include 2019-2021 data

Latest WorldClim data include data 2010-2019; prior efforts only included 2010-2018. URLs for datasets are updated at https://www.worldclim.org/data/monthlywth.html. Need to update src/data/prep-climate-data.R with new URLs. Should just need to change this line:
time_periods <- c("2000-2009", "2010-2018")

to this (2018 -> 2019):
time_periods <- c("2000-2009", "2010-2019")

Could be done in parallel with #94 .

Addressed in db71171.


df.hvplot() not working in colab

The example from the usage page returns no output in a python3 colab notebook:
!pip install hvplot holoviews

import pandas as pd, numpy as np
idx = pd.date_range('1/1/2000', periods=1000)
df  = pd.DataFrame(np.random.randn(1000, 4), index=idx, columns=list('ABCD')).cumsum()

import hvplot.pandas
df.hvplot()

With a python2 runtime, the second cell hangs.

I got this too.... (with python3)
I think hvplot is not using the index as the x variable... doing
df=df.reset_index()
df.hvplot(x='index')

works. Is this a bug?

This worked for me; I think you need to add two steps in Colab: hv.extension('bokeh') and hvplot.show()
import holoviews as hv
hv.extension('bokeh')

import hvplot.pandas
import pandas as pd, numpy as np

idx = pd.date_range('1/1/2000', periods=1000)
df  = pd.DataFrame(np.random.randn(1000, 4), index=idx, columns=list('ABCD')).cumsum()

hvplot.show(df.hvplot())

hello @ahuang11
I tried the same code but there is an ImportError
Do you have any idea?
WARNING:param.notebook_extension: Holoviews bokeh extension could not be imported, it raised the following exception: ImportError('cannot import name 'GridBox'')
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-5ffd9cebd254> in <module>()
      1 import holoviews as hv
----> 2 hv.extension('bokeh')
      3
      4 import hvplot.pandas
      5 import pandas as pd, numpy as np

2 frames
/usr/local/lib/python3.6/dist-packages/holoviews/util/__init__.py in __call__(self, *args, **params)
    698
    699         if selected_backend is None:
--> 700             raise ImportError('None of the backends could be imported')
    701         Store.set_current_backend(selected_backend)
    702

ImportError: None of the backends could be imported

@loopinf !pip install -U bokeh should fix it. Bokeh is probably outdated on Colab.

Right; you need bokeh 1.1.0 or later.

So I'm going to close since this is a Panel/HoloViews issue and there's already this issue: https://github.com/holoviz/holoviews/issues/3551


Implementation of MOI

Apart from this typo, optimize!(model) will still not work

Apart from this typo, optimize!(model) will still not work

Most probably. But I can't see the error unless you post it, or it fails on Travis so I can see the log. Have you uploaded the example that is failing? The SparseMatrixCSC was suspicious in your Gitter message.

Ok I'll add a Example in test suite, you see it on Travis log where it fails.

julia> using CharibdeOPtim

julia> model = Model(with_optimizer(CharibdeOptim.Optimizer))
A JuMP Model
Feasibility problem with:
Variables: 0
Model mode: AUTOMATIC
CachingOptimizer state: EMPTY_OPTIMIZER
Solver name: CharibdeOptim

julia> @variable(model, 1<=x<=2)
x

julia> @variable(model, 1<=y<=2)
y

julia> @NLobjective(model, Min, x^2+y^2)

julia> optimize!(model)

Apparently, Travis is not able to load JuMP, I don't understand why, can you please test the example locally ?

@odow Please see the log of Travis for Test #95.2

expr have variables like x[MOI.VariableIndex(1)] and  x[MOI.VariableIndex(2)]  intstead of x[1] and x[2]
and thats why it giving error.
ERROR: MethodError: no method matching getindex(::Tuple{Int64,Int64}, ::MathOptInterface.VariableIndex)
Closest candidates are:
  getindex(::Tuple, ::Int64) at tuple.jl:24
  getindex(::Tuple, ::Real) at tuple.jl:25
  getindex(::Tuple, ::AbstractUnitRange{#s57} where #s57<:Real) at range.jl:268

So you need to walk the expression graph, and replace instances of x[MOI.VariableIndex(i)] with x[I]. Here is the code to do so:
https://github.com/odow/MathOptFormat.jl/blob/39048ec3d50153b38992f8a6b38a60396fa3433a/src/MOF/nonlinear.jl#L19-L35
You can call it like so:
https://github.com/odow/MathOptFormat.jl/blob/39048ec3d50153b38992f8a6b38a60396fa3433a/src/MOF/nonlinear.jl#L74

julia> optimize!(model)
ERROR: MethodError: no method matching isless(::Int64, ::MathOptInterface.VariableIndex)
Closest candidates are:
  isless(::Missing, ::Any) at missing.jl:66
  isless(::Real, ::AbstractFloat) at operators.jl:149
  isless(::Integer, ::ForwardDiff.Dual{Ty,V,N} where N where V) where Ty at /Users/yash/.julia/packages/ForwardDiff/N0wMF/src/dual.jl:140
  ...
Stacktrace:
 [1] <(::Int64, ::MathOptInterface.VariableIndex) at ./operators.jl:260
 [2] <=(::Int64, ::MathOptInterface.VariableIndex) at ./operators.jl:309
 [3] substitute_variables(::Expr, ::Array{MathOptInterface.VariableIndex,1}) at /Users/yash/code/CharibdeOptim/src/MOI_wrapper.jl:157
 [4] substitute_variables(::Expr, ::Array{MathOptInterface.VariableIndex,1}) at /Users/yash/code/CharibdeOptim/src/MOI_wrapper.jl:164 (repeats 2 times)

Error is occuring in if !(1 <= index <= length(variables)) in substitue_variables

Oh, that might be going to opposite way. You might need
function substitute_variables(expr::Expr)
    if expr.head == :ref && length(expr.args) == 2 && expr.args[1] == :x
        return :(x[$(expr.args[2].index)])
    else
        for (index, arg) in enumerate(expr.args)
            expr.args[index] = substitute_variables(arg)
        end
    end
    return expr
end
substitute_variables(arg) = arg

Thanks @odow , It worked..!!

Codecov Report

Merging #21 into master will decrease coverage by 3.34%.
The diff coverage is 56.98%.

@@            Coverage Diff             @@
##           master      #21      +/-   ##
==========================================
- Coverage    65.3%   61.96%   -3.35%
==========================================
  Files           7        8       +1
  Lines         147      234      +87
==========================================
+ Hits           96      145      +49
- Misses         51       89      +38

Impacted Files
Coverage Δ

src/IBC.jl
100% <100%> (+4.54%)
:arrow_up:

src/CharibdeOptim.jl
85.71% <100%> (ø)
:arrow_up:

src/DiffrentialEvolution.jl
100% <100%> (ø)
:arrow_up:

src/MOI_wrapper.jl
54.54% <54.54%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15653d9...6eb34b6. Read the comment docs.

@everywhere was not needed there, they were there by mistake, @everywhere has been used just once in @everywhere using CharibdeOptim to load the package on both procs


Playback Speed

a modifier to speed up or slow down songs, can be enabled on the Gameplay Changers menu (both on Story Mode and Freeplay)

Aprrove for public please

Too bad lime doesn't have a proper way of setting stuff like pitch

It's not better putting this condition of song ending instead of making a FlxTimer? idk.
if ((FlxG.sound.music.length / playbackRate) - Conductor.songPosition <= 0)
{
	shouldEndSong = true;
	finishSong();
}

BUILDING GAME
source/PlayState.hx:2184: characters 6-19 : Unknown identifier : shouldEndSong

done.
Press any key to continue . . .

Fixed compiling issue by changing some stuff
		if (generatedMusic && !endingSong)
		{
			if (!startingSong && !endingSong)
			{
				if ((FlxG.sound.music.length / playbackRate) - Conductor.songPosition <= 0)
				{
					endingSong = true;
					finishSong();
				}
			}
		}

This still causes menu music to be sped up though, but its a temp fix for me meanwhile at least.

will reopen this once I figure out a few annoyances I have with this, plus I committed everything to main which is quite annoying too

so i added it to my source and the results was nice, BUT the song stills without the slow shit, why this happens?

psych already has this officially

psych already has this officially, i've helped with chart editor stuffs a bit

i mean, i needed this for my mod but it's glitched af in this vers, the mod engine it's outdated af so i guess it's bc the mod engine its outdated so np


\log with different mathvariant

Currently, there are two different identifiers for \log. When \log is used in theorem environment, its mathvariant is "roman", though when \log is used in other environment, its mathvariant is "default" resulting in different identifiers.
The following example is from 2004.08500:

The default variant is context-dependent: for identifiers of multi-character string, it normally stands for roman, but sometimes italic as such in the theorem environment.
We should not leave the attribute of variant in the concept dictionary as default, but should expand it to the concrete typeface.

This issue sounds related to more general one #60 .


C/CPP : Implementation of queue using stack

Description 📜
I will explain enqueue and dequeue function using stack. ###Documentation

Domain of Contribution 📊

[ x ] C/CPP

/assign

/assign

Hey! I want to work on this issue.

Closing as the PR got merged !


Remove DictionaryArray::keys_array

The keys_array method is a performance pitfall if used in inner loops like in a group by operation. At the time the method was added, the related keys method was returning an iterator, making it difficult to randomly access elements. Since then the code was refactored so that keys already returns a reference to an Array, which has much less overhead than the code in keys_array.
This means the keys_array method can be removed and its usages directly replaced with keys.
This would be a backwards incompatible change. For the stable release branch we can think of marking the method as deprecated and direct users to keys instead, which might updates to the next major release then less painful.

@alamb for the release process would it be best to create two consecutive PRs? The first marking the method as deprecated, which can be cherry picked into the branch, and then a followup one which deletes the method in the master branch. Or should we go with a longer deprecation period?

@jhorstmann  my plan was just to release master as arrow-rs-5.0 (next major release) without going through any deprecation period.
I just don't want to push an incompatible API change to a release that cargo will automatically update (and thus break anyone using the arrow 4.x release line without them taking any action).
By releasing as part of 5.0 projects will have to explicitly update to arrow; 5.x in their Cargo.toml files and can handle whatever updates are necessary at that time (and on their schedule)
cc @jorgecarleitao @andygrove @nevi-me @Dandandan

AFAI understand, @jhorstmann 's point is that we can create two commits: a backward compatible one that adds a "deprecated", which can be released in 4.X, and a follow-up one backward incompatible one that removes the method.
This way, we can cherry-pick the first one into 4.X, informing users about the deprecation, and then remove the method in 5.X.

As long as whatever deprecation warning we add does not break CI for people (e.g. who have clippy set to make errors out of warnings) I think a PR to active_release marking the API as being changed would be fine.
I have had the unpleasant experience of my projects breaking due to some new release of a python package being published upstream and my CI picked it up automatically and started failing when we had made no changes to our code.


Ensure that IAM users are not assigned the Service Account User or Service Account Token Creator roles at project level

Possible check to target the following resource

[ ] google_project_iam_member

closing, tracking in spreadsheet


Use bridge network and expose port on macOS when launch docker image

This is related to #2270. For Mac users, running docker/bash.sh command to launch docker image doesn't really work, because Docker's host networking driver isn't supported on macOS.
This patch adds a detect to the script. For macOS, it simply uses bridge network and expose port 8888. Then users can start jupyter network in the launched container.
Install docs for docker is also updated.

Looks like the errored check isn't related. How to retest it?

You can just make a trivial change to an existing file (e.g., comment) to retrigger CI.
In the meantime we should also look into the flaky test.

Thanks, @viirya @eqy this is now merged!


how to remove an item in using section

How Can I remove an item from sections items let's say when on loadmore is clicked ?
https://github.com/CymChad/BaseRecyclerViewAdapterHelper/blob/749371b75edda1da0eab98e7726a833d87ff7729/app/src/main/java/com/chad/baserecyclerviewadapterhelper/SectionUseActivity.java

you can remove section data and then call adapter.notifyDataChange


App Service plan BadGateway

When I follow the instructions here:
https://www.youtube.com/watch?v=B5l5p9LvDSU
At the final step when I hit review and create, I'm getting an error,BadGateway.  It just keeps showing "Deployment is in progress"

I'm using Windows 11.
Any suggestions?

@Tom-H2oThermo I believe this might be related to the user permissions for creating app service plan. Please contact your Azure subscription owner for more help on this.

I figured out the problem.  I had to go to my app registration for the Single Tenant and Multi Tenant apps, which are pre-requisites for deploying the SaaS Accelerator.  Then under Manage/Owners I had to make myself an Owner.

I figured out the problem.  I had to go into my Single Tenant and Multi Tenant app registrations which are pre-requisites for deploying the SaaS Accelerator.  Then under Manage/Owners I had to make myself an owner.  Problem solved.


module not found

when trying to import validators in a main.py it works fine, but when I try to convert the project to exe
it shows that module not found
I used pyinstaller
also tried to user the command pyinstaller --onefile --hidden-import=validators main.py but again doesn't work

also this doesn't work pyinstaller --onefile --hidden-import=validator-collection main.py

sorry, my bad
it was working in main.py cause I download the package where I work -in PyCharm-
but when I decide to download it in my pc as usual, i downloaded this package validator-collection which isn't the same, but I didn't know that there are several with similar names