Custom Serializers

What was wrong?
The pickle/unpickle is a performance bottleneck that isn't necessary for simple payloads.
How can it be fixed?
Extend the API to allow custom serializers for payloads sent to/from a child process.

I ran some benchmarks to try and find where the bottlenecks are. Here are some numbers I got by running 1k calls to run(), using a no-op function and a single worker process.
pickling fn/args every run() call (standard behavior)

1KB arg: 2.93s
1MB arg: 4.62s

pickling fn/args only once, but re-sending over pipe every run() call (https://github.com/gsalgado/trio-run-in-process/commit/475445908f1b40b29446d76854b77f82bafcb9c5)
This means we just skip the pickling of fn/args, so the time difference (from above) should be the pickling overhead

1KB arg: 2.52s
1MB arg: 3.58s

hard-coded fn/args in child, never passed over pipe (https://github.com/gsalgado/trio-run-in-process/commit/9fd45163a255837aa17a8df172c34dd3f0a61008)
Here the difference (from above) is the overhead of passing the data over the pipe

1KB arg: 1.36s
1MB arg: 1.78s

https://gist.github.com/gsalgado/8631145aa42a95ba44a645796e444b46 is the script I used to get these numbers.
And here are some numbers for pickling (1k times) different types of objects. It's interesting to notice that pickling a no-op instance method takes twice as long as pickling a 1MB bytestring. (https://gist.github.com/gsalgado/4a12e1b178232f74d1d1417bcfeedbf7)
Pickling instance method: 0.07781410217285156 seconds
Pickling keccak(): 0.005815982818603516 seconds
Pickling 256 bytes of data: 0.0022199153900146484 seconds
Pickling 1MB of data: 0.0367732048034668 seconds

So, since the pickling overhead is on the order of 0.4s for 1k calls passing an instance method and 1KB of data (or 0.6s for 1MB of data), ISTM that is not a bottleneck as in any real-world use-cases the function will not be a no-op and thus would make 0.4s an insignificant amount of time. And although the overhead of passing data over the pipe is relatively higher, I don't think it'd be significant in real-world use cases either

I ran some benchmarks to try and find where the bottlenecks are. Here are some numbers I got by running 1k calls to run(), using a no-op function and a single worker process.
pickling fn/args every run() call (standard behavior)

1KB arg: 2.93s
1MB arg: 4.62s

pickling fn/args only once, but re-sending over pipe every run() call (https://github.com/gsalgado/trio-run-in-process/commit/475445908f1b40b29446d76854b77f82bafcb9c5)
This means we just skip the pickling of fn/args, so the time difference (from above) should be the pickling overhead

1KB arg: 2.52s
1MB arg: 3.58s

hard-coded fn/args in child, never passed over pipe (https://github.com/gsalgado/trio-run-in-process/commit/9fd45163a255837aa17a8df172c34dd3f0a61008)
Here the difference (from above) is the overhead of passing the data over the pipe

1KB arg: 1.36s
1MB arg: 1.78s

https://gist.github.com/gsalgado/8631145aa42a95ba44a645796e444b46 is the script I used to get these numbers.
And here are some numbers for pickling (1k times) different types of objects. It's interesting to notice that pickling a no-op instance method takes twice as long as pickling a 1MB bytestring. (https://gist.github.com/gsalgado/4a12e1b178232f74d1d1417bcfeedbf7)
Pickling instance method: 0.07781410217285156 seconds
Pickling keccak(): 0.005815982818603516 seconds
Pickling 256 bytes of data: 0.0022199153900146484 seconds
Pickling 1MB of data: 0.0367732048034668 seconds

So, since the pickling overhead is on the order of 0.4s for 1k calls passing an instance method and 1KB of data (or 0.6s for 1MB of data), ISTM that is not a bottleneck as in any real-world use-cases the function will not be a no-op and thus would make 0.4s an insignificant amount of time. And although the overhead of passing data over the pipe is relatively higher, I don't think it'd be significant in real-world use cases either

So the conclusion here if I'm reading your analysis correctly is that the serialization accounts for a trivial percentage of the total execution time and thus, custom (more efficient) serialization isn't going to bag us any real performance gains.  :+1:

So the conclusion here if I'm reading your analysis correctly is that the serialization accounts for a trivial percentage of the total execution time and thus, custom (more efficient) serialization isn't going to bag us any real performance gains.  :+1:

For the fn/args combinations I tested (and I also did it for a nontrivial instance method from a trinity component), yes. There might be fn/args that are really slow to pickle, but I say we worry about that whenever we run into one of those cases

For the fn/args combinations I tested (and I also did it for a nontrivial instance method from a trinity component), yes. There might be fn/args that are really slow to pickle, but I say we worry about that whenever we run into one of those cases


more chord support

It seems a lot of things could be added to the chord support.

No facility to specify inversions.  Also not clear that is_triad correctly works for inversions or other voicings that scramble the note order.
make_triad is tied to a scale, whereas it would be natural to have an interface that lets you specify a root note, major/minor/diminished/augmented, and optionally an inversion, since very often you want a chord that borrows from another mode and it would be annoying to always have to create a new scale every time you use a chord with accidentals.
is_triad doesn't seem to allow augmented triads
No easy support for extended chords

It might be nice to have a function like make_chord(root, major/minor/diminished/augmented, extensions..., inversion=...) where extensions could be scale degree like 7, 9, …, with some keyword arguments to indicate voicings like inversions, though to have complete freedom over voicing you might still have to specify it as just a set of notes in some cases.   Would also be nice to be able to specify standard chord symbols in some natural way, e.g. C9/G or Em*add9.
Would also be nice to be able to be able to take a set of notes and a given root note and identify a conventional name for the chord, although sometimes this is not unique.  e.g. if you give the set {G,A,C,E} and a root note A, then it should identify it as Am7/G, whereas if you give the same set and a root note C it should identify it as C6/G.

See e.g. lilypond's chord-naming facilities.

There is also the tension between an abstract notion of a chord as a collection of intervals (e.g. "dominant seventh chord"), independent of particular set of notes, and its realization with a particular root (e.g. "G7").
And of course there are other systems of naming chords, e.g. the "ii7" chord, that are relative to a particular root.
It's a bit hard to pin down what functionality you want here, since it depends a lot on what you want to use it for.

It would be awesome if this could be achieved!
I think discussing the implementation of so many features in the same issue might cause confusion.I suggest opening separate issues to discuss individual features step by step.
I think the first step is to perfect the calculation of intervals so that we can subsequently construct any chord structure we want.(I haven't tried this package yet, so I don't know if the interval calculation function is fully complete.)


Add a "Reposts" tab on channel pages

I would like to suggest the addition of a 4th tab on channel pages called [Reposts] (probably between [Content] and [About] tabs) I am not a coder, but I am a content creator. As such, I want to make it as easy as possible for folks to view MY content on my channel. I also want to share content from other creators. (Reposts) From creators that I want to support/like/approve of/etc. What I don't want, is for people to see a single list which combines both of those different categories. (The current [Content] tab) The way it is set up now seems to blur the line between:

Things that I have actually created and published.

and

Content that I like, that did NOT create, but have reposted to my channel.

Sure, I am sharing it on my channel page because I think "you" might like it too. But I also do not want someone to have to scroll through 10+ reposts to finally get back to content I DID actually make. Because, by then, I may have lost their interest. I know how quickly people abandon new adventures into the unknown online. Generally people will not go beyond a single clickable link to "try out/explore" something new. (Like a new LBRY channel) I want to do anything that I can to make that 30 seconds to a minute of someone's curiosity journey easier for them, and more beneficial for me, and the platform as well.
It would be amazing if someone could build a filter that automatically sorts out MY publishes (Content tab) from OTHER creators' content that I am also sharing on my channel (Reposts tab). Thus simplifying the content search process for platform users/potential channel followers. I think it adds a useful additional layer to the personality of any given channel. It helps people differentiate between "Hey this is me, and here is some cool stuff I have done" and "Hey if you like my stuff, then you are gonna love these guys" Which again, I think would help the content creator community to better support one another as well.
Again, I am NOT a coder. So although my proposed solution seems easy enough, I understand that the behind the scenes side of things is probably much more difficult. I truly appreciate ALL of the hard work that everyone here has put into, and continue to put into this project. Thank you all for reading through all of that and considering my suggestion. I hope that my small contribution here can help the project out somehow. 🍻📚✊

I have to agree with this one. It only makes sense. I would repost more myself if it was seperated from my original content. I personally, when visiting someones channel, have made it my personal policy that I will not follow a channel if all I see are reposts without scrolling. If I have to scroll to find someone original content then they do not make enough for me to follow.

As someone who uploads content daily I find it difficult to justify reposting videos because it'll always be making it harder for someone to see my video which has only just come out, I completely agree that it should be separated, I'd be way more likely to use the feature.

Feedback heard that we should do something here, thanks

This would be an awesome feature and help keep the interface clean 👍

This is now live :)

I know! I saw that and was SUPER excited! Thank you so much. You guys rock!
🤘🍻📚✊
On Fri, Jun 11, 2021, 4:30 PM Thomas Zarebczan @.***>
wrote:

This is now live :)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/lbryio/lbry-desktop/issues/4327#issuecomment-859895390,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AP24MS6SHBGBCRO4UJAI6YDTSJW6BANCNFSM4NTHJGVQ
.


Maintained

Hi @sripathikrishnan , is the repository still maintained?
If not, I'd like to fork the repository, and maintain the forked repository.
After that, I'll try to publish the Python package with the similar name.

您好，我已收到您的邮件，谢谢！！~~


Update office-365-management-activity-api-schema.md

Added the following under the heading "Azure Active Directory Secure Token Service (STS) Logon schema"
For a complete description of logonerrors refer to the list of Authentication and authorization error codes
Based on research and information from DaveFor -- contact DaveFor for any questions.

Docs Build status updates of commit 1fdc507:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

office-365-management-api/office-365-management-activity-api-schema.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


branch: oqsrepo Build Error

The error:
Traceback (most recent call last):
  File "build.py", line 270, in <module>
    build_openvpn_linux()
  File "build.py", line 185, in build_openvpn_linux
    shutil.copy('../oqs-openssl-output/openssl/lib/libcrypto.so.1.0.0', stagepath + '/' + OPENVPN_LINUX_PREFIX + '/lib')
  File "/usr/lib/python2.7/shutil.py", line 139, in copy
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 97, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: '/opt/PQCrypto-VPN/openvpn/build/scratch/stage//usr/local/openvpn/lib'
The command '/bin/sh -c cd /opt/PQCrypto-VPN/openvpn/build && python build.py' returned a non-zero code: 1

This is a simple error to fix, but after I fix it more errors just keep coming. If you look at "'/opt/PQCrypto-VPN/openvpn/build/scratch/stage//usr/local/openvpn/lib'" it has two forward slashes instead of one. Please take a look.
Dockerfile:
FROM ubuntu:19.10

RUN apt update && apt install -y \
        liblz4-dev               \
        liblzo2-dev              \
        libpam-dev               \
        libssl-dev               \
        libtool                  \
        libtool-bin              \
        cmake                    \
        make                     \
        autoconf                 \
        python                   \
        git

RUN cd /opt && git clone --branch oqsrepo https://github.com/microsoft/PQCrypto-VPN.git
RUN cd /opt/PQCrypto-VPN/openvpn/build && python build.py

I should mention that python3 has the built in path module in pathlib which handles path independent of operating system. If the build script is ever ported to python 3, path error of this kind can be avoided all together.
Edit 1:  The "oqsrepo" branch was merged into master at commit 1d358944dbe0e4483a578aa141d3772a096423dd for future reference.

I'm surprised the two forward slashes ends up being a problem. This doesn't happen on my Ubuntu 18.04 build VM. #8 fixes the build problem there with the OQS-OpenSSL repo and I get a successful build, so I'm going to merge that fix.
I'll then look at reproducing your error in a container per your instructions to see what's causing those problems. Depending on the cause and how much work will be required to do a fix will determine if I can put a fix up sooner, or if it'll have to wait and get done with our next release.
Thanks again for the reports!

I tested with Docker base image ubuntu:18.04 and the issue is the same:
Cloning openvpn ...
 I'll check it out.

The double forward slash wasn't actually an issue. The issue there was not all of the needed directories were being created ahead of time, so file operations to place files in them were failing.
It turns out we also rely on a few other command line utilities that are standard on typical Ubuntu installs but are not present in the Docker image.
The script will now skip the Windows build without failure if the Windows-built OQS-OpenSSL DLL's aren't present. At the point where we snapped their tree, OQS-OpenSSL for Windows couldn't be cross-compiled on Linux.
See #10.

Great. That works! ( That is, #10  fixes all the issues for building.)


adding else if example

Good to merge for me :+1:

If we don't get any objection until tomorrow, I'm gonna merge it.

+1


bug fix

bug fix: The function GetModelFolder used in the code is actually named get_model_folder in the deeplabcut.
Move some of the imports to the top.

@CBroz1 Have you noticed a problem with the GetModelFolder? I wonder if they renamed the function in another version of the deeplabcut

Let's use the most recent one and require deeplabcut to be >= 2.2.1

Sure, sounds good. We can also remove this version checking logic

@CBroz1 scorer_legacy will be 0?

@CBroz1 Please don't merge this one until we decide on the Equipment-Device decision

What's the problem there?

What's the problem there?

Original comment
The docstring for get_trajectory does not specifically state that "all" is an option

I think it's good now. What do you think?


Use pycord

Although discordpy has resumed development, it has still not released in the past year. For now, Pycord seems to be the best, active fork of discordpy which has released a new version featuring slash commands (nextcord is also available, but does not look as reliable imo).

Closed in favor of using<EMAIL_ADDRESS>


Same as ContextualVersionConflict exception in Google Colab issue #108

CodeCarbon version:1.0.0
Python version: 3.6.9
Operating System: Ubuntu 18.04.5 LTS (Bionic Beaver)

Description
Describe what you were trying to get done. Your example
Tell us what happened, what went wrong, and what you expected to happen.
Example works fine before 2021/01/15 never after
What I Did
Your example
!pip install codecarbon

import tensorflow as tf

from codecarbon import EmissionsTracker

mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential(
    [
        tf.keras.layers.Flatten(input_shape=(28, 28)),
        tf.keras.layers.Dense(128, activation="relu"),
        tf.keras.layers.Dropout(0.2),
        tf.keras.layers.Dense(10),
    ]
)

loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)

model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])

tracker = EmissionsTracker()
tracker.start()
model.fit(x_train, y_train, epochs=10)
emissions: float = tracker.stop()
print(emissions)_

Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

Collecting codecarbon
  Downloading https://files.pythonhosted.org/packages/1d/78/0f6f06250558ca5dc21abb28660fecb78f09b4d5258d8e634393a780d1a6/codecarbon-1.0.0-py3-none-any.whl (49kB)
     |████████████████████████████████| 51kB 2.8MB/s
Collecting APScheduler
  Downloading https://files.pythonhosted.org/packages/5d/e9/b5e4f7aaea076bbfb241b04824c63cd42fe8029358103c27375e690bddc0/APScheduler-3.7.0-py2.py3-none-any.whl (59kB)
     |████████████████████████████████| 61kB 4.1MB/s
Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from codecarbon) (2.23.0)
Collecting dash
  Downloading https://files.pythonhosted.org/packages/bc/b4/0bd5c94fdcb0eccb93c3c8068fe10f5607e542337d0b8f6e2d88078316a9/dash-1.19.0.tar.gz (75kB)
     |████████████████████████████████| 81kB 3.9MB/s
Collecting fire
  Downloading https://files.pythonhosted.org/packages/11/07/a119a1aa04d37bc819940d95ed7e135a7dcca1c098123a3764a6dcace9e7/fire-0.4.0.tar.gz (87kB)
     |████████████████████████████████| 92kB 4.7MB/s
Requirement already satisfied: dataclasses in /usr/local/lib/python3.6/dist-packages (from codecarbon) (0.8)
Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from codecarbon) (1.1.5)
Collecting dash-bootstrap-components
  Downloading https://files.pythonhosted.org/packages/1d/26/17f8b18a79bae9e278b7674ad0aef8007e87b4d2280525a95103fd2a8fe5/dash_bootstrap_components-0.11.1-py2.py3-none-any.whl (187kB)
     |████████████████████████████████| 194kB 6.4MB/s
Collecting pynvml
  Downloading https://files.pythonhosted.org/packages/1b/1a/a25c143e1d2f873d67edf534b269d028dd3c20be69737cca56bf28911d02/pynvml-8.0.4-py3-none-any.whl
Requirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from APScheduler->codecarbon) (1.15.0)
Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from APScheduler->codecarbon) (2018.9)
Collecting tzlocal~=2.0
  Downloading https://files.pythonhosted.org/packages/5d/94/d47b0fd5988e6b7059de05720a646a2930920fff247a826f61674d436ba4/tzlocal-2.1-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=0.7 in /usr/local/lib/python3.6/dist-packages (from APScheduler->codecarbon) (53.0.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (1.24.3)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->codecarbon) (2020.12.5)
Requirement already satisfied: Flask>=1.0.4 in /usr/local/lib/python3.6/dist-packages (from dash->codecarbon) (1.1.2)
Collecting flask-compress
  Downloading https://files.pythonhosted.org/packages/b2/7a/9c4641f975fb9daaf945dc39da6a52fd5693ab3bbc2d53780eab3b5106f4/Flask_Compress-1.8.0-py3-none-any.whl
Requirement already satisfied: plotly in /usr/local/lib/python3.6/dist-packages (from dash->codecarbon) (4.4.1)
Collecting dash_renderer==1.9.0
  Downloading https://files.pythonhosted.org/packages/be/a6/dd1edfe7b1102274e93991736c35b2a5e1a63b524c8d9f41bbb30f17340b/dash_renderer-1.9.0.tar.gz (1.0MB)
     |████████████████████████████████| 1.0MB 7.2MB/s
Collecting dash-core-components==1.15.0
  Downloading https://files.pythonhosted.org/packages/22/78/ae0829e673f3df77403bcdb35073b1ed2f156080f5bcac6f21c1047d73fe/dash_core_components-1.15.0.tar.gz (3.5MB)
     |████████████████████████████████| 3.5MB 13.2MB/s
Collecting dash-html-components==1.1.2
  Downloading https://files.pythonhosted.org/packages/3f/25/56de2708128fe375eecc2e18e0ccdc3a853494966e36334ec8a30be99b94/dash_html_components-1.1.2.tar.gz (188kB)
     |████████████████████████████████| 194kB 40.6MB/s
Collecting dash-table==4.11.2
  Downloading https://files.pythonhosted.org/packages/3d/ae/96cb64b58d76391604b57f8c747f9a19ab2122e7ba214e2e0cf35484962b/dash_table-4.11.2.tar.gz (1.8MB)
     |████████████████████████████████| 1.8MB 38.3MB/s
Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from dash->codecarbon) (0.16.0)
Requirement already satisfied: termcolor in /usr/local/lib/python3.6/dist-packages (from fire->codecarbon) (1.1.0)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.6/dist-packages (from pandas->codecarbon) (2.8.1)
Requirement already satisfied: numpy>=1.15.4 in /usr/local/lib/python3.6/dist-packages (from pandas->codecarbon) (1.19.5)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (1.1.0)
Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (7.1.2)
Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (2.11.2)
Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.4->dash->codecarbon) (1.0.1)
Collecting brotli
  Downloading https://files.pythonhosted.org/packages/b4/d3/7c98f05b7b9103e2f3a112ba42f269c798155b3e5404fb80bb8f823aaebe/Brotli-1.0.9-cp36-cp36m-manylinux1_x86_64.whl (357kB)
     |████████████████████████████████| 358kB 41.2MB/s
Requirement already satisfied: retrying>=1.3.3 in /usr/local/lib/python3.6/dist-packages (from plotly->dash->codecarbon) (1.3.3)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10.1->Flask>=1.0.4->dash->codecarbon) (1.1.1)
Building wheels for collected packages: dash, fire, dash-renderer, dash-core-components, dash-html-components, dash-table
  Building wheel for dash (setup.py) ... done
  Created wheel for dash: filename=dash-1.19.0-cp36-none-any.whl size=84013 sha256=08dc3f0960a54ea6da81e46a8b2a86f2edbe642fc90919781281d6a8886e4dee
  Stored in directory: /root/.cache/pip/wheels/f7/50/a7/a230ff7f503b10120bff18c2524a375bb85a61ce6b519c8a77
  Building wheel for fire (setup.py) ... done
  Created wheel for fire: filename=fire-0.4.0-py2.py3-none-any.whl size=115928 sha256=98b65b034d06b4f56aa4bb96d6f370f73979c28c85c859735497f9730ea62c04
  Stored in directory: /root/.cache/pip/wheels/af/19/30/1ea0cad502dcb4e66ed5a690279628c827aea38bbbab75d5ed
  Building wheel for dash-renderer (setup.py) ... done
  Created wheel for dash-renderer: filename=dash_renderer-1.9.0-cp36-none-any.whl size=1014869 sha256=50e43efd2f7bce53545b685a442ae465944268e14b39046367d6bf9065392c26
  Stored in directory: /root/.cache/pip/wheels/46/a0/ec/2be2e8fc750e623d76f9690c397cc5ab28b33d0a16a49e10c5
  Building wheel for dash-core-components (setup.py) ... done
  Created wheel for dash-core-components: filename=dash_core_components-1.15.0-cp36-none-any.whl size=3527014 sha256=3aa1d57e14283ec396209981d532ce23ccc4797878ad6bccbbc0382cdfe67191
  Stored in directory: /root/.cache/pip/wheels/53/3d/be/d628d6f66eedf9597f0c89c8ff43a5020ad1c25152c77d8e9f
  Building wheel for dash-html-components (setup.py) ... done
  Created wheel for dash-html-components: filename=dash_html_components-1.1.2-cp36-none-any.whl size=427830 sha256=f8dd0350e4bbd15712dc7b37d39081308888de23f12117990ed289595fc9be13
  Stored in directory: /root/.cache/pip/wheels/44/95/70/0dc41f9b4e31b8a7ea22193aad5647b2c85cfab37bf13c0242
  Building wheel for dash-table (setup.py) ... done
  Created wheel for dash-table: filename=dash_table-4.11.2-cp36-none-any.whl size=1839870 sha256=d164847d466bc4fc01a6162506ce077e1e4ee4f8831b83b176c153e5d351f9fe
  Stored in directory: /root/.cache/pip/wheels/72/d9/f6/2ad62ac0037f1f0c0d3d10948a596b594a069057df0656ac3f
Successfully built dash fire dash-renderer dash-core-components dash-html-components dash-table
Installing collected packages: tzlocal, APScheduler, brotli, flask-compress, dash-renderer, dash-core-components, dash-html-components, dash-table, dash, fire, dash-bootstrap-components, pynvml, codecarbon
  Found existing installation: tzlocal 1.5.1
    Uninstalling tzlocal-1.5.1:
      Successfully uninstalled tzlocal-1.5.1
Successfully installed APScheduler-3.7.0 brotli-1.0.9 codecarbon-1.0.0 dash-1.19.0 dash-bootstrap-components-0.11.1 dash-core-components-1.15.0 dash-html-components-1.1.2 dash-renderer-1.9.0 dash-table-4.11.2 fire-0.4.0 flask-compress-1.8.0 pynvml-8.0.4 tzlocal-2.1
---------------------------------------------------------------------------
ContextualVersionConflict                 Traceback (most recent call last)
<ipython-input-2-43bdafe14602> in <module>()
      3 import tensorflow as tf
      4
----> 5 from codecarbon import EmissionsTracker
      6
      7 mnist = tf.keras.datasets.mnist

6 frames
/usr/local/lib/python3.6/dist-packages/codecarbon/__init__.py in <module>()
      3 """
      4
----> 5 from .emissions_tracker import (
      6     EmissionsTracker,
      7     OfflineEmissionsTracker,

/usr/local/lib/python3.6/dist-packages/codecarbon/emissions_tracker.py in <module>()
     13 from typing import Callable, List, Optional
     14
---> 15 from apscheduler.schedulers.background import BackgroundScheduler
     16
     17 from codecarbon.core import cpu, gpu

/usr/local/lib/python3.6/dist-packages/apscheduler/__init__.py in <module>()
      2
      3 try:
----> 4     release = get_distribution('APScheduler').version.split('-')[0]
      5 except DistributionNotFound:
      6     release = '3.5.0'

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in get_distribution(dist)
    464         dist = Requirement.parse(dist)
    465     if isinstance(dist, Requirement):
--> 466         dist = get_provider(dist)
    467     if not isinstance(dist, Distribution):
    468         raise TypeError("Expected string, Requirement, or Distribution", dist)

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in get_provider(moduleOrReq)
    340     """Return an IResourceProvider for the named module or requirement"""
    341     if isinstance(moduleOrReq, Requirement):
--> 342         return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
    343     try:
    344         module = sys.modules[moduleOrReq]

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in require(self, *requirements)
    884         included, even if they were already activated in this working set.
    885         """
--> 886         needed = self.resolve(parse_requirements(requirements))
    887
    888         for dist in needed:

/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in resolve(self, requirements, env, installer, replace_conflicting, extras)
    775                 # Oops, the "best" so far conflicts with a dependency
    776                 dependent_req = required_by[req]
--> 777                 raise VersionConflict(dist, req).with_context(dependent_req)
    778
    779             # push the new requirements onto the stack

ContextualVersionConflict: (tzlocal 1.5.1 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('tzlocal~=2.0'), {'APScheduler'})

If you ever get a ContextualVersionConflict on jupyter notebook or Google Colab, try restarting the Runtime/Kernel. The error means that a new version has been installed of a Python package that was already loaded.

@Lothiraldan closed?

This error persists in multiple colab notebooks.. after colab runs on a fresh instance (new notebook or Runtime > Factory restart runtime) the same ContextualVersionConflict error pops up. My current workaround after this error is to restart kernel after refreshing instance (Runtime > Factory restart & Runtime > Restart and run all) which seems messy.
try:     from codecarbon import EmissionsTracker     from codecarbon import track_emissions except:     !pip install --upgrade codecarbon     !pip install --upgrade tzlocal # also tried with ==2.1 and ==2.0     from codecarbon import EmissionsTracker     from codecarbon import track_emissions

@Lothiraldan in my experience this issue still persists.. Can i contribute anything to solving this?

@grimelda After installing packages on Google Colab (I recommend using %pip install -U codecarbon to do so), you need to restart the Runtime. Google Colab seems to start from a non-empty state, some packages are pre-installed, you can see which ones with %pip list. From this list, it looks like tzlocal is preinstall in version 1.5.1. While installing codecarbon, we install apscheduler which requires a higher version so the version tzlocal-2.1 is installed instead. Then when loading apscheduler, an automatic check happens on the dependencies:
/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py in require(self, *requirements)
    884         included, even if they were already activated in this working set.
    885         """
--> 886         needed = self.resolve(parse_requirements(requirements))

And it looks like that the old version 1.5.1 is still in memory somewhere. After restarting the Runtime, the version on disk is 2.1 so everything works as expected.

I'm trying to get my team to log the use of intensive scripts, and the decorator seems like a great and easy to implement option that i can ask my team to adopt, however I can hardly ask them to restart their kernel every time a script is loaded. A workaround involving I could instruct to os._exit(0) to force a kernel restart but this would be extremely ugly.
This is the kind of workflow that I would expect to work, but doesn't in Colab:
  !pip install -U codecarbon
  from codecarbon import track_emissions

@track_emissions(project_name=f"projectname")
def main():
    print('Something')

Additional attempts to run codecarbon in colab without triggering this warning on first run that have failed:
# to get rid of tzlocal==1.5.1  in memory does not work, prompts y/n
!pip uninstall tzlocal
!pip install -U codecarbon
from codecarbon import track_emissions

# to force reload (and purge previous versions using importlib) does not work
!pip install -U codecarbon
import importlib, tzlocal, pytz
importlib.reload(pytz)
importlib.reload(tzlocal)
from codecarbon import track_emissions

# to get rid of old version tzlocal==1.5.1 initialised by colab does not work
!pip uninstall tzlocal
!pip install -U codecarbon
from codecarbon import track_emissions

# force reinstall of tzlocal==2.0 does not work, triggers warning also:
# WARNING: The following packages were previously imported in this runtime: [pytz] You must restart the runtime in order to use newly installed versions.

!pip install --force-reinstall tzlocal==2.0
!pip install -U codecarbon
from codecarbon import track_emissions

Would this make the kernel re-read files and update the in-memory version of tzlocal?
from importlib import reload
import tzlocal
---
!pip uninstall tzlocal
!pip install --force-reinstall tzlocal==2.0
---
reload(tzlocal)

Tried something similar which didn't work, and updated it to vict0rsch' method, sadly does not work. I saved a bunch of attempts in the gist below for documentation.
My main finding is that the error pops up earlier by the importing of pytz which is a dependency of tzlocal and APScheduler. This goes over my head wrt how to solve this. Is this a colab issue?
WARNING: The following packages were previously imported in this runtime:
  [pytz]
You must restart the runtime in order to use newly installed versions.
\nContextualVersionConflict: (tzlocal 1.5.1 (/usr/local/lib/python3.7/dist-packages), Requirement.parse('tzlocal~=2.0'), {'APScheduler'})\n

https://gist.github.com/grimelda/2f1d1e382b0b3984e95f9f2582848613


Documentation is not switched according to the version

When I switch the documentation of version 0.42.0, it shows me the
https://facebook.github.io/react-native/docs/0.42/using-a-listview.html
which explains SectionList usage.
But the SectionList component is not available for version 0.42.0.

Is this a bug report?
yes
Have you read the Contributing Guidelines?
yes
Environment
Environment:
OS: macOS High Sierra 10.13.1
Packages: (wanted => installed)
Steps to Reproduce

Switch documentation to 0.42.0

Problem is
There has https://facebook.github.io/react-native/docs/0.42/using-a-listview.html
which explains SectionList usage.
But the SectionList component is not available for version 0.42.0.
Expected Result is
Display the ListView in side bar instead of Using a ListView that explains some components do not exist in version 0.42.0


v1.32.45 Redesigned HAL Profile

The HAL profile has been redesigned to allow all AVR-DU families, including 14P, to use ICSP/6P connectors (with VCP functionality) with PDI support.
To achieve this, a 1-gate logic == 3-state buffer is required, and a dedicated control line is used to switch the PDI_CLK output destination.
Similarly, a preliminary design of the HV control expansion board circuitry required for 20P/28P/32P/CNANO has been done, and the GPIOs have been rearranged accordingly. RS232 signal inputs and outputs are no longer included.
Only the CNANO expansion board has FTDI-style DTR/CTS support pins to evaluate the VCP functionality as a generic USB-Serial converter. All other boards only have pure programming tool functionality and two-wire VCP functionality.
Some documentation has been revised in connection with the AVRDUDE 8.0 release.
Additional information
There was confusion about "erase EEPROM page" just before the AVRDUDE 8.0 release, but the provisional code for that has not yet been removed. This issue is still in flux.
By the way, it seems that erasing an EEPROM page can only be properly implemented in AVR-Ex family and PDI type silicon with JTAG interface. (To the extent that this project supports)
In principle, all EEPROMs in the AVR series are based on 1 byte per page, which does not match the definition of an EEPROM page in the standard JTAG commands.
On the contrary, there is no standard JTAG command that corresponds to the AVR series specific memory section erase function. Prior to JTAGmkII, this was done using a mechanism to send extended commands that encapsulated native ISP scripts. (The PICKit5 mechanism took this idea even further)

To support PDI, it needs to run at 3.3V.
This problem is serious in nature and is the main reason why the official debugger is not self-powered, but all externally powered. But this is really inconvenient.
Since Curiosity Nano only works at 5V, I have to install an additional 3V3LDO, add a level shifter and a voltage selector switch. As a result, the number of parts needed is three times higher than originally estimated. Plans to sell a cheap kit with board and parts have been cancelled. Only a semi-finished kit with everything populated except for the sockets is a realistic plan. It's a pity that the CNANO doesn't have a 3.3V selector jumper.
We found that 14P chip can also support PDI type, but it needs to support 3.3V at the same time, so we can't simplify the power domain circuit. The only way to save cost is to design a 5V only board without PDI support, or a 3.3V+PDI only board, otherwise we will miss the advantage of 14P.


Ensure latest causes failures on ubuntu 12.04

TLDR: The post install in 1.0.912 and some earlier packages doesn't handle upgrades if the agent is already running. The default should be change to installed, or the package needs to try restarting the service too.
The /var/lib/dpkg/info/newrelic-infra.postinst is

#!/bin/sh
if [ -e "/etc/init/newrelic-infra.conf" ]; then
initctl start newrelic-infra || exit $?
fi

And http://manpages.ubuntu.com/manpages/precise/man8/initctl.8.html states that:

If the job is already running, start will return an error.

This gives apt errors:

Setting up newrelic-infra (1.0.909) ...
initctl: Job is already running: newrelic-infra
dpkg: error processing newrelic-infra (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
newrelic-infra
E: Sub-process /usr/bin/dpkg returned an error code (1)

Example in the community are seen at the bottom of https://discuss.newrelic.com/t/upgrading-the-agent-fails-to-restart-properly-on-ubuntu/42840/6

Hi, we've submitted a patch to it fixes this issue.
Thanks for reporting it, it will be available in the next version of the agent.


Update formula for frequency response

Hi, this is using the formula I found in this article, which seems consistent with  apple's guidelines. I used it to make this interactive demo.
I wasn't sure whether to add more logic around guarding against invalid values, please let me know if that would be preferred.
(This pr was made in response to this comment by @aleclarson.)

This is perfect, thanks for contributing. I don't remember where I got the original formula. 😅

I wasn't sure whether to add more logic around guarding against invalid values, please let me know if that would be preferred.

Yeah sure why not!


[v1.10] Backport #18486

Backport of https://github.com/cilium/cilium/pull/18486 to v1.10 (cc @joamaki).

/test-backport-1.10

@bmcustodio What's the status on this please? Was it expected to drop the backport? Can we please update the backport-pending/1.10 label accordingly on the original PR?

Apologies, I see this was backported in https://github.com/cilium/cilium/pull/18835 instead, sorry for the noise.


[FEATURE] Add blocked users repository for the specific user

Add the feature for the specific student to have the IEnumerable of blocked users.

Hi, i am interested in resolving this issue.

Hi @Pantex9,
Thank you for your interest in resolving this issue. Currently, we are actively working on updating the microservice APIs and integrating these updates into the Blazor application. This includes implementing the feature for managing blocked users.
If you're still interested in contributing, you could assist by focusing on specific parts of the integration process or testing the implemented features to ensure smooth operation within the Blazor front-end. Please let us know how you would like to contribute, and we can coordinate the tasks.
Looking forward to your responce!
Best regards,
Development Team


General question - Why use Rust instead of Kotlin

Just curious to know why Rust was used to write this tool, or if there were any advantages in mind. Thanks in advance

Hey! thanks for the question. I actuallly received this question when i was defending my thesis. It's a valid question to ask.
There was not well thought out reason for choosing rust. This project stemmed from a software engineering capstone class my last semester of undergrad. My team and i decided to build a tool from the ground up instead of forking an existing tool like PIT. No of us wanted to do Java or Python and i was the only one who knew Kotlin but at the time i was very confident in Rust and suggested it to my team and we ran with it.
Since building it out though i have seen that rust has been incredibly helpful when it came to concurrency. It just made sense to me and was super easy to implement.
Writing tests have been super easy as well within rust and the borrow checker has been a friend here more than an enemy.
Hope this answers your question! I was actually thinking of writing a blog post talking about my experience building this tool out.
@anonymous1654


Backstage Glossary Documentation

Hey, I just made a Pull Request!
This PR adds a Backstage Glossary to the documentation.

:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)

@AlleyWei bringing you in just be sure we're sync'd up on the messaging

@Rugvip I've synced with Alex and Jeff and made changes based on their feedback.

@AlleyWei bringing you in just be sure we're sync'd up on the messaging

@Rugvip I've synced with Alex and Jeff and made changes based on their feedback.


WIP: Refactoring schema generation and annotation application

Fix https://github.com/pydantic/pydantic/issues/9340
In general, working on tasks related to https://github.com/pydantic/pydantic/labels/performance
I currently have a bunch of broken tests because I'm changing the way in which we internally apply constraints to schemas in some cases. Hopefully this approach is both helpful logically and in the performance department.
 I'll also write some benchmarks to explain which cases this helps in terms of schema build time.
Also TODO: make some edits to this PR to make it more easily reviewable :)

Having major doubts about this approach given https://github.com/pydantic/pydantic/pull/5737

I'm going to go ahead and close this, as I'm going to cherry pick a bunch of changes into separate PRs and not implement the most major changes here.


Any plans for updating to angular 6?

Hi, I've updated my project to the latest release candidate of Angular 6 and Angular cli 6 but this breaking change: https://github.com/angular/angular-cli/issues/9827#issuecomment-369578814
is causing AWS Cognito library to fail

The problem with net/global is not so straightforward. We are removing it for 6.x, yes (#9812). And we do expect this to cause problems to some projects. But we're not doing it because we want to break projects, we're doing it because leaving it in also breaks other projects, and it is incorrect.

The incorrect part is libraries that are meant to run in the browser relying on node globals being available. Browser code runs in the browser, not in node, and shouldn't expect things that are not available in a browser context to be there.

So when you serve an Angular 6 project with AWS Cognito you are getting an error "Uncaught ReferenceError: global is not defined" in Buffer.js

Closign in favor of https://github.com/aws/aws-amplify/issues/678


update PyPI package to 0.8.0

The package version on PyPI is still 0.7.0.
Is there any chance this will be updated?
Thank you

Greetings!
The new version will be published on PyPI soon.


[for review]layers: use std mutex

Mark and I would like to move to std::mutex. Is this okay? If so, I'll move all the layers to this method.

As I've said before, I'm very in favor of doing this in general and would like to see this used everywhere.

Sounds good to me. I'm fine with global_lock. I'll work on updating all the
layers.
On Fri, Apr 8, 2016 at 10:24 AM, Michael Lentine<EMAIL_ADDRESS>wrote:

As I've said before, I'm very in favor of doing this in general and would
like to see this used everywhere.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/KhronosGroup/Vulkan-LoaderAndValidationLayers/pull/300#issuecomment-207501392

I agree with previous comments:

Kill "using std"
rename "mutex"->"global_lock"

Looks good to push with those changes. Just assign back to me if you want me to merge it.


Fixed handling of defaults and nodefaults in channel lists

Fix for the issue described in  https://github.com/Anaconda-Platform/anaconda-project/pull/387 and https://github.com/Anaconda-Platform/anaconda-project/issues/388.
In short:

The defaults channel was implicitly picked up from the .condarc
nodefaults was ignored by anaconda-project (though it is respected by conda itself)
The latest anaconda-project version now (correctly) ignores the ~/.condarc which has implications for backwards compatibility.

This PR attempts to fix this issue as follows:

Make defaults explicit everywhere by appending it to the channel list if it is not stated. This should be backwards and forwards compatible with anaconda-project.
Remove any instances of nodefaults, making sure defaults appears at the end of the list and noting that the intention was to avoid using defaults in a comment.

I believe this should fix compatibility but this PR has the possibility of breaking lots of things so please make sure you agree with this strategy @jbednar @maximlt !

Note: A separate PR should go through the projects trying to use nodefaults with the dev version of anaconda-project that actually respects it, to see if those projects actually function without defaults.
Also, I believe these changes shouldn't require relocking everything but I will have to double check.

Tests currently failing due to carbon flux not having test data. I'll have a look into this shortly.

I totally approve this approach.
Note that you won't be able to build many examples, even before anaconda-project 0.11 they were mostly all broken. I have to resume the work on fixing that.

Tests currently failing due to carbon flux not having test data. I'll have a look into this shortly.

That sounds strange. For reference here's a table that sums up what data the projects need/use: https://github.com/pyviz-topics/examples/issues/202#issuecomment-1166689039
Also and seems to be of importance, anaconda-project is pinned to an older version in the ecosystem_setup command:
https://github.com/pyviz-topics/examples/blob/10772f10bc5e8dfd7c3bd8a894d372135176b39a/dodo.py#L17-L23

... Also and seems to be of importance, anaconda-project is pinned to an older version in the ecosystem_setup command

Well spotted!
That said, this pin is for the infrastructure here, this PR as a whole is for users who may install any version of anaconda-project they want to try running these projects locally. Knowing the anaconda-project version is useful though, to debug any issues cropping up due to channel behavior.

Also, I believe these changes shouldn't require relocking everything but I will have to double check.

I believe projects don't need their lock to be updated to run, but I also believe users will get a warning. I think for now it's fine and will go ahead and merge. Thanks Jean-Luc!


[MWPW-158471] Add phone number feature for columns block

Fixes following issues|Adds following features:

Issue/Feature 1
Adds phone number link to columns block.

Resolves: https://jira.corp.adobe.com/projects/MWPW/issues/MWPW-158471
Steps to test the before vs. after and expectations:

Steps for feature 1
Visit the first link to verify that the phone number shows up and no significant misalignment occurs.
Visit the other links to verify no regression in the columns block occured.

Authoring Note

Pages to check for regression and performance:

https://columns-phone-number--express--adobecom.hlx.page/drafts/echen/why-choose-express-copy
https://columns-phone-number--express--adobecom.hlx.page/br/express/create/brochure
https://columns-phone-number--express--adobecom.hlx.page/express/create/banner/graphic
https://columns-phone-number--express--adobecom.hlx.page/express/feature/image/convert/png-to-jpg
https://columns-phone-number--express--adobecom.hlx.page/express/create/logo/t-shirt
https://columns-phone-number--express--adobecom.hlx.page/express/create/poster/classroom
https://columns-phone-number--express--adobecom.hlx.page/express/discover/quotes/success
https://columns-phone-number--express--adobecom.hlx.page/express/discover/quotes/motivational
https://columns-phone-number--express--adobecom.hlx.page/express/create/video/marketing
https://columns-phone-number--express--adobecom.hlx.page/express/create/menu

from the ticket it seems like they want the color of that new CTA to stay dark?

from the ticket it seems like they want the color of that new CTA to stay dark?

Amanda didn't seem to mind, but I changed it to black just in case.


fix: add ebl fields

add placeOfReceipt and placeOfDelivery fields

:tada: This PR is included in version 1.15.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


How often does the "random" resolver change?

Hello, glad you finally add the "random resolver" function. I want to know how often does it choose a new random resolver - only on start, or every x minutes, or random on each new query? Thank you.

Only on start.
Changing frequently is a really bad idea IMHO.


extract kube proxy images information into json file

In order to better manage the images version information, and support automatically patch image versions, extract kube proxy images information into separate files.

/lgtm

Overall I like this change, but why are we creating separate file instead of adding these to the https://github.com/Azure/AgentBaker/blob/master/vhdbuilder/packer/components.json file?

the reason why Zuo put it into a separate file is because if we put it to the components.json.
then I think all the images will be pulled?


Dumped JSON file does not conform with JSON standard which leads to decoding problems

Dumped JSON file, contains encoded strings, which could error prone while decoding in scripting languages like Python.
These encoded strings, especially those in the name table, are directly dumped as bytes, and are not always uniformly encoded because they are stored encoded in fonts as bytes when the corresponding platform is Windows.
JSON decoding, for example, in Python 3, the json.load() or json.loads() accept str instead of bytes, and when we try to decode the bytes, the problem would occurs because the JSON file could contain mixed encoded bytes. The same could also happen when we consider the JSON generation problem in Python 3. Most third party JSON parsing packages in Python faces the same problem. And many C/C++ JSON parsing library also assume the JSON should use Unicode encoding (RapidJSON, for example). But there are ShiftJIS and GBK in these dumped bytes for many Chinese and Japanese fonts. Decisions made by these software is reasonable because according to RFC-7159, section 8.1,

JSON text SHALL be encoded in UTF-8, UTF-16, or UTF-32.  The default encoding is UTF-8, and JSON texts that are encoded in UTF-8 are interoperable in the sense that they will be read successfully by the
maximum number of implementations; there are many implementations that cannot successfully read texts in other encodings (such as UTF-16 and UTF-32).

Similar articles appear in ECMA-404 as well.
I definitely can write a JSON parsing package based on YAJL's tokenizer and its corresponding JSON parser, which uses bytes as its sole data exchanging type even in Python 3, and I am actually working on one. However, I am hoping there could be a more elegant solution. For example, when dump JSON, use base64 to do an additional encoding, or decode them and encode to Unicode before dump. Or provide a manipulate API for other language is also preferable.
Hope these information is self-contained and can help you understand the problem.

@chazeon Could you please provide some examples?
Currently Windows NAME entries are encoded in UTF-8, but for other entries it would use BASE64, since they may contain legacy encoding.

@be5invis I don't know if you can get hold of any Founder Type font, I get one from their official website for example, FZXSSK (方正新书宋_GBK) has this kind of problem.
I dump the FZXSSK.TTF to FZXSSK.json using
 .\otfccdump.exe .\FZXSSK.TTF -o .\FZXSSK.json

with otfcc release 0.9.6 then run Python 3 and get problems like these
>>> import json
>>> with open('FZXSSK.json', 'r') as f:
...     json.load(f)
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "C:\ProgramData\Anaconda3\lib\json\__init__.py", line 296, in load
    return loads(fp.read(),
UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 5445: illegal multibyte sequence
>>> with open('FZXSSK.json', 'r', encoding='utf8') as f:
...     json.load(f)
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "C:\ProgramData\Anaconda3\lib\json\__init__.py", line 296, in load
    return loads(fp.read(),
  File "C:\ProgramData\Anaconda3\lib\codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb1 in position 3718: invalid start byte
>>>

I do not dig in this time, but from my previous experience, the copyright line at least, could cause the problem.

They are encoding CJK characters in Mac Roman encoding?

I realized I have mistaken the platform Mac for Windows. I am not familiar with Mac Roman encoding, but it seems they are doing something like that.


[feature request] delta upload of locally modified files

Let's say I have a 1000MB file locally and I push it up to my Google drive. Let's say the 'drive push' command takes 100 seconds to complete (I guess this is roughly the upload time). Now let's say I locally modify 100MB in that 1000MB file and run 'drive push' again. I expect this operation take 10 seconds to complete. My testing seems to show that this second push takes 100 seconds to complete.
Is my expectation unreasonable?
Can a delta upload feature be added easily?

That's going to take a while until I incorporate change tracking [which I already started working on https://github.com/odeke-em/drive/tree/change-tracking]. This is because Google Drive doesn't directly expose the ability to check the integrity of chunks but there is away around this by tracking changes. Please be patient, or if you have any suggestions feel free to provide them.

After doing a bit of reading, it looks like Google Drive doesn't support they type of delta sync I'm asking about here. Pity.
It seems like this is a feature request for Google rather than for this project at this point. I'm going to close it.
My gut tells me that effort to implement this feature in drive without proper API support from Google will be too much of a hack and is likely wasted effort :-(

Yes in deed it is a very tricky feature since as previously mentioned Google Drive doesn't support it, yet.

FYI, http://librsync.sourcefrog.net/doc/rdiff.html

That is pretty cool, you deserve a cookie @kcwu! Surely something to take a look at.

We cannot use it directly since we cannot control server side. I guess some ideas still could be reused.

Oh yes the ideas from this are useful. Since we are already talking about ideas: my ideas for solving this would involve constructing merkle trees or tiger hash trees and then comparing these from past revisions and figuring out what chunks changed. However I have a way simpler idea but it will require Google Drive support for which I have to go through discussions and proposals that might not be accepted but we shall see.

Delta sync would be a really great feature. @odeke-em has there been any progress on this?

@greyltc nope, no progress unfortunately has been made on my part here.


Using graal as a multi-threading

Hello, I'm replacing java's script engine with graal from Nashorn.
However, unlike Nashorn, graal does not support multi-threading, so the following methods were used
To briefly explain my code, :

When initializing the server, create a specific number of script engines and add them to the queue

Context newEngine = Context.newBuilder("js").option("engine.WarnInterpreterOnly", "false").allowAllAccess(true).allowHostClassLookup(className -> true).build();

ScriptEngineQueue.getInstance().add(newEngine );

Then poll the script engine and evaluate the script in Java.

Context engine = ScriptEngineQueue.getInstance().poll();
engine.eval("js", script);

Add the evaluation script engine back to the queue.

ScriptEngineQueue.getInstance().add(engine);
It takes up quite a lot of resources to create an engine, so I've created it in advance.
About 200 script engines are generated, and memory leaks are suspected in this situation, so I would like to ask if there is any problem in using them with the above usage.
Please reply. Thank you.

You need to use something like vertx to be more clear this is what you want:
@oubidar-Abderrahim @iamstolis @kingkong3
https://reactiverse.io/es4x/
the es4x java package is a JS Context Factory linking each context with a vertx eventbus as IPC Replacement.

I'm also interested in this pooling design. As GraalJS GC is integrated with the JVM GC, does a GC of a specific ScriptEngine generate impacts to the rest of engines in the pool?

Stop the world GC
Actively force GC in other engines
...

@caoccao The JS Engine has no real own GC Everything is the JVM GC if you tune that it will have effects on the JS GC only when you use graal-node then you will use libuv and the NodeJS GC. GC is nothing that is JS Specific it comes always from the Engine that Embeddeds JS. JS is always only the single context everything else comes from the outside.
And you can even avoid GC in JS If you simple do not assign as much.

Thank @lemanschik for the detailed explanation.
That confirms my concern: the script engines in GraalJS are not completely isolated as they share the same GC. In some critical situations, the GC intensive script engines might impact the other CPU (real-time) intensive script engines.
There are other solutions which have V8 embedding in JVM with standalone GC. That allows fine-grain controls over the individual V8 instances to avoid the case above.
Hope this makes sense. I'm not here to discredit GraalJS. I'm just curious in the tech detail and the implications behind the high level design.

@caoccao your going highly theoretical here i could now explain in detail while this will have no effect for you but we make it short simple code and benchmark and you will see there are no issues with that. You did read a few papers or docs that did confuse you a bit. I am Chromium Platform Architectural Engineer. I have knowledge about the JVM and its internals V8 and the Internals as also Many other Engines Systems (CPU/RAM/BUS) things. In Theory your correct but the engine authors did take care of such things.
in Real world nothing will be unresponsive or something else measurable will have impact to your software

@caoccao if you want to mix Software GraalVM is great for that and if you thought that your code runs in the engine no matter which name then your wrong. When you compile a GraalVM Project via Substrate VM to binary you get ASM so not your code anymore its highly optimized.
Also in the browser the result of your JS is Assembler it does not feel and look like that but that's what happens.

Well, now the topic seems to move to AOT. I'm fine with everything changed/optimized in AOT, but JS interoperability is lost in AOT. In quite a lot of scenarios, JS interoperability is the key feature that blocks folks from moving towards AOT. Please correct me if I'm wrong.

@caoccao your partial wrong you did forget the fact that you can compile the whole JS Engine AOT you see a example with graal-node. And even if you do not use AOT you can fine tune the JVM with over 1k parameters but you should dig into that when the time and need did come up and that will not happen.

I'm confused about the definition of right or wrong. I'm here doubting the unified GC vs. isolated GC. I'm told unified GC could achieve all what the isolated GC could do with 0 side effects. If I disagree with that, I'm wrong. Hope that's not what is happening here.

@caoccao it is because your not aware of GC Implementation details for example GraalVM ships with multiple once that you can switch to be more exact the JVM in this case (HotSpot) does so and there are even GC less JVM's but as said before you will understand all that once it comes to real life.
In Real life languages like Rust also have some kind of GC and they also stop the world thats because the OS Does so. But in rust or other languages you would need to tune the OS. the JVM is its own OS. hope that gives you some insights.
in real world when you would see GC issues you could move the GC operation or avoid it or tune it. And GC is Implemented per OS Process. so if you eg: run 2 Processes they got indipendent GC rhythms.

@caoccao your touching here the topic "Realtime Computation" and for that you would need a special operating system and special hardware. In all other Modern Operating Systems you get something that you can call instant computation because scheduling is managed for you via a
FIFO Algo First in First Out.

@lemanschik I wonder what makes saying the 2 types of GC have their PROS and CONS so hard. There's a guy who told me they are the same and I'm wrong. I think I will quit this discussion.

@caoccao nothing has universal pro's and cons you need to know your payloads and workloads after that you could in theory say what would be a better fit until a external contributor delivers code that makes that assumption obsolete.
The only consistent way to handle GC and even none GC is to measure and then take actions.
eg: adjusting some of the JVM start parameters, compiler options, or do code refactoring.


QUESTION: Is it possible to map InlineFragments in v0.8.0?

Hi!
Great repo, thanks for sharing!  This might be a shot in the dark as my question pertains to an old version of AutoGraph, so feel free to say you can't help.
I'm using v0.8.0 (we utilize Realm integration so aren't in a position to upgrade just yet to 0.9.0) and am trying to figure out the proper way to set up mappingKeys on my Request object for inline fragments.  I am able to build and send my request fine with InlineFragment objects but I'm failing to come up with what feels like the correct way to map/parse the response.  Issue #107 came close to answering my question, but I'm wondering if there's a more proper way of handling the mapping (via mappingKeys) instead of doing it all manually from the raw json response?
Example query:
query getFeed {
    id
    blocks {
      __typename
      id
      ... on BlockTypeOne {
        title
      }
      ... on BlockTypeTwo {
        message
        userName
      }
    }
}

My attempt at mappingKeys:
var mappingKeys: Set<FeedKey> = [
  .id,
  .blocks([
      .typename,
      .id
      .title,
      .message,
      .username
    })
]

Since the 2 inline fragments have different fields, I'm trying to figure out a "correct" way to set up my mappingKeys.  Using the above example, if I try to include title, message and userName at all times in the Request object's mappingKeys, I end up with a Crust.MappingError.noDataAtKeyPathFromKeyToMapFrom(keyPath:key:) error in Crust as all values are not present on all response objects.  Again, I realize my question involves a legacy version that includes old dependencies, but I thought I'd ask.  At worst, I can probably just take the root json response and map everything the old fashioned way by hand.
Thanks again for your time and the great repo!

 If what you want is a field to not be mapped to because the value is not there, even though a key exists for it, that's a semantic contradiction.
In your Mapping implementation you'll need to do the __typename checking yourself then for value existence and then map the appropriate keys. I've debated adding a IgnoreMissingValueMappingKey but I think it sends the wrong signal so decided not to go down that route long ago.
The additional option, which I think is the better one, is to structure your code so that BlockTypeOne and BlockTypeTwo are two separate substructures of the blocks object, with their own mappings. You still have to check on __typename but it's clearer from data shape as to why. This leans in more naturally to the impedance mismatch between Swift and GraphQL.

Thank you so much for your detailed response.  From the sound of it, I'm going to have to do the majority of the mapping manually based on the __typename check.  I failed to mention that my blocks object is actually an array, which is populated by different object types ("different" insofar as they are denoted differently by the value of their respective __typename) each with their own set of keys.  Thus, it sounds like your second proposed option of using substructures is out.
I'm going to close this with the assumption I indeed am going to have to parse/map everything the long way by hand directly from the JSON.  It's what I was afraid of originally, but wasn't 100% sure if maybe I'd missed something when setting up my Request and Mapping objects if there were some way to conditionally map keys from the response.  If I come up with a concise way of doing it, I'll be certain to report back here for reference.
Thanks again for taking the time to respond, I really appreciate it!


[sonarqube]: multiple replicas

I am trying to set replicaCount: 2 in the values.yaml file, but as soon as the replica set is ready and I try to log on SonarQube, it simply return the log-in screen again.
Is there any additional option to set to have multiple replicas?
I have tried the following:
jvmOpts: "-Djgroups.dns.query=sonarqube-sonarqube.<NAMESPACE>:9000 -Djava.net.preferIPv4Stack=true"
where <NAMESPACE> is the namespace where SonarQube is deployed, but unfortunately it did not help.
Does actually SonarQube Community support multiple replicas? Thanks in advance.

Hello @vittoriocanilli ,
We had that issue also, and was able to surpass it by changing the ingress controller to nginx. We found out that we needed to have the cookies options also with affinity for the login to be persistent.

Hello @ricardojdsilva87 ,
thanks for your suggestion: I have indeed added these Nginx annotations to my ingress:
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/affinity-mode: balanced

and I confirm that it works, but that makes all requests going to the same pod and the session is lost in case the pod is deleted: the pods run in this configuration as stand-alone, and not as in a cluster.

In our case we don't mind that the session is lost after the pod is recreated, the user just needs to relogin. For nginx I'm not sure if there is any option to store the session across multiple pods. The cookie must need to have probably some kind of different options needed to pass trough the configuration others than the default ones to have the behaviour you've mentioned. Not sure which ones.
The behaviour we intended to have was to redeploy at any time sonarqube and the user won't notice. because it's a Rolling-update deployment. The only issue the user might find is having to relogin but as we have the database centralized the sonarqube pod is used only as a frontend, the changes would be saved anyway after relogin.

Hi,
is it possible to have Sonarqube configured as a cluster (HA mode) ? thanks

btw: i set replicaCount: 2 and was not able to get 2nd up...anything else I might be missing? @ricardojdsilva87 @vittoriocanilli
2020.09.15 02:33:04 WARN  es[][o.e.b.ElasticsearchUncaughtExceptionHandler] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/opt/sonarqube/data/es6]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])? 	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.8.4.jar:6.8.4] 	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.8.4.jar:6.8.4] 	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.8.4.jar:6.8.4] Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/opt/sonarqube/data/es6]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])? 	at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:300) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.node.Node.<init>(Node.java:296) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.4.jar:6.8.4] 	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.4.jar:6.8.4]

Hello @vittoriocanilli , we have the pods running in different nodes 1 per node not sure if that we didn't encounter that issue. Running in 3 AZs. We also are using an external database common to all 3 pods. For all of the pods we have also not set a PV, we are running them with ephemeral storage, we saw that it was not needed because all the settings are saved in the RDS and we can set most of the things in the helm chart, plugins included.

Yes. It probably requires sticky session. This is why we need to relogin because there's no session on that random server.

Hello and sorry for the late reply!
@ricardojdsilva87 @stevefan1999-personal I confirm that the solution with the sticky sessions works fine for me as I have as well a common database behind the two pods.
@peppe77 did you manage to solve your problem meanwhile? did you have it even with 1 replica? As far as I remember I had problems with ElasticSearch at the beginning even with one replica, so I have disabled it with these values:
elasticsearch:
  configureNode: false
  bootstrapChecks: false
initSysctl:
  enabled: false
  securityContext:
    privileged: false

I did not research further how to setup SonarQube as cluster: not sure if somebody else in this thread had success meanwhile.
I hope it helps.

Hello and sorry for the late reply!
@ricardojdsilva87 @stevefan1999-personal I confirm that the solution with the sticky sessions works fine for me as I have as well a common database behind the two pods.
@peppe77 did you manage to solve your problem meanwhile? did you have it even with 1 replica? As far as I remember I had problems with ElasticSearch at the beginning even with one replica, so I have disabled it with these values:
elasticsearch:
  configureNode: false
  bootstrapChecks: false
initSysctl:
  enabled: false
  securityContext:
    privileged: false

I did not research further how to setup SonarQube as cluster: not sure if somebody else in this thread had success meanwhile.
I hope it helps.

Same problem here ... When I set the replicaCount to 2, the second pod crash.

Not sure if this is related, but we have the replicas in separated nodes since we have our setup in HA.
Also have the following affinity setting to prevent 2 pods to be scheduled to the same node:
affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
    - labelSelector:
        matchExpressions:
        - key: app
          operator: In
          values:
          - sonarqube
      topologyKey: "kubernetes.io/hostname"

Not sure if this is the different for your setup

@vittoriocanilli I was able to get it up and running with 1 replica though have not yet found a way to get multiple replicas working :( @ricardojdsilva87 : how did you handle pv/pvcs or you do not have persistence turned on? we have 1 replica bound to pvc/pv though if we add 2 more replicas (let's say one per region) then persistence becomes a challenge. any thoughts?

Hello @peppe77
To have multiple replicas I have disabled the persistence option. The pods spin up with ephemeral storage, you can configure basically every option of sonarqube via the helm chart.
To have the option to remove the persistent storage you'll need to have an external database in place, we are using an postgresql RDS with the multi AZ option. But you can have the same RDS with only 1 single AZ, also you can run multiple pods in the same node since they don't need to share the same information on a PV, all the projects should be saved on the RDS itself.
The reason for us to have the affinity option for the pods is to enable HA, having multiple pods running in different nodes in different AZs.
Hope this explanation helps you

@vittoriocanilli I was able to get it up and running with 1 replica though have not yet found a way to get multiple replicas working :(

@peppe77 sorry for the late reply; I can have 2 replicas running on my AKS cluster and I use an Azure-hosted PostgreSQL (Basic, 2 vCores, 20 GB storage) as database; the persistent storage is disable in the Helm chart. I hope it helps (maybe you have already solved the issue meanwhile).

@vittoriocanilli I am seeing issues where the result is not saved to the DB sometimes and this seems to be a problem when you use the same db for all replicas. Have you experienced this?

@ArjonBu I use a database and this morning I have noticed your same error in my logs:
2022.02.07 08:07:51 INFO ce[XXXXX][o.s.c.t.s.ComputationStepExecutor] Trigger refresh of Portfolios and Applications | status=SUCCESS | time=0ms
2022.02.07 08:07:51 INFO ce[XXXXX][o.s.c.t.p.a.p.PostProjectAnalysisTasksExecutor] Webhooks | globalWebhooks=0 | projectWebhooks=0 | status=SUCCESS | time=24ms
2022.02.07 08:07:52 ERROR ce[XXXXX][o.s.c.t.CeWorkerImpl] Failed to finalize task with uuid 'XXXXX' and persist its state to db
java.lang.IllegalStateException: Task does not exist anymore: CeTask{type=REPORT, uuid=XXXXX, component=Component{uuid='YYYYY', key='com.company.project:project-name', name='project-name'}, mainComponent=Component{uuid='YYYYY', key='com.company.project:project-name', name='project-name'}, submitter=User{uuid='project-name', login='project-name'}}
at org.sonar.ce.queue.InternalCeQueueImpl.lambda$remove$0(InternalCeQueueImpl.java:123)
at java.base/java.util.Optional.orElseThrow(Optional.java:408)
at org.sonar.ce.queue.InternalCeQueueImpl.remove(InternalCeQueueImpl.java:123)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.finalizeTask(CeWorkerImpl.java:245)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.afterExecute(CeWorkerImpl.java:239)
at org.sonar.ce.taskprocessor.CeWorkerImpl$ExecuteTask.close(CeWorkerImpl.java:199)
at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:161)
at org.sonar.ce.taskprocessor.CeWorkerImpl$TrackRunningState.get(CeWorkerImpl.java:135)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:87)
at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:53)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022.02.07 08:07:52 INFO ce[XXXXX][o.s.c.t.CeWorkerImpl] Executed task | project=com.company.project:project-name | type=REPORT | id=XXXXX | submitter=project-name | status=SUCCESS | time=112231ms

Anyway I can find the task with that ID as background task of the project shown in the SonarQube GUI with CANCELED as status. My colleagues using SonarQube have never complained of missing data so far.


[Bug] SaveMode.Overwrite failed write data to doris

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
spark-doris-connector-3.2_2.12:1.3.2
What's Wrong?
When use SaveMode.Overwrite to write dataframe, 'No suitable driver found for jdbc'  exception  occured.

What You Expected?
success write data to doris by SaveMode.Overwrite
How to Reproduce?
No response
Anything Else?
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

This is due to missing JDBC driver package.
You need to place the mysql jdbc driver package in the spark classpath, or set the --jars option to specify the corresponding jar when executing the spark-submit command.


Adopt HttpClientFactory

Summary
In v5 we can move ahead and add full support for HttpClientFactory. This issue involves analyzing the best way of adding this to the SDK.

We will not address this at the moment


Configure branch protection for release notes branch, allow committers/members to merge PRs

The new method of pushing release notes to dedicated branches prevents non-owners from merging PRs. I think this can be fixed by adding users/teams to the branch protection configuration.
https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches#restrict-who-can-push-to-matching-branches
We should at least add all committers, probably all members.

I’ve removed the relevant branch protection (which only allowed owners to merge).


Validate checksum of downloaded image

YTL is adding sha256 checksums for the images.
They are implementing checksum validation to Naksu (digabi/naksu#57), see their PR for their implementation.

Hmm, doesn't etcher have hash validation?

Yes, it may have.
On Mon 9. Aug 2021 at 13.47, Developer From Jokela @.***>
wrote:

Hmm, doesn't etcher have hash validation?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Testausserveri/abitikku/issues/1#issuecomment-895123923,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ALDCUSUJJ6EFJHBODRJX72TT36W4BANCNFSM5BZVKVGQ
.

Seems like hash exists only for server image.
I feel no reason to validate any hashes (besides etcher's existing mechanism) if client image doesn't have them.
When they'll provide the hash, we can think about it. For now, notfix.


Error when updating job

Error: status: 400, body: {"status":{"code":400,"is_success":false,"user_message":"The request was invalid. Please double check the provided data and try again.","developer_message":""},"data":{"schedule":{"time":{"interval":"extra keys not allowed","hours":"required key not provided"}}}}

Though error isn't reproducible in curl command, may need to log request to debug

the way schedules behave is just mental, Interval is a required key even if not used but Hours is not, but they can't appear in the same request and Cron behaves similar to Hours

Comes down to not having a default for hours, for some reason DBT Cloud requires it to be [9] or similar even if not using a schedule (like the default in the UI)

Fixed.


Package is incompatible with current setuptools releases

The durationpy package (version 0.7) can't be installed due to an incompatibility with recent versions of the setuptools package. When running pip3 install -U durationpy on a python3 instance that makes use of setuptools versions 71.0.0 and later the installation fails:
Collecting durationpy
  Using cached durationpy-0.7.tar.gz (3.2 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [53 lines of output]
      /usr/local/lib/python3.9/site-packages/setuptools/dist.py:458: SetuptoolsDeprecationWarning: Invalid dash-separated options
      !!

              ********************************************************************************
              Usage of dash-separated 'description-file' will not be supported in future
              versions. Please use the underscore name 'description_file' instead.

              This deprecation is overdue, please update your project and remove deprecated
              calls to avoid build errors in the future.

              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************

      !!
        opt = self.warn_dash_deprecation(opt, section)
      running egg_info
      creating /tmp/pip-pip-egg-info-edc6fbr8/durationpy.egg-info
      writing /tmp/pip-pip-egg-info-edc6fbr8/durationpy.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-pip-egg-info-edc6fbr8/durationpy.egg-info/dependency_links.txt
      writing top-level names to /tmp/pip-pip-egg-info-edc6fbr8/durationpy.egg-info/top_level.txt
      writing manifest file '/tmp/pip-pip-egg-info-edc6fbr8/durationpy.egg-info/SOURCES.txt'
      reading manifest file '/tmp/pip-pip-egg-info-edc6fbr8/durationpy.egg-info/SOURCES.txt'
      adding license file 'LICENSE'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-n0jlbnp0/durationpy_a7d027a4f8de4830a6154a389911846b/setup.py", line 3, in <module>
          setup(
        File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 108, in setup
          return distutils.core.setup(**attrs)
        File "/usr/local/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
        File "/usr/local/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/usr/local/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 970, in run_commands
          self.run_command(cmd)
        File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 956, in run_command
          super().run_command(command)
        File "/usr/local/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 989, in run_command
          cmd_obj.run()
        File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 310, in run
          self.find_sources()
        File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 318, in find_sources
          mm.run()
        File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 544, in run
          self.prune_file_list()
        File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 610, in prune_file_list
          base_dir = self.distribution.get_fullname()
        File "/usr/local/lib/python3.9/site-packages/setuptools/_core_metadata.py", line 266, in get_fullname
          return _distribution_fullname(self.get_name(), self.get_version())
        File "/usr/local/lib/python3.9/site-packages/setuptools/_core_metadata.py", line 284, in _distribution_fullname
          canonicalize_version(version, strip_trailing_zero=False),
      TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

This seems to be caused by description-file in setup.cfg.

Fix is published in 0.8


How to work with domains that return string instead of numbers?

With Serenity it is possible to work with an enum or some other type of property that returns an Enum with string values? Example:
using Serenity.ComponentModel;
using System.ComponentModel;
namespace MovieTutorial.MovieDB
{
[EnumKey("MovieDB.MovieKind")]
public enum MovieKind
{
[Description("Film")]
Film = "F",
[Description("TV Series")]
TvSeries = "J",
[Description("Mini Series")]
MiniSeries = "M"
}
}

See select with hardcoded values sample

Thank you Vokan, worked properly.


Better Pose

The recently updated Huggingface demo incorporates several pose controlnets in lieu of its face keypoint detector. I was wondering if this could be implemented here as well, or is that demo just applying the face keypoint controlnet behind the scenes?

The keypoints are added to an additional controlnet (any controlnet really) as you can see from here:
https://github.com/InstantID/InstantID/blob/main/infer_full.py#L112
This is already doable with the current implementation in ComfyUI.

Nice results !
It could be useful to add that workflow to the examples folder, as well.

I will thanks


Issue with "IP Allow List API" in @site/api/cloud/cspm/ipallowlist.mdx

Describe the problem

Created a network list called "example-dev"

:warning: the cidrs field is not returned in the response

Added cidr "<IP_ADDRESS>/32" to the list "example-dev"
Created a new network list called "example-prod"
Added cidr "<IP_ADDRESS>/32" to the list "example-prod"

:warning: 400 returned, adding any other public IP works as expected. I am left to assume that CIDRs must be unique across network lists but this isn't expected since each network list and cidr object has a uuid.

Suggested fix

Either update the docs to reflect that CIDRs must be globally unique or allow for the same CIDRs to be added across different network lists.
Update the docs to reflect that the cidrs field is not returned in the create network list API return

Thanks for this feedback. I verified the use case and updated the docs for Add Network and Add CIDR to:

Reflect that  the response object for Add Network doesn't include CIDR blocks
CIDR blocks cannot overlap other CIDR blocks in your networks.


TypeScript removing comments where it shouldn't

comment.ts
if (false) {
    //I will be removed
}
//I will be removed
else {
    //I will not be removed
    var tmp;
    //I will be removed
}

try{
    //I will not be removed
    var tmp;
    //I will be removed
}
//I will be removed
catch(e){
    //I will not be removed
    var tmp;
    //I will be removed
}

comment.js
if (false) {
}
else {
    //I will not be removed
    var tmp;
}
try {
    //I will not be removed
    var tmp;
}
catch (e) {
    //I will not be removed
    var tmp;
}

Are there any additional flags that can be set for TS to keep all comments?
Same issue from CodePlex

I have "Keep comments in JavaScript output" checked but my header comment, at the top of my TS is being ignored. Really, I only want to keep comments in the /** Comment */ format for documentation purposes, and this is mostly working. The header comment is something like this:
/** Whatever JavaScript API v<IP_ADDRESS> */
module whatever {
    export class API { ... }
}

The comment on the first line is excluded meanwhile other comments in the file are kept.

@JoshMcCullough Block comments are also excluded at those locations (end of if/try statements).
I know comments at the top of file are kept, but this is not what I need.

No, I wasn't suggesting this solves your problem, I was suggesting that comments at the top of my TS file are being excluded from the JS. E.g., we're in the same boat - our comments are being stripped!

+1

PRs are welcomed

TypeScript also appears to be removing comments in this fashion:
ts:
const foo: boolean = var1 || /* istanbul ignore next */ null;
js:
var foo = var1 || null;
This is painful when trying to get code coverage and are setting ignore statements.

3rd party hints within code? :-/

There are other places where it's been removed, I created this one last week without knowing it was a dup
https://github.com/Microsoft/TypeScript/issues/13602

Still does not work

This is a big issue for us as well. We are trying to use jsDoc with Typescript, and Typescript's habit of removing comments, even with removeComments=false, is making this very painful. Please provide some way to keep all comments in emitted JS. In our case, I am trying to document interfaces and other things that do not emit JS code, and this makes it very difficult.

Comments on interfaces will not be emitted, since the interfaces themselves are not emited.
For other issues please be specific on what declarations are not getting their comments emitted properly.

Yes, I understand the rationale for removing the comments before interfaces. In many cases it is correct and understandable. But what I am saying is that there are significant reasons to include an option to not remove any comments, including those before interfaces. In fact, there is one very significant reason and use case - documentation with tools like jsDoc. Think of why we use interfaces. They explicitly define the shape of objects for clients of a package. This functionality is a major advantage of using Typescript. Having no option to allow comments about these structures to flow through to documentation is a major failing, in my view. Why would adding a compiler option to suppress all comment removal be a problem? It would be up to the author to use it correctly.

I would like to add another to this list. Let me know if it deserves a separate issue.
TypeScript removes top level comments when a file contains a shebang.
We have a company standard that requires copyright statements be added to all of our ts/js files.
Given the file below:
#! /usr/bin/env node
/**
 * @copyright <copyright goes here>
 */
export const foo: string = "bar";

The emitted output is:
#! /usr/bin/env node
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.foo = "bar";
//# sourceMappingURL=test.js.map

I would like to add few more cases. All comments in the example below are removed when compiling the code:
var a;
a
  // comment
  .b()

if (/*comment*/1) {
}

a(/*comment*/)

if /*comment*/(1) {

}
else /*comment*/{}

Also removes
for (const movieFile of fm.movieFiles) { // comment


CssProvider::load_from_file has the wrong signature

https://github.com/gtk-rs/gtk4-rs/blob/master/gtk4/src/auto/css_provider.rs#L55 needs a filename annotation upstream

I'll update https://github.com/gtk-rs/gir-files/pull/100 this evening. That should contain this change then.


Generated Sitemap has wrong Content-Type

The generated sitemap.xml has a wrong Content-Type according to a validator I use. Message I get back: Incorrect http header content-type: "text/plain" (expected: "application/xml").
Is there a way of setting it?

I may have fixed it, by entering the full url with the protocol.


test fail with pgp-utils version 0.0.32; 0.0.31 works fine

From what I understand, behavior of pgp-utils was changed in 0.0.31, and we had to fix tests in https://github.com/keybase/kbpgp/commit/76b7fcac3ab44703c1138815fe1ae359b29ca457
but it looks like they reverted the behavior in 0.0.32
~/pgp/kbpgp » cat node_modules/pgp-utils/package.json | grep version | grep 0
  "version": "0.0.31"
~/pgp/kbpgp » make test
...
Tests: 240/240 passed
~/pgp/kbpgp » rm -rf node_modules/pgp-utils
~/pgp/kbpgp » npm install<EMAIL_ADDRESS><EMAIL_ADDRESS>/home/zapu/pgp/kbpgp
└──<EMAIL_ADDRESS>~/pgp/kbpgp » cat node_modules/pgp-utils/package.json | grep version | grep 0
  "version": "0.0.32"
~/pgp/kbpgp » make test
- Assertion failed: control characters not allowed in armor
In reading /home/zapu/pgp/kbpgp/test/files/openpgp_js_cure53_audit.iced: TypeError: Cannot read property 'message' of null
TypeError: Cannot read property 'message' of null
...
✖ Failure due to test configuration issues
Tests: 239/240 passed

(sorry for ghetto grep version checking)
@maxtaco Refreshing the buildbot would probably expose this bug. The last "successful" job https://travis-ci.org/keybase/kbpgp/jobs/185556861 used 0.0.31, if it fails with 0.0.32 it would confirm this issue.

Oh, I didn't notice that pgp-utils is a "first party" package as well... pinging @oconnor663 then (sorry if for no good reason)

you're right, we forgot to back and change this!  we'll get it done
shortly...
On Sat, Jan 28, 2017 at 2:05 PM, Michał Zochniak<EMAIL_ADDRESS>wrote:

Oh, I didn't notice that pgp-utils is a "first party" package as well...
pinging @oconnor663 https://github.com/oconnor663 then (sorry if for no
good reason)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/keybase/kbpgp/issues/133#issuecomment-275867622, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA05_-SyBwVOT4klDveqKGwCadP4jg3Kks5rW5GKgaJpZM4LwlYI
.

Ah yep, my mistake. Fix PR is up.


Bad benchmarks on initial test

I've setup an initial test to see how an OK server would do with a decent load and the results don't look so I thought I would share them here and get some feedback on what I'm (probably) doing wrong.
First thing to address would be storage footprint since that is relatively easy to compare apples to apples. The docs say Chronix will take 5-171 times less space (I assume this is compared to CSV or some relatively raw/simple data format). My data rows look like this:
18,25,8547.736954352933,1523318400.973
2,43,1980.6639051377774,1523318401.176
17,69,9500.832828241511,1523318402.991
13,12,1442.8229313976187,1523318403.377
8,66,4088.2959033363563,1523318404.812
5,84,5772.630417804327,1523318405.002
1,54,7276.800267948981,1523318406.934

After importing 16mm of these, I saw a storage footprint of 2.3GB. This is way over than what I expected. I have this data stored in CSV format at a rate of about 50mm data points in 2.2 GB without compression. Does this seem fishy?
The second thing to address is insert rate. I've tried writing in batches of 100, 1000, 2500, 5000 using the golang lib (s_err := c.Store(series, true, time.Second)). The best I have seen was using a single thread to write 5000 data point batches at which point I was able to ingest ~ 800 datapoints/second. After this I tried writing with 10 threads and 20 threads in batches of 1000 and 2500 and max out around ~500 data points / second.
Now I'm on a digital ocean server with 2 VCPU's, 4GB RAM, and Ubuntu but this still seems fishy based on some comparisons. Actually I'm not to worried about the write speed for my specific use case but it would be worth touching on as I'll probably want to be inserting at ~100-1000 data points / second. Is chronix/solr slow at inserts but fast at queries (is that the tradeoff?).
Also I was definitely expecting a lighter storage footprint based on the claim. Am I doing something wrong?

Hi @tamoyal,

First thing to address would be storage footprint since that is relatively easy to compare apples to apples. The docs say Chronix will take 5-171 times less space (I assume this is compared to CSV or some relatively raw/simple data format). My data rows look like this:

Yes, is compared to simple csv files. This paper at Chronix describes also a quantitative evaluation.

After importing 16mm of these, I saw a storage footprint of 2.3GB. This is way over than what I expected. I have this data stored in CSV format at a rate of about 50mm data points in 2.2 GB without compression. Does this seem fishy?

This is no what i expect. How many points do you store in a time series? It seems that the chunks (n points in a solr document) are too small. The example dataset chronix example data has around 76,439,668 points and 5,836 time series. It needs compressed around 56 MB and stored in Chronix with the chronix importer (examples project, java) around 120 MB.

The second thing to address is insert rate. I've tried writing in batches of 100, 1000, 2500, 5000 using the golang lib (s_err := c.Store(series, true, time.Second)). The best I have seen was using a single thread to write 5000 data point batches at which point I was able to ingest ~ 800 datapoints/second. After this I tried writing with 10 threads and 20 threads in batches of 1000 and 2500 and max out around ~500 data points / second.

The chronix-importer  writes 1415549 points / second (one commit at the end of the import), while a chunk has around 7821 points. I have to use the go lib to import the time series and see whats going on...

Also I was definitely expecting a lighter storage footprint based on the claim. Am I doing something wrong?

This could be of too many commits with expensive index creations.

@FlorianLautenschlager thanks for the response and sorry for the delay, i was traveling
This is no what i expect. How many points do you store in a time series?
Not sure I understand this question but I think they are all part of one time series in all data points are associated with the same Name.
It seems that the chunks (n points in a solr document) are too small.
Ok - I used the default installation params. So I need to tune to take advantage of the compression? Can you suggest a setting for this?
while a chunk has around 7821 points
By chunk do you mean the write batch size as in you commit every ~7821 data points?
This could be of too many commits with expensive index creations.
What defines an expensive vs cheap index creation?