VM: MegamorphicLookup #3: large hierarchy with single target

This issue is split out from https://github.com/dart-lang/sdk/issues/26017
Data taken from running dart2js on a large app.
The HInstruction hierarchy (pkg/compiler/lib/src/ssa/nodes.dart) has 91 classes (17 abstract). The base class declares next and previous fields which are not shadowed. These field accessses are always megamorphic.
Consider the HInstructionList.detach method below. Each of the .previous and .next  field access is megamorphic.
Looking at the cids, it appears that a range check might be an adequate test.
With such a large number of classes, it is likely that some kind of CHA is necessary to discover the range and to avoid deopts as rare classes are instantiated.
abstract class HInstruction implements Spannable {
  ...
  HInstruction previous = null;
  HInstruction next = null;
}

class HInstructionList {
  HInstruction first = null;
  HInstruction last = null;

  void detach(HInstruction instruction) {
    if (instruction.previous == null) {
      first = instruction.next;
    } else {
      instruction.previous.next = instruction.next;
    }
    if (instruction.next == null) {
      last = instruction.previous;
    } else {
      instruction.next.previous = instruction.previous;
    }
    instruction.previous = null;
    instruction.next = null;
  }
}

class HBasicBlock extends HInstructionList {
 ...
}

This also, not surprisingly, impacts analyzer. The class AstNodeImpl defines the method _becomeParentOf(AstNodeImpl node). Although there are a large number of subclasses of AstNodeImpl, there is only one implementation of this method. Unfortunately, this becomes a megamorphic lookup. A fix for this issue would significantly help our performance.


DNN-7201: move the setting into host setting.

Assigned to Joe - based on a comment in the JIRA

@ashishpd Why was this merged?


Update GPUManager documentation

Original issue: https://charm.cs.illinois.edu/redmine/issues/1491

Update the documentation of GPUManager to match the redesign.

Original date: 2017-11-29 20:43:11

Gerrit patch: ~~https://charm.cs.illinois.edu/gerrit/#/c/3330/~~ https://github.com/UIUC-PPL/charm/commit/c925ff9485b53e906d8e88e1d3cc550425a808aa
Still undergoing modifications, though.

Original date: 2018-02-07 03:41:18

Completed first draft.


Update dependencies in preparation for release

Updating dependency combine from 2.1.1 to 3.6 will require some code rework since there were non-compatible api changes.
Updating the rest of the dependencies to the following did not change any test results:
combine = "2.1.1"
-libc = "0.2.0"
+libc = "0.2"
time = "0.1"
-byteorder = "1.2.1"
+byteorder = "1.2"
[dev-dependencies]
elf = "0.0.10"
-json = "0.11.4"
+json = "0.11"

Thanks! I had a look at combine, I don't believe it's worth changing the code to compile with 3.6.0 at the moment. We can still update to 2.5, though.
I'll push a commit with those changes.


Add Travis CI badge to README

https://travis-ci.org/opencontainers/ocitools
Signed-off-by: Chris Aniszczyk<EMAIL_ADDRESS>Backported to v1.0.0.rc1 from d0ad8ca #170 (cherry-pick applied cleanly).
Signed-off-by: W. Trevor King<EMAIL_ADDRESS>

LGTM


Issue about DataFrame index

If passing a DataFrame whose index column is named something other than 'index' (e.g. 'id'), ImagePredictor will fail because of this line: https://github.com/awslabs/autogluon/blob/93fab8dd18918b96d886a9f81b162a57ebed0a0b/vision/src/autogluon/vision/predictor/predictor.py#L499

you can reset the data frame meanwhile if the maintainer considers adding this functionality.

Thanks for mentioning this bug! I have fixed this in #1177.


How to?

Did you manage to integrate firebase login in discourse ?

nope. but I know it is possible.


RFC UserspacePtr

We really need a UserspacePtr that checks address can be deref'd.
#102
Here's my first attempt at an implementation, please give me your comments:
Jobs
The UserspacePtr is expected to fulfill the following goals:

Check that a given address is ok to derefence.
Check that it won't be unmapped while we have a reference to it.

We also choose to check user permissions. We define the rules as "we do only what the process could have done himself":

For UserspacePtr we read from: USER_ACCESSIBLE | READABLE
For UserspacePtr we write to: USER_ACCESSIBLE | WRITABLE

This means we'll never output in a read only mapping.
We might want to use a QueryMemory to get the mapping.
Limitations
You cannot make a UserspacePtr pointing to the address space of another process.
Our UserspacePtr just derefs address, this will only work if address is in our cr3.
To address a value in another address space, you need to create a CrossProcessMapping,
and might consider add some additional checks for user accessible and readable-writable
Pseudo code
/// Pointer to a userspace value.
///
/// Can be created mindlessly, check happens at deref time.
///
/// Can only represent pointers in the **current** address space.
pub struct UserspacePtr<T: Sized> {
    address: VirtualAddress,

    // marker for T
    phantom: PhantomData<T>
}

impl<T> UsersapacePtr<T> {

    /// Gets a reference to the pointed value.
    ///
    /// Checks that `self.address` is valid and is mapped in the current process.
    ///
    /// Returns a reference whose lifetime is tied to the lifetime of the lock, this way
    /// we can guarantee that the mapping in which `address` falls will not be unmapped
    /// while we're holding a reference to it.
    /// A side effect is that we can't modify the ProcessMemory while we hold a deref'ed UserspacePtr,
    /// but I don't see it causing any problem.
     pub fn<'lock> deref(&self, lock: &'lock MutexGuard<ProcessMemory>) -> Result<&'lock T, KernelError> {

        // Check address + size_of<T> is in UserLand
        // Check that if falls in a mapping at least USER_ACCESSIBLE | READABLE
        // Check size and alignment of * T
        // Check that `lock` corresponds to the current address space, if it isn't we can't simply make `address` a pointer, the kernel would need to do a CrossProcessMapping.

        // As we're going to interpret T, maybe it should be POD ?
    }

    /// Gets a mutable reference to the pointed value.
    ///
    /// Same as `deref`, but mut.
    ///
    /// We choose to only write to mappings that are user writable, i.e. that the user could
    /// have written to himself.
    pub fn<'lock> deref_mut(&self, lock: &'lock MutexGuard<ProcessMemory>) -> Result<&'lock T, KernelError> {

        // Same as deref
        // Checks that if falls in a mapping at least USER_ACCESSIBLE | WRITABLE

        // lift the restriction on POD ? But we're still constructing a ref to it ...
    }
}

bikeshed time:

The functions should be get and get_mut.
T should definitely be a POD.
Instead of a MutexGuard, we should just take a &ProcessMemory. Makes it easier to setup testing and change lock kind. That remark holds true throughout the codebase: I think we should avoid taking MutexGuard unless we need to precisely control the dropping behavior.

Related; #102

I'm worried about having a &/&mut in kernel on a memory region that can be modified by another user thread. Should we use UnsafeCell somehow, to prevent compiler optimisations ? (Same reasoning as Mmio #202 )

Note that deref_mut() (or get_mut() as roblabla suggests) takes only a const reference to the ProcessMemory.
This is ok since we only check the bookkeeping, and we choose it to be allowed since the ProcessMemory semantically does not track the content of the memory, only the layout of its mappings.
This means that we can have several UserspacePtr mutably derefed at the same time, or a const source and a mutable destination. This is great for most use cases.
However no effort is made to check that they do not overlap, and this might become a problem if we hold several references to the same memory zone.

Actually, get should probably return T and not &T ? This would allow two things:

We could use read_volatile(), avoiding optimization problems
We could have a special version for ARM using ldtr instead of manually querying the page tables.

Returning a T instead of &T also haves the advantage to make it impossible to mess up rust's aliasing rules, especially for the overlapping source and dest case, where it would be a nightmare to check, as even UserspacePtrs borrowed from different ProcessMemory's, could still point to the same physical memory through a shared memory mapping.

So right now, we use UserSpacePtr almost everywhere HOS does. There's one place where we use UserSpacePtr even though we really shouldn't though: The address of the IPC command buffer. The kernel is supposed to remap it in kernelspace in order to access it.
IMO we should probably just reserve two pages of kernel VMEM that IPC command buffers will be remapped to when doing the copying. Then everything else can use UserSpacePtr.
UserSpacePtr should have a get (and set for UserSpacePtrMut) which does a ptr::read() (and a ptr::write()). It should not have a Deref, we should never be dereferencing userspace memory. This is especially important since we'll want to enable SMEP and SMAP (prevents execution and access of userspace memory from the kernel), which requires we centralize all access to userspace memory in a single function (which will take care of temporarily disabling SMAP).

From discord:
So here's the thing: We want a copy_from_user function whose goal is to safely copy bytes from a user-provided address (pointing to the user address-space) into the kernel address-space. If the user-provided address points to unmapped memory, non-readable memory, kernel-memory, or anything else that's potentially invalid, it should return an error.
On ARM, we have a function that basically does just that, which HOS uses. It drops its privileges by using ldtr, which will execute an ldr as if it was ran from EL0. The HOS exception handler checks if the instruction that faulted is within one of the "user to kernel/kernel to user" copy routines, and simply erets to the calling function (which is stored in X30, thankfully) with X0=0.
On x86, the situation is a lot more complicated. We have no ldtr instruction. "Temporarily" dropping privileges is not doable. Furthermore, we don't have access to the parent function easily. So the problem is, how do you prevent faults if the user passes an address that's just unmapped for instance?
Now, we could do something very simple like walking the TLB to figure out if the access is OK before doing it (Asking for permission instead of asking for forgiveness). In the simple, single-processor no preemption world that SunriseOS currently live in, that would work fine, but it'd be a very brittle mechanism. But it's slow, and would probably be a bit more complicated in an SMP-enabled world.
Here's what linux does: It has a special section, called __ex_table, that contains ranges of kernel addresses and an exception handler associated with that range. When the pagefault handler executes, it first checks if the faulting IP is contained within these ranges, and if it does, it will iret to the exception handler specified. here's a more in-depth look.
So copy_from_user has basically two asm stubs, the "normal" paths just does the copy, and a separate "failure handling" subroutine that just reports the error. And the pagefault will iret to the failure handling subroutine. This should be performant, especially in the happy path.

I dislike the linux solution, so I went on a quest to see if I could do something better, something that didn't go through the exception handler. I've found something that could work, but relies on extensions that aren't present in most processors, the Intel Transactional Synchronization Extensions (TSX), and in particular the Restricted Transactional Memory (RTM) extension. Those extensions are absolutely not meant for what I'm going to show here, but it should, in theory, work if they are present.
RTM provides two new instructions of interest: XBEGIN and XEND. The idea is to make arbitrary memory accesses "atomic" - XBEGIN has a "failure subroutine" opcode executed when the transaction fails. A transaction can fail for a wide variety of reasons, like if another processor accesses the memory we were writing in our transaction. However, there is one reason that's very interesting to us: A transaction will fail if we attempt accessing unmapped or otherwise unavailable memory. We can effectively bypass the Page Fault handler and handle it entirely in userspace!
A copy_from_user would look like this:
mov esi, $0 // Move source
mov edi, $1 // Move dest
mov ecx, $2 // Move byte count

xbegin failed
rep movsb
xend
failed: ret

Upon returning from this function, if eax == 0, the copy from user succeeded.
This is a super elegant solution, but the TSX instructions aren't widely available, so relying on it for such a critical aspect of the OS isn't a great idea. In particular, neither my desktop nor my server have the RTM feature enabled.

So we're back to square one. Still no good solution available. I'm currently swinging towards doing something similar to what linux is doing. In theory, we only need two such "special" functions that can be seen by the fault handler: a copy_from_user and a copy_to_user. Having special asm stub for them, that the fault handler detects, and returns to a special handler somehow should be doable.


Error while installing

Could not find a matching version of package alchemyguy/YoutubeLaravelApi. Check the package spelling, your version constraint and that the package is available in a stability which matches your mini
mum-stability (stable).
laravel 5.4

@shanka12 sorry for the inconvenience, the package name was updated recently.
please use:
composer require alchemyguy/youtube-laravel-api

ok. Thanks


[feat] add support to create virtual display

with option --create-new-display=WxH you may cast a new virtual screen
based on the fork https://github.com/anirudhb/scrcpy/

Superseded by #5370.


Why we get last 4 layers while residual connection transfer useful knowledge to the subsequent layers?

As in BERT structure, residual connection transfer knowledge (let's say for classification) to the next layers, why in some cases contacting (or even adding last 4 layers) improve the performance of the layer?
If some useful information gets missed in the middle layers isn't it better to use middle layers as output for the syntactical task as the model embed semantic information on the top and syntactical information in the middle.
I appreciate if you can share your intuition into it or your experience.

From my view, for classification task, the syntactical information is useless, so just use the top-level information is enough


bollinger mark & transform

Fixes #548.

Nice, we get a build failure now when we don’t document something. 😁 I’ll add that to the PR.

Nice! It’s a useful and sound method for general time series analysis.
I'm just wondering if we should keep that name and defaults of the transform. “Bollinger” seems to be confined to finance (according to wikipedia), and has the default multiplier = 2 (not a bad choice, but mathematically surprising).
From https://www.bollingerbands.com/bollinger-bands, that "bands" is plural—they represent the three lines y = moving average MA, y2 = MA+2 Dev, and y1 = MA-2 Dev. Whereas how I see it, is a singular band, the area between y1 and y2.
Also, there is a trademark on the name.
Even in finance, people seem to be using “volatility” often (shown as a 1-sigma band). The 1-sigma band excludes 31.8% of the distribution on a normal model, so it makes sense to use a 2-sigma band as the default (4.6% outliers), but it's also legit to do 3-sigma (0.3%), etc. Other data science domains seem to be using words like “rolling window variance” or  “rolling window deviation”, and plotting them with 1-sigma, 2-sigma, 3-sigma lines or bands, or adapting the sigmas to match 10%, 5%, 1%….


NSEC Bunker failed to propogate url after paying

Hey guys, Paid for NSEC bunker. I have transaction ID for proof of payment, looks like it stuck on generating bunker. Logged into Coracle, but can't sign notes with bunker.

Maybe talk to @pablof7z? Bunker problems are currently very hard to debug, even not remotely.


Windowing over time feature TODO

While running some custreamz pipelines which require a window-over-time functionality, I came across a TODO in the existing streamz codebase here, which I wasn't able to understand.
Is anyone familiar with what needs to be done here? I'm assuming this is not some sort of bug, but something which must be done to improve efficiency (maybe throughput or memory-utilization).

I believe it is only stating that .append() is an inefficient process for dataframes, and can be avoided in this line, if the original dataframe has already become empty.


Regarding the availability of features related to ParameterSource

Expected Behavior
Be able to use something like the following callback for an option parameter:
def process_xmin_args(ctx, param, xmin_args):
    print(ctx.get_parameter_source(param.name))

Actual Behavior
Get AttributeError: 'Context' object has no attribute 'get_parameter_source'
Environment

Python version: 3.8.1
Click version: 7.0

I have the head of the source cloned, so I do see that methods for Context like set_parameter_source and get_parameter_source are in the codebase as of around June of 2019, however these are not yet in the latest 7.0 release. So guess I'm just wondering approximately when the next release will be, which presumably will include this among other new feature additions.
(apologies if this question sounds impatient and pushy, not my intention at all, just curious)

I'm sorry, we can't make predictions about release schedules. Click is under active development. I will release 7.1 when I am able to.


fix: Image layer load big svg with width and height will not be see a…

…nd imageSize property is invalid

If you have an SVG without width and height attributes this approach will not work with Firefox.  See https://github.com/openlayers/openlayers/issues/14788#issuecomment-1575561642  There is a similar issue with unsized SVGs in Icon styles.  The imageSize option was originally intended for Internet Explorer which could not detect the size of an SVG.

If you have an SVG without width and height attributes this approach will not work with Firefox. See #14788 (comment) There is a similar issue with unsized SVGs in Icon styles. The imageSize option was originally intended for Internet Explorer which could not detect the size of an SVG.

Thank you for your suggestion, but my scenario may not be described clearly. My SVG file has attribute widths and heights, but the values are very large. width=52354, height=33614, and I do not want to change this size, which will cause the displayed SVG to be blurry. Before submitting the pull request, I printed the width and height of the canvas in the getImage method of the ol/image.js file, which are 52354 and 33614, This will result in not being able to see the specific content of the SVG file, so I added a check to see if imageSize has been set, so that after fixing it, it can meet my usage scenarios

Yes, this will help prevent exceeding canvas limits in your situation
The change will also make the current handling of imageSize in ol/source/ImageStatic handleImageChange obsolete so
  let imageWidth, imageHeight;
  if (this.imageSize_) {
    imageWidth = this.imageSize_[0];
    imageHeight = this.imageSize_[1];
  } else {
    imageWidth = image.width;
    imageHeight = image.height;
  }

could be simplified to
  const imageWidth = image.width;
  const imageHeight = image.height;

I have added new test cases and updated the comments on the imageSize attribute in ol/images.js. In the test cases, due to the inconvenience of publishing SVG, I have removed many paths, which appear vague and do not affect the actual effect. If you need to download it, please download it in a timely manner as SVG may be deleted in the future

For testing purposes you could take a copy of one of the existing test files cross.svg or me0.svg and edit the width and height attributes.  I think you need to test with an imageSize which matches the extent aspect ratio such as [900, 600] as well as one which does not such as [800, 600].  Apart from some differences in blurring I would expect these to look similar when rendered, but I get very different results using your code.

Firstly, I modified the test case and added a new test. svg. When the width and height were adjusted significantly, the content of the svg would shift to the upper left corner. However, after testing, expected. png did not capture the entire rendered area, so I adjusted the center point of the view so that I could only see the content of the svg after rerunning the test case. However, when I switched to the large svg I used for local debugging, I obtained the same result, Based on this test result, if imageSize is not set, no content will be captured
Secondly, I created a new image static. html locally to test the modified test. If you do not set imageSize, you will not see any content in the viewport area, and the same result will be obtained for both SVG and SVG that already have a large width and height

Ok,thanks


Add Mulltiprocessing to how market update observers

We found that multiprocessing increases the processing time, and therefore we are not going to implement it.


run test on pull request.

We can improve to only run the project changed in the pr in the future.
Good Idea. Let's me add a Task or Issue for that


June release

[x] Build stable for all platforms @isidorn
[x] Check translations (check i18n folder git history for latest translations, then build locally and see the amount of untranslated strings) @sandy081
[x] All candidate issues verified https://github.com/Microsoft/vscode/issues?q=milestone%3A"June+2018"+is%3Aclosed+label%3Acandidate
[x] Add a git tag to HEAD of release/<x.y> in format x.y.z (for vscode.d.ts download)  @isidorn
[x] Make rpm signing request @Tyriar
[x] Sanity check of installable bits

[x] Windows

[x] signed installer 32-bit @jrieken
[x] signed installer 64-bit @jrieken
[x] zip 32-bit @dbaeumer
[x] zip 64-bit @dbaeumer

[x] OS X - @weinand
[x] Linux

[x] deb package 32-bit @joaomoreno
[x] deb package 64-bit @joaomoreno
[x] rpm package 64-bit @joaomoreno
[x] rpm package 32-bit @joaomoreno
[x] archive 32-bit @joaomoreno
[x] archive 64-bit @joaomoreno

[x] Publish website @gregvanl
[x] Publish to stable @chrisdias
[x] Publish deb and rpms to repositories manually @RMacfarlane
[x] Enable scheduled insider builds @isidorn
[x] Twitter announcement @chrisdias

We shipped yeeeey
Closign this issue


Add refund method for collective to collective transactions

Resolve https://github.com/opencollective/opencollective/issues/1935
See: https://github.com/opencollective/opencollective/issues/1656

292939339394948


Kamstrup Multical 401

The request
Hi,
I just want to inform you that this integration works with the Kamstrup Multical 401.
Would be great if you could update this integration to work like the Kamstrup 401 integration, so you could change the settings via the home assistant ui.
Thanks for the great work!
Additional information
No response

I agree, that would be nice. However, I don't have this meter anymore which makes it hard for me to verify any changes.
When looking at the current install base, 6 (for people who have analytics enabled), and 19 stars, it doesn't have a lot of community impact.
The code for this integration is very outdated, I do try to keep a few things up to date but not really touching any code, afraid of breaking anything.
It would maybe be better for anyone to fork my 403 integration and replace the meter specific code with the code from here.


update modules to make subnets, availability zones, and similar more dynamic

This project pre-dates quite a few of Terraform's features that support dynamic modules, and the original modules (asg, cluster-network, consul-cluster, and related, in particular), ought to be updated to make use of these features. As part of this, we should also evaluate all modules using string-lists to see if that can be avoided.

The asg module, and the other modules which use asg, have all been updated to be more flexible, and to drop use of string-lists. The cluster-network and consul-cluster modules could use other updates, so I will leave this ticket open.

Can we maybe break this into different issues for each kind of change?

I hit up the changes I wanted just a couple of weeks ago, and those changes are on master. I think we close this as complete.


Incorrectly splits the *.blogspot.com

Any idea, why "sergueiko.blogspot.com" is wrongly split? E.g it gives me:
Domain: sergueiko
Hostname: sergueiko.blogspot.com
SubDomain: null
TLD: blogspot.com
Whereas I'd expect the Domain: "blogspot'.

blogspot.com is listed as a TLD in the https://publicsuffix.org/list/public_suffix_list.dat


Right-side TOC not fits on <1280, no scrollbar

Right table of contents is cut on resolutions <1280px, and no scroll bar at the bottom, so can't move left TOC out of view. In Ubuntu with non-hiding left application bar (window.innerWidth -> 1215), it's even more grave.

Seen at: https://docs.docker.com/compose/compose-file/

Closing as duplicate of #2730


multiple errors building with ghc-7.10

When trying to build elm-compiler with ghc 7.10 and cabal <IP_ADDRESS>, I am getting the following error (which looks like it is coming from language-glsl):
$ cabal install elm-compiler-0.15
Resolving dependencies...
Configuring language-glsl-0.1.1...
Building language-glsl-0.1.1...
Preprocessing library language-glsl-0.1.1...
[1 of 4] Compiling Language.GLSL.Syntax ( Language/GLSL/Syntax.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL/Syntax.o )
[2 of 4] Compiling Language.GLSL.Parser ( Language/GLSL/Parser.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL/Parser.o )

Language/GLSL/Parser.hs:202:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:209:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:219:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:226:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:236:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:244:3: Warning: Defined but not used: ‘m’
[3 of 4] Compiling Language.GLSL.Pretty ( Language/GLSL/Pretty.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL/Pretty.o )
[4 of 4] Compiling Language.GLSL    ( Language/GLSL.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL.o )
In-place registering language-glsl-0.1.1...
Preprocessing executable 'glsl-pprint' for language-glsl-0.1.1...

bin/glsl-pprint.hs:4:8:
    Ambiguous module name ‘Text.PrettyPrint.HughesPJClass’:
      it was found in multiple packages:
      pretty-<IP_ADDRESS>@prett_7jIfj8VCGFf1WS0tIQ1XSZ prettyclass-<IP_ADDRESS>@prett_2FbqlbOn8rU3vmeTF7SGKM
Failed to install language-glsl-0.1.1
cabal: Error: some packages failed to install:
elm-compiler-0.15 depends on language-glsl-0.1.1 which failed to install.
language-glsl-0.1.1 failed during the building phase. The exception was:
ExitFailure 1

The author of language-glsl appears to be aware of the issue but no fix has yet been posted (noteed/language-glsl#11).  However, there is a pull request fixing the issue (noteed/language-glsl#10)
This can be fixed by installing a version of language-glsl that includes the pull request.
After doing that, I again try to install elm-compiler, but I get the following errors:
$ cabal install elm-compiler-0.15        Resolving dependencies...
Configuring language-glsl-0.1.2...
Building language-glsl-0.1.2...
Preprocessing library language-glsl-0.1.2...
[1 of 4] Compiling Language.GLSL.Syntax ( Language/GLSL/Syntax.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL/Syntax.o )
[2 of 4] Compiling Language.GLSL.Parser ( Language/GLSL/Parser.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL/Parser.o )
[3 of 4] Compiling Language.GLSL.Pretty ( Language/GLSL/Pretty.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL/Pretty.o )
[4 of 4] Compiling Language.GLSL    ( Language/GLSL.hs, dist/dist-sandbox-8d402a8b/build/Language/GLSL.o )
In-place registering language-glsl-0.1.2...
Preprocessing executable 'glsl-pprint' for language-glsl-0.1.2...
[1 of 1] Compiling Main             ( bin/glsl-pprint.hs, dist/dist-sandbox-8d402a8b/build/glsl-pprint/glsl-pprint-tmp/Main.o )
Linking dist/dist-sandbox-8d402a8b/build/glsl-pprint/glsl-pprint ...
Creating package registration file:
/tmp/pkgConf-language-glsl-0.12947025671726956429.2
Installing library in
/home/illabout/filesystem/elm/.cabal-sandbox/lib/x86_64-linux-ghc-7.10.1/langu_FRDQaCdWaxKBqHAjmOOXtu
Installing executable(s) in /home/illabout/filesystem/elm/.cabal-sandbox/bin
Registering language-glsl-0.1.2...
Installed language-glsl-0.1.2
Downloading elm-compiler-0.15...
Configuring elm-compiler-0.15...
Building elm-compiler-0.15...
Preprocessing library elm-compiler-0.15...
[ 1 of 63] Compiling Type.PrettyPrint ( src/Type/PrettyPrint.hs, dist/dist-sandbox-8d402a8b/build/Type/PrettyPrint.o )
[ 2 of 63] Compiling Paths_elm_compiler ( dist/dist-sandbox-8d402a8b/build/autogen/Paths_elm_compiler.hs, dist/dist-sandbox-8d402a8b/build/Paths_elm_compiler.o )
[ 3 of 63] Compiling AST.Helpers      ( src/AST/Helpers.hs, dist/dist-sandbox-8d402a8b/build/AST/Helpers.o )
[ 4 of 63] Compiling AST.PrettyPrint  ( src/AST/PrettyPrint.hs, dist/dist-sandbox-8d402a8b/build/AST/PrettyPrint.o )

src/AST/PrettyPrint.hs:4:1: Warning:
    Module ‘Control.Monad.Trans.Error’ is deprecated:
      Use Control.Monad.Trans.Except instead

src/AST/PrettyPrint.hs:58:10: Warning:
    In the use of type constructor or class ‘Error’
    (imported from Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/AST/PrettyPrint.hs:58:10: Warning:
    In the use of type constructor or class ‘Error’
    (imported from Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/AST/PrettyPrint.hs:63:10: Warning:
    In the use of type constructor or class ‘ErrorList’
    (imported from Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/AST/PrettyPrint.hs:63:10: Warning:
    In the use of type constructor or class ‘ErrorList’
    (imported from Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"
[ 5 of 63] Compiling AST.Annotation   ( src/AST/Annotation.hs, dist/dist-sandbox-8d402a8b/build/AST/Annotation.o )
[ 6 of 63] Compiling AST.Literal      ( src/AST/Literal.hs, dist/dist-sandbox-8d402a8b/build/AST/Literal.o )
[ 7 of 63] Compiling Generate.JavaScript.Helpers ( src/Generate/JavaScript/Helpers.hs, dist/dist-sandbox-8d402a8b/build/Generate/JavaScript/Helpers.o )
[ 8 of 63] Compiling AST.Variable     ( src/AST/Variable.hs, dist/dist-sandbox-8d402a8b/build/AST/Variable.o )
src/AST/Variable.hs:3:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[ 9 of 63] Compiling AST.Type         ( src/AST/Type.hs, dist/dist-sandbox-8d402a8b/build/AST/Type.o )

src/AST/Type.hs:11:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[10 of 63] Compiling Generate.JavaScript.Port ( src/Generate/JavaScript/Port.hs, dist/dist-sandbox-8d402a8b/build/Generate/JavaScript/Port.o )
[11 of 63] Compiling Type.Type        ( src/Type/Type.hs, dist/dist-sandbox-8d402a8b/build/Type/Type.o )

src/Type/Type.hs:3:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()

src/Type/Type.hs:6:1: Warning:
    Module ‘Control.Monad.Error’ is deprecated:
      Use Control.Monad.Except instead

src/Type/Type.hs:198:11: Warning:
    In the use of type constructor or class ‘Error’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/Type/Type.hs:198:31: Warning:
    In the use of type constructor or class ‘ErrorT’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/Type/Type.hs:198:62: Warning:
    In the use of type constructor or class ‘ErrorT’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/Type/Type.hs:204:17: Warning:
    In the use of type constructor or class ‘Error’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/Type/Type.hs:204:37: Warning:
    In the use of type constructor or class ‘ErrorT’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/Type/Type.hs:204:68: Warning:
    In the use of type constructor or class ‘ErrorT’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"
[12 of 63] Compiling Type.Fragment    ( src/Type/Fragment.hs, dist/dist-sandbox-8d402a8b/build/Type/Fragment.o )
[13 of 63] Compiling Type.Hint        ( src/Type/Hint.hs, dist/dist-sandbox-8d402a8b/build/Type/Hint.o )

src/Type/Hint.hs:3:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[14 of 63] Compiling Type.State       ( src/Type/State.hs, dist/dist-sandbox-8d402a8b/build/Type/State.o )
src/Type/State.hs:4:1: Warning:
    The import of ‘<*>, Applicative, <$>’
    from module ‘Control.Applicative’ is redundant
[15 of 63] Compiling Type.ExtraChecks ( src/Type/ExtraChecks.hs, dist/dist-sandbox-8d402a8b/build/Type/ExtraChecks.o )

src/Type/ExtraChecks.hs:10:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()

src/Type/ExtraChecks.hs:11:1: Warning:
    Module ‘Control.Monad.Error’ is deprecated:
      Use Control.Monad.Except instead

src/Type/ExtraChecks.hs:29:26: Warning:
    In the use of type constructor or class ‘ErrorT’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"

src/Type/ExtraChecks.hs:41:8: Warning:
    In the use of type constructor or class ‘ErrorT’
    (imported from Control.Monad.Error, but defined in Control.Monad.Trans.Error):
    Deprecated: "Use Control.Monad.Trans.Except instead"
[16 of 63] Compiling AST.Pattern      ( src/AST/Pattern.hs, dist/dist-sandbox-8d402a8b/build/AST/Pattern.o )
[17 of 63] Compiling AST.Expression.General ( src/AST/Expression/General.hs, dist/dist-sandbox-8d402a8b/build/AST/Expression/General.o )
[18 of 63] Compiling Transform.Expression ( src/Transform/Expression.hs, dist/dist-sandbox-8d402a8b/build/Transform/Expression.o )

src/Transform/Expression.hs:4:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[19 of 63] Compiling AST.Expression.Source ( src/AST/Expression/Source.hs, dist/dist-sandbox-8d402a8b/build/AST/Expression/Source.o )
[20 of 63] Compiling AST.Expression.Valid ( src/AST/Expression/Valid.hs, dist/dist-sandbox-8d402a8b/build/AST/Expression/Valid.o )
[21 of 63] Compiling AST.Expression.Canonical ( src/AST/Expression/Canonical.hs, dist/dist-sandbox-8d402a8b/build/AST/Expression/Canonical.o )
[22 of 63] Compiling AST.Declaration  ( src/AST/Declaration.hs, dist/dist-sandbox-8d402a8b/build/AST/Declaration.o )
[23 of 63] Compiling Transform.Substitute ( src/Transform/Substitute.hs, dist/dist-sandbox-8d402a8b/build/Transform/Substitute.o )
[24 of 63] Compiling Generate.Cases   ( src/Generate/Cases.hs, dist/dist-sandbox-8d402a8b/build/Generate/Cases.o )

src/Generate/Cases.hs:3:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[25 of 63] Compiling Transform.SortDefinitions ( src/Transform/SortDefinitions.hs, dist/dist-sandbox-8d402a8b/build/Transform/SortDefinitions.o )

src/Transform/SortDefinitions.hs:5:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[26 of 63] Compiling Transform.Definition ( src/Transform/Definition.hs, dist/dist-sandbox-8d402a8b/build/Transform/Definition.o )

src/Transform/Definition.hs:4:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[27 of 63] Compiling Elm.Compiler.Type ( src/Elm/Compiler/Type.hs, dist/dist-sandbox-8d402a8b/build/Elm/Compiler/Type.o )
src/Elm/Compiler/Type.hs:7:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[28 of 63] Compiling Elm.Compiler.Type.Extract ( src/Elm/Compiler/Type/Extract.hs, dist/dist-sandbox-8d402a8b/build/Elm/Compiler/Type/Extract.o )
[29 of 63] Compiling Elm.Compiler.Version ( src/Elm/Compiler/Version.hs, dist/dist-sandbox-8d402a8b/build/Elm/Compiler/Version.o )
[30 of 63] Compiling Elm.Utils        ( src/Elm/Utils.hs, dist/dist-sandbox-8d402a8b/build/Elm/Utils.o )

src/Elm/Utils.hs:5:1: Warning:
    Module ‘Control.Monad.Error’ is deprecated:
      Use Control.Monad.Except instead
[31 of 63] Compiling Parse.Helpers    ( src/Parse/Helpers.hs, dist/dist-sandbox-8d402a8b/build/Parse/Helpers.o )

src/Parse/Helpers.hs:210:1:
    Non type-variable argument in the constraint: Stream s m Char
    (Use FlexibleContexts to permit this)
    When checking that ‘betwixt’ has the inferred type
      betwixt :: forall b s u (m :: * -> *).

                     Stream s m Char =>
                     ([Char] -> a) -> ParsecT s u m a
    In an equation for ‘shader’:
        shader
          = do { try (string "[glsl|");
                 rawSrc <- closeShader id;
                 case glSource rawSrc of {
                   Left err -> parserFail . show $ err
                   Right tipe -> return ... } }
          where
              closeShader builder = choice [do { ... }, ....]
Failed to install elm-compiler-0.15
cabal: Error: some packages failed to install:
elm-compiler-0.15 failed during the building phase. The exception was:
ExitFailure 1

This looks like it is caused by this.

This looks like it is fixed on #921.

I still have this

What does it mean you still "have this"? Which version of elm-compiler are you trying to build, via which route/commands, and what is the output (log of the failed attempt)?

When running the BuildFromSource script for 0.15 I get the following, which causes the build for elm-compiler 0.15 to fail. I also have some other dependency failures.
Failed to install language-glsl-0.1.1
Build log ( /home/joren/bin/Elm-Platform/0.15/.cabal-sandbox/logs/language-glsl-0.1.1.log ):
Configuring language-glsl-0.1.1...
Building language-glsl-0.1.1...
Preprocessing library language-glsl-0.1.1...
[1 of 4] Compiling Language.GLSL.Syntax ( Language/GLSL/Syntax.hs, dist/dist-sandbox-24cedcf5/build/Language/GLSL/Syntax.o )
[2 of 4] Compiling Language.GLSL.Parser ( Language/GLSL/Parser.hs, dist/dist-sandbox-24cedcf5/build/Language/GLSL/Parser.o )

Language/GLSL/Parser.hs:202:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:209:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:219:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:226:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:236:3: Warning: Defined but not used: ‘m’

Language/GLSL/Parser.hs:244:3: Warning: Defined but not used: ‘m’
[3 of 4] Compiling Language.GLSL.Pretty ( Language/GLSL/Pretty.hs, dist/dist-sandbox-24cedcf5/build/Language/GLSL/Pretty.o )
[4 of 4] Compiling Language.GLSL    ( Language/GLSL.hs, dist/dist-sandbox-24cedcf5/build/Language/GLSL.o )
In-place registering language-glsl-0.1.1...
Preprocessing executable 'glsl-pprint' for language-glsl-0.1.1...

bin/glsl-pprint.hs:4:8:
    Ambiguous module name ‘Text.PrettyPrint.HughesPJClass’:
      it was found in multiple packages:
      pretty-<IP_ADDRESS>@prett_7jIfj8VCGFf1WS0tIQ1XSZ prettyclass-<IP_ADDRESS>@prett_2FbqlbOn8rU3vmeTF7SGKM

Can you try 0.15.1 to see whether that works?
Are you sure you started in a completely fresh directory?
If 0.15.1 works for you, but 0.15 doesn't, then we might be able to tell what is the cause for compilation failure at your end (by looking at what changed in between the two that might have an effect on glsl stuff).
On the other hand, note that the README of https://github.com/elm-lang/elm-platform explicitly says that GHC 7.8 is still "the norm", but GHC 7.10 should work from Elm 0.15.1 onwards. So if you are trying to build 0.15 with GHC 7.10, you are operating outside the specified parameters. I'm not sure we will be able to accommodate this.
Why do you want to build 0.15 instead of 0.15.1?

I have just the same issue with ghc 7.10.3, cabal <IP_ADDRESS>, ubuntu 14.04


delete config

Tests that check the config is valid also need to be removed

There are still references to this file in docs and a(n unused) test fixture that relies on it. It would be good if the docs could be updated to either include some example config or link to somewhere some examples can be found.

hmmm, please say which fille

Sorry, the defaults.yaml file
(⇅ 3) v3.10.13 (blueapi) /scratch/athena/blueapi (main)[*?]
❯ rg -C2 'defaults.yaml'
<EMAIL_ADDRESS>tests/test_config.py-54-def default_yaml(package_root: Path) -> Path:
tests/test_config.py:55:    return package_root.parent.parent / "config" / "defaults.yaml"
tests/test_config.py-56-
tests/test_config.py-57-
--
docs/how-to/configure-app.md-5-can be passed to the `blueapi` command.
docs/how-to/configure-app.md-6-
docs/how-to/configure-app.md:7:An example of this yaml file is found in `config/defaults.yaml`, which follows
docs/how-to/configure-app.md-8-the schema defined in `src/blueapi/config.py` in the `ApplicationConfig`
docs/how-to/configure-app.md-9-object.

That would be good as well, but I think having examples in the docs without having to read through the code would be helpful for setting up new environments.


chore: update google.golang.org/protobuf to v1.33.0

This is to move away from CVE-20204-24786: infinite loop vulnerability.

Thanks @israel-hdez!
/lgtm


Ansible error after merging with upstream

Hello!
I've merged upstream/master today and running install or install -f immediately after sudo password gives the following output:
ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '/Users/macbook/wincent/roles/vim/tasks/main.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- import_tasks: bundle.yml
  ^ here

The error appears to have been in '/Users/macbook/wincent/roles/vim/tasks/main.yml': line 2, column 3, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

---
- import_tasks: bundle.yml
  ^ here

How can I fix this?

You probably just need "git submodule update" (not sure if you need "--recurse" or "--recursive" or something as well: I'm on my phone).

Done that. The error remains. git submodule status gives among other submodules the following
 cdec853e37d2342346f54e51f4ec4bfbd2ee0031 vendor/ansible (v<IP_ADDRESS>-1)

Can this be the cause?

That looks like the wrong rev. It's supposed to be pointed at https://github.com/ansible/ansible/tree/bf43eb92f50ea8edc9e7b704ef91b3121ec972f0
Ansible deprecated the old "include" directive in favor of "import_tasks" (and "include_tasks", if I recall correctly), so until you update the Ansible submodule I expect you'll continue to see that error.

I added branch=devel to .gitmodules, then git submodule update -- remote and the ./install -f worked. I guess I'm not that good at git.
Anyways, thanks.
PS I'm in the middle of your vim screencasts. After half a year of using your dotfiles they are a great addition! Thank you for such a great work!

Glad to hear you got it sorted out (although surprised it didn't just work the first time!).


[Feature] Auto highlight all text when double clicking system icons

Request:
Enable automatic select all if possible on text entry fields for the on map signatures.
Purpose:
Saves a few clicks to speed up the process of entering unique sig id names if used.

Sorry, did´t get your point. :( any example?

What I mean is this:
When you double click on a signature to update the naming info by default the cursor starts at the first character. What I am asking is if the behaviour so that the entire text in the text entry field is highlighted so we can just quickly copy paste info or just overwrite the entire field without having to hit CTRL+a.
http://i.imgur.com/MzOJtSo.png
Was an ask from my CEO

Ok, what do you think about this approach:
We could add a new context-menu option "copy information" (to clipboard). This would copy a string with more system information to clipboard. e.g. "J110043 (WH)  (occupied)" or something like this?

I like that option. Good for quick exports.
We don't use the jcodes in the signature names, so one reason we want to have the text entry field behaviour changed is to just make the adding sig names quicker.

sure, we could add the alias name as well in front to it: "My alias - J110043 (WH)  (occupied)"

Sure..
Only real change i wanted to convey at this time is the text box presentation on double clicking a signature.
Steps as per current

Enter new system and wormhole populates itself on map
double click the sig id to add the tag we want to have as the label
Have to select all in the text box for overwrite
change label

How my CEOs request would work

Populate sig
double click to enter text edit mode
Enter label with out having to select all before changing text


Make SourceMap infaillible

This is based on the work done in #2405.
By specifically marking the missing AST nodes and with a bit of refactoring, I was able to make the HIR->AST map infallible.
This required to remove the typing of AST nodes in the HIR->AST map. I believe this is a good thing, because it allows to record more precise information when desugaring.

Hm, various Ok(id) and .unwraps here make me think that we put type safety in the wrong place here. What do you think about https://github.com/rust-analyzer/rust-analyzer/pull/2405#issuecomment-558543297? Basically, use Result<_, MissingExpr> as a return-type of the source map, and not a the result of loweing.

I have another version where the parent nodes are tracked for missing expressions, as you suggest. The lowering code still needs a bit of cleanup, but it can already be put for comments.

This version tracks the parent node in order to provide a fallback mechanism. I still need to double check lowering, but tests pass right now.

Yeah, I think the general shape here is just right! However, I am not sure about many detail. I see that this adds a new type X = Either, but ideally we should move away from Eithers to explicitly named types.
I suggest, as a first step, just changing the return type of expr_syntax to Result<SyntaxPtr, ()>. Ie, we can just skip parent tracking altogether as a first step. This should break (rightlfully so) the ton of code that relies on ? for propagating Nones. By looking at those broken call-sites we should be able to see if the design makes sense. Ie, every call-site should be able to deal with Err without unwraps.

Actually, the modified places correspond to the changes in commit https://github.com/rust-analyzer/rust-analyzer/pull/2417/commits/cf57ce008de7c8629b75304f0b14e5b0c807b814. Some of them sould be made more robust (like ra_hir::code_model::Local::source), the other I don't really know.

Hm, that's really exciting!
Looks like all changes fall into two categories:

error reporting (where we want to ignore missing expressions, b/c the
errors were already reported)
Dealing with PatIds, which are guaranteed to point at Pat::Bind. Looks
like here is the opportunity to install some more type safety!

On Tue, 3 Dec 2019 at 20:16, Camille Gillot<EMAIL_ADDRESS>wrote:

Actually, the modified places correspond to the changes in commit cf57ce0
https://github.com/rust-analyzer/rust-analyzer/commit/cf57ce008de7c8629b75304f0b14e5b0c807b814.
Some of them sould be made more robust (like
ra_hir::code_model::Local::source), the other I don't really know.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/rust-analyzer/rust-analyzer/pull/2417?email_source=notifications&email_token=AANB3M7X4BL7LRJJMMCNPY3QW2V7DA5CNFSM4JR3M7YKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEF2PYRA#issuecomment-561314884,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANB3MYIGKL3UCSVO2HK7TDQW2V7DANCNFSM4JR3M7YA
.

Where would the information about the missing node's parent be needed? This type safety is precisely why I like #2405 better: once we have a PatId, nothing can fail.

Where would the information about the missing node's parent be needed?

I think only for debugging of rust-analyzer itself. Parent info is nice, because it makes sure that every Missing expression has a certain provenance, and not just synthesized out of thin air. OTOH, I now realize that correctly tracking parents would be a pain (b/c we get parent's ID only after we've constructed children: your typical cyclic data-structure problems!), so I am fine with using just a ZST as an "error" type.

This type safety is precisely why I like #2405 better:

Note that it won't be enough to make BindPats type safe.
Local::Source looks like this at the moment:
    pub fn source(self, db: &impl HirDatabase) -> InFile<Either<ast::BindPat, ast::SelfParam>> {
        let (_body, source_map) = db.body_with_source_map(self.parent.into());
        let src = source_map.pat_syntax(self.pat_id).unwrap(); // Hmm...
        let root = src.file_syntax(db);
        src.map(|ast| {
            ast
                .map_left(|it| it.cast().unwrap().to_node(&root)) // ! cast().unwrap()
                .map_right(|it| it.to_node(&root))
        })
    }

Notably, we use .cast().unwrap() to say "Hey, I know that this PatId is in fact a BindPatId, so it's source is not an arbitrary ast::Pat, but precisely ast::BindPat".
That is, it seems like in the places where we need type safety, "not missing" is not enough, we need "is a BindPat" (which is a subset of "not missing").

I see what you mean with having exactly a BindPat. Removing this unwrap is very tricky: the only proof we have that the id corresponds to a BindPat is in the Local::from_source expression.
Keeping this kind of information everywhere would require some kind of refinement type over PatId. A way forward is to introduce
struct PatIdFrom<T>(PatId)

impl SourceMap {
  pub fn node_pat_typed(&self, node: InFile<&T>) -> Option<PatIdFrom<T>>
    where T: Into<ast::Pat>
  {
    let id = self.node_pat(self, node.map(Into::into));
    PatIdFrom(id)
  }

  pub fn syntax_pat_typed(&self, id: PatIdFrom<T>) -> InFile<&T>
    where T: Into<ast::Pat>
  {
    let node = self.syntax_pat(id.0);
    node.map(|n| n.cast().unwrap())
  }
}

This moves the burden of correctness on lowering.

Exploration from this PR is now merged as #3489!


fix(iac): make "Invalid API key" error message consistent with secret scan

When calling ggshield iac scan with an invalid API key, the error message says:
status_code=401 detail=Invalid API key.
Usage: ggshield iac scan [OPTIONS] DIRECTORY
Try 'ggshield iac scan -h' for help.

Error: Invalid API key.

This is confusing because the user did not make any error with command-line arguments, so showing the command CLI help is not helpful. Furthermore, the first line is a less human-friendly format of the rest.
Rework this to match the error message printed by secret scan. Now it only says:
Error: Invalid API key.

Remove the logger.error() call because it duplicates what display_error() prints.

Codecov Report

Merging #496 (e35b628) into main (30484c0) will decrease coverage by 0.01%.
The diff coverage is 80.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main     #496      +/-   ##
==========================================
- Coverage   93.85%   93.84%   -0.01%
==========================================
  Files          78       78
  Lines        3610     3609       -1
==========================================
- Hits         3388     3387       -1
  Misses        222      222

Flag
Coverage Δ

unittests
93.84% <80.00%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

ggshield/cmd/iac/scan.py
98.61% <80.00%> (-0.02%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Apparently "var" inside a function inside a function isn't cleared each time

In testing changes for #451, inside "dateformat" using "var len" inside the nested function "finishRun" gives this error:
Error: Duplicate local variable name len not allowed at line 325.

This is the last thing that happens:
evaluate: push function FunctionScope@3f857e08, called from info.rlwhitcomb.calc.CalcObjectVisitor.visitVarExpr(Unknown Source)
FunctionVarContext: currentScope = FunctionScope@3f857e08
  [ _*,_#,ch,_funcname,len ]
evaluate: pop function FunctionScope@3f857e08
evaluate: pop function FunctionScope@5a090f62
Error: Duplicate local variable name len not allowed at line 325.

Notice that there is no "setupFunctionCall" in here, simply look up the id value, get a FunctionScope and execute it. Question is: how does that FunctionScope get saved under the function name??

The problem was the "leave" (without a value) exiting the function. This meant the "returnValue" from "evaluate" was left as-is, meaning the FunctionScope used to call the function, but which had already been called, so the symbol table was already setup, leading to the duplicate variable error.  Setting the return value to null if none was given on the "leave" statement fixed everything.


PWV Audit Reproducible Build Pipeline Suggested Changes

Reproducible Build Pipeline Audit/Suggestions
Updates made
The build system PR currently includes the following updates:

Updates the manifest file, so that only one language can be specified.
Created a Dockerfile.mustache which can be used as a template, and is compiled before building docker image.
Created a build manifest file which specifies required configuration variables, their types, and the shape of the payload expected.
Created a configuration file (web3api.build.yaml) which specifies the dockerfile to use, along with variables being sent to the mustache template for outputting the final dockerfile.

Pros

Highly customizable build process with customization available for any user via mustache templates allowing the Dockerfile to be updated easily.
Everything out of the box just works for developers and build should be the same across any system.
Any form of customization is saved locally in the repository, allowing for shared build processes with all people working on a project.

Cons

It is relatively complex, and may be daunting for some developers. Particularly those not familiar with Docker.
The templating of the Dockerfile can lead to confusing errors, as there is a middleware process that is building the actual Dockerfile process being used.
Limits build systems to a single language - IE you can't have some modules that are written in Rust, once support is available, and others written in AssemblyScript.
We find the configuration/manifest files to be verbose, and difficult to write by hand.

Concerns
Development Overhead
Docker can be daunting for some developers, and one of the goals of this project is to lower the necessary knowledge for developers to get up and running as quickly and efficiently as possible. However, we find the current system as implemented isn't very straight-forward.
The use of the Mustache templating engine is a bit difficult to read in the docker container. We feel the {{}} template tags feel verbose and make it difficult to understand what is happening. It also requires a lot of reverse engineering to understand how the Dockerfile is generated during the build process.
We also believe that writing the schema of the template context is difficult to write by hand. While JSON is leaps and bounds ahead of XML in this regard, we still think it's mostly a data format best handled by having a machine create/read it.
Recommendations/Considerations

As pointed out, environmental variables cause more headaches for this process than not. We recommend having a simple interpolation system has a very simple annotation. The annotation system will replace all of the mustache template engine that is currently developed. Our thought is something like the list below for the annotation system (feel free to change):

$varname for simple variables that need to be passed. Vars should be letters and numbers only to create a boundary and allow them to stand out more in the document. That way $nodeVersion-alpine only gets interpolated for the $nodeVersion.
$listname[] for a list of items that should be passed
$!varname or $!listname[] for a required field
After a list is defined, we need a !! to denote the end of the interpolation.
If a list is just a list of primitive values then $1 will denote it's value.
If a list is an object properties can be accessed via $prop_name within the list.

This annotation system allows for removal of the schema as well, as it's not part of the actual docker file. It also follow rules similar to GraphQL which is used heavily in the repository, so developers should feel relatively familiar and comfortable with it. In the end the docker file would look something like this:
FROM node:$!nodeVersion-alpine as base

RUN apk --no-cache --virtual build-dependencies add \
    bash \
    git \
    openssh \
    make \
    g++

WORKDIR /project

# Install deps in its own step, making rebuilds faster
# when just the Web3API schema & implementation files change
COPY package.json .
RUN yarn

# Copy all manifest files
$web3apiManifests[]
COPY $1 .
!!

Another alternative to this format to simplify the file would be to remove the need for an end indicator for lists - so no !!. This would allow for a single line only, but since most of these are bash commands using the RUN module you can simply concatenate the commands with &&, or use the same list variables twice. Example below:
FROM node:$!nodeVersion-alpine as base

RUN apk --no-cache --virtual build-dependencies add \
    bash \
    git \
    openssh \
    make \
    g++

WORKDIR /project

# Install deps in its own step, making rebuilds faster
# when just the Web3API schema & implementation files change
COPY package.json .
RUN yarn

# Copy all manifest files
$web3apiManifests[]
COPY $1 .

# Copy all source files
$!include[]
COPY $1 $1

# Run some command on all includes to copy to another folder
$!include[]
RUN cp $1 /usr/bin/$1 && chmod /usr/bin/$1

$web3apiModules[]
COPY $dir $dir

$web3apiModules[]
# Build the module at $dir
RUN ./node_modules/.bin/asc $dir/w3/entry.ts \
    --path ./node_modules \
    --outFile ./build/$name.wasm \
    --use abort=$dir/w3/entry/w3Abort \
    --optimize --debug --importMemory \
    --runtime stub

Since we can now define whether or not something is required within the actual Dockerfile template, we don't see the need for the additional schema file. The types seems not so important, and would be easier for the developer to identify any issues that occur by viewing the interpolated Dockerfile.
If necessary we could at minimum remove the web3api.build.json file, and implement needed functionality in the YAML manifest file instead. This removes the need for writing JSON by hand for the developers to YAML which is preferred in our opinion, and also removes the additional file. But again, unless types are really an important factor here it doesn't seem like there is much benefit from doing it with the annotation system implemented.
A good example of this is the Swagger implementation. This would allow us to add a property to this file called schema which specifies whether something is required and not and the types. Below is an example:
  ...
schema:
  SomeObject:
    type: object
    required:
      - name
      - totalOrders
    properties:
    name:
      type: string
    totalOrders:
      type: integer
  ...

I think this can be closed and (probably) re-ask for this with the new changes introduced in #986


Fix typos

Fix small typos from cli.rs.

Codecov Report

Merging #43 (28ccb8f) into master (436d808) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #43   +/-   ##
=======================================
  Coverage   68.03%   68.03%
=======================================
  Files          12       12
  Lines        1142     1142
=======================================
  Hits          777      777
  Misses        365      365

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 436d808...28ccb8f. Read the comment docs.


[* DateTimeV2] Support for dates after 31st December 2099

I would like to see support for dates that are greater than 31st December 2099
We have the requirement to recognise dates this far in advance as we are dealing with property leases of longer periods

@IanBoggs, how far in the future are you looking into? Unfortunately due to legacy reasons we currently have to  define a cutoff for mentions of the year alone by itself.

@tellarin, should we proceed with this one? I tried to set the cutoff at 3100 and the tests do no show negative effects.

My apologies, I must have missed the original question. I think I'd be looking at at least 999 years from now in my application, but would probably need a bit of overhead above that.


[Adapter] Spec it well

adapaters are used for hub to hub communication

methods

subscribe

unsubscribe

write

push

connect (needs to accept a clientid field, also needs a reconnect / tries option (but later))

disconnect

we need to be able to pass an instanceID (e.g. user.token)

and in every method if no instanceID is passed we can create one and return it

adapter has live on the upstream level (multiple upstreams per hub)

protocols have to be injectable

module.exports = new Hub({
  adapter: {
    inject: require('../../lib/adapter/websocket'),
    on: {
      connection () {
        console.log('\n',
        uuid.green.bold,
        'connected',
        '\n    url:', this.val
        )
      },
      error (err) {
        console.log(err)
      },
      close () {
        console.log('o noes!'.red)
      },
      listens (data) {
        console.log(
          '\n',
          uuid.green.bold,
          'listening',
          '\n    protocol:', data,
          '\n    port:', this.listens.val
        )
      }
    }
  },
  clients: {
    on: {
      property: require('./dev').clients
    }
  },
  on: {
    data: {
      performance: require('./dev').performance,
      log: require('./dev').data
    }
  }
})

example of a server can be found in examples/basic

https://github.com/juliangruber/reconnect-core  -- may be a good idea for reconnection

testing the modules reconnect-core and retry

need to check for different adapters on different instances -- may prove problematic


Upgrading Prometheus 2.0 from beta2 to beta5 fails to start.

What did you do?
Updated from v2.0.0-beta.2 -> v2.0.0-beta.5
What did you expect to see?
I expected prometheus beta5 to start properly.
What did you see instead? Under which circumstances?
beta5 failed to start complaining about missing lock file.  tried to touch a lockfile and then it complained about a missing chunks dir.  removed my touched lock file and reverted back to beta2 and things look fine again.
Environment
k8s 1.7.4 on bare-metal

System information:
Linux 4.9.13-1 x86_64

Prometheus version:

prometheus, version 2.0.0-beta.5 (branch: HEAD, revision: 64c7932e851f3276391fc3e216cd30f447d4924e)
build user:       root@df40cc0bd79b
build date:       20170921-16:23:04
go version:       go1.8.3

prometheus, version 2.0.0-beta.2 (branch: HEAD, revision: a52f082939a566d5269671e98be06fc6bdf61d09)
build user:       root@41a0740ea598
build date:       20170818-08:16:50
go version:       go1.8.3

Logs:

level=error ts=2017-09-24T23:40:02.426894698Z caller=main.go:317 msg="Opening storage failed" err="open block /prometheus/data/01BTTYEYSJJFT86B8X7286ETW9: try lock file: open /prometheus/data/01BTTYEYSJJFT86B8X7286ETW9/index: no such file or directory"

level=error ts=2017-09-24T23:38:06.766703158Z caller=main.go:317 msg="Opening storage failed" err="open block /prometheus/data/01BTTYEYSJJFT86B8X7286ETW9: open /prometheus/data/01BTTYEYSJJFT86B8X7286ETW9/chunks: no such file or directory"`

It's recommended to read the release notes before attempting an upgrade. The beta.4 release is incompatible with previous versions. https://github.com/prometheus/prometheus/releases


fix: changes ConvertToAssociationSet methods to allow for duplicate c…

…hild manifest
Currently, if two images with an index manifest contain the same child manifest,
the conversion to an AssociationSet set fails because child manifest records are being
deleted after processing to identify parent image manifests. This change adds the child manifest
to a visited map and checks if it exists instead to allow for duplicate child manifests.
Fixes #402
Signed-off-by: Jennifer Power<EMAIL_ADDRESS>Description
^^
Type of change
Please delete options that are not relevant.

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

[X] Unit test added to test for shared child manifests

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

cc @ianwatsonrh

/lgtm


Fix broken HEAD

PR #672 was based off HEAD which didn't yet have PR #510, and the former PR happened to make an incompatible change to an API (verifyReducerShape) used by the latter PR.
This PR fixes the incompatibility and fixes HEAD.

Landing without approvals, because it's pretty late in Pacific Time, and we need to unbreak HEAD as soon as possible.


hidpp20: use macros for key+modifiers

Since a0830a8aaaa5168f4a2bb1c0d7594de50fcb9fb0, we can re-add the simple keybindings capabilities. It's time the HID++ 2.0 devices get this.
This used to be working with the KEY API, but this has been dropped in favor of the macro one.
This effectively replaces #402

Sorry for taking so long to look into this. I had to deal with macros for properly fixing #380, and I realized the code was now very simple to add.

merci


simplify code & correct some typo mistakes

Signed-off-by: kevinetc123<EMAIL_ADDRESS>

Codecov Report

Merging #536 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #536   +/-   ##
=======================================
  Coverage   62.12%   62.12%
=======================================
  Files           3        3
  Lines         404      404
=======================================
  Hits          251      251
  Misses         93       93
  Partials       60       60

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56460b9...2b455ea. Read the comment docs.

Maybe, make the error message more helpful?

@stevvooe  I have replaced the stdlib errors with pkg\errors and added some help message for that error.

LGTM


Cannot save memory during the FP

Hi, part of my work now follows the project and I try to quantize the activation during the forward pass.
However, i just noticed that though i can modify the ctx.saved_tensor to be my new compressed activation, the overall cuda memory occupation doesn't decrease even increase.
What i found is that the original fp32 activation will not be freed and still be count as part of the cuda memory usage. I just wonder the reason behind this and seeks for a solution.
For your reference, what i did is like
def forward(self, input):
      qconv2d.apply(input, weight, .....)

Where inside the qconv2d, i did
input_int8, scale_inp = quantize_int8(input)
...
ctx.save_for_backward(input_int8, scale_inp, weight, bias)

However, using the torch.cuda.memory_allocated(0), the input & input_int8 will be both saved during the path?
Hoping for your reply.

Try to reproduce our results and check the difference between your code and our code
Make sure there is no other reference to your fp32 activation tensors, so they can be freed by the garbage collector.

Try to reproduce our results and check the difference between your code and our code
Make sure there is no other reference to your fp32 activation tensors, so they can be freed by the garbage collector.

Yeah i can reproduce the mem results of ActNN by direct run the sample you provided. And it turns out the cuda allocated memory result exactly follows the compression rate. However, when i extract a single layer out and use the quantization equation you provided, it turns out the 8-bit version will have a higher allocated memory occupation.
I got a result for fp32 - quantized, scale = input, 0 as:

While int8 quantized, scale, q_scale, q_min = quantize_activation(input) or my personal int8 quantization method quantized, scale = quantize_int8(input) with:

Do i miss / misunderstand something in this part that makes the result wrong?
My code as follows:
import torch.nn as nn
import torch
import torch.nn.functional as F
from utils import get_memory_usage
from quant import quantize_int8, dequantize
from torch.autograd import Function

import actnn
import actnn.cpp_extension.quantization as ext_quantization
import actnn.cpp_extension.minimax as ext_minimax

def quantize_and_pack(data, bits, mn, mx):
    # Pack to bitstream
    if isinstance(bits, int):
        pack_func = ext_quantization.pack_single_precision
        print("here")
    else:
        pack_func = ext_quantization.pack_mixed_precision
    # import pdb; pdb.set_trace()
    output, scale = pack_func(data, mn, mx, bits, True)

    return output, scale

def no_scheme_compute_quantization_bits(input):
    N = input.shape[0]
    D = input.shape[1]
    input_flatten = input.view(N, -1)
    num_features = input_flatten.shape[1]
    num_pixels = num_features // D

    input_groups = input_flatten.view(-1, 256)
    mn, mx = ext_minimax.minimax(input_groups)
    # import pdb; pdb.set_trace()
    b = 8
    return input_groups.view(N, -1, 256), b, mn.view(N, -1, 1), mx.view(N, -1, 1)

def quantize_activation(input, scheme=None):

    N = input.shape[0]
    if scheme:
        input_groups, q_bits, q_min, mx = scheme.compute_quantization_bits(input)
    else:
        input_groups, q_bits, q_min, mx = no_scheme_compute_quantization_bits(input)

    q_input, q_scale = quantize_and_pack(input_groups, q_bits, q_min, mx)

    # TODO convert q_bits to int8
    if input.dtype == torch.float32:
        return q_input, q_bits, q_scale.to(torch.bfloat16), q_min.to(torch.bfloat16)
    else:
        return q_input, q_bits, q_scale, q_min

class convnd(Function):
    @staticmethod
    def run_forward(n, forward_op, ctx, input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1, scheme=None):
        # if not ctx.needs_input_grad[1]:
        #     assert not ctx.needs_input_grad[0] and not ctx.needs_input_grad[1]
        #     return F.conv2d(input, weight, bias, stride, padding, dilation, groups)
        # quantized, scale, q_scale, q_min = quantize_activation(input)
        # quantized, scale = quantize_int8(input)
        quantized, scale = input, 0
        # ctx.scheme = scheme

        ctx.saved = quantized, weight, bias
        ctx.other_args = (input.shape, stride, padding, dilation, groups)
        ctx.scale = scale

        torch.cuda.empty_cache()
        get_memory_usage(True)

        return forward_op(input, weight, bias, stride, padding, dilation, groups)

    @staticmethod
    def run_backward(n, ctx, grad_output, bias_reduce_dims, aug):
        # if not ctx.needs_input_grad[1]:
        #     assert not ctx.needs_input_grad[0] and not ctx.needs_input_grad[1]
        #     return None, None, None, None, None, None, None, None
        q_input_shape, stride, padding, dilation, groups = ctx.other_args
        padding = aug(padding)
        stride = aug(stride)
        dilation = aug(dilation)
        scale = ctx.scale

        quantized, weight, bias = ctx.saved
        input = dequantize(quantized, scale)
        del quantized, ctx.saved

        torch.cuda.empty_cache()

        # get_memory_usage(True)

        grad_input, grad_weight = ext_backward_func.cudnn_convolution_backward(
            input, grad_output, weight, padding, stride, dilation, groups,
            True, False, False,
            [ctx.needs_input_grad[0], ctx.needs_input_grad[1]])

        if bias is not None and ctx.needs_input_grad[2]:
            grad_bias = grad_output.sum(bias_reduce_dims)
        else:
            grad_bias = None

        return grad_input, grad_weight, grad_bias, None, None, None, None, None

class conv2d(Function):
    @staticmethod
    def forward(ctx, input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1, scheme=None):
        return convnd.run_forward(2, F.conv2d, ctx, input, weight, bias, stride, padding, dilation, groups, scheme)

    @staticmethod
    def backward(ctx, grad_output):
        return convnd.run_backward(2, ctx, grad_output, [0, 2, 3], _pair)

class QConv2d(nn.Conv2d):
    def __init__(self, in_channels, out_channels, kernel_size,
                 stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', group=0):
        super(QConv2d, self).__init__(in_channels, out_channels, kernel_size,
                                      stride, padding, dilation, groups, bias, padding_mode)

    def forward(self, input):

        return conv2d.apply(input, self.weight, self.bias, self.stride,
                                self.padding, self.dilation, self.groups, None)

get_memory_usage()
qconv2d = QConv2d(3,3,7).cuda()
inp = torch.rand(32,3,224,224).cuda()
inp.required_grads = False
before_ = get_memory_usage()
loss = qconv2d(inp)
after_ = get_memory_usage()
print(
    "Act", (after_ - before_) / 1024 / 1024
)

Many thanks for your reply!

FYI, i finally solved the problem by replacing all the non-gemm function to the version that provided by actnn. (ReLU, BatchNorm & Maxpool, etc.) There is a wierd phenomenon that, when using the model provided by the torchvision, torch seems to optimize its fp32 memory allocation during the forward propagation in these ops. :)
Again, greatly appreciate the help you provided!


Run locust using a schedule from yaml file and RampUpRateLimiter not working as expected

yaml file being read into locustConfig:
uiMaxRPS: 20
autoRun: true
steps:

numUsers: 10
hatchRate: 1
durationInSec: 30
maxRPS: 20
name: Initial Load
numUsers: 15
hatchRate: 2
durationInSec: 60
maxRPS: 30
name: Ramp up
numUsers: 20
hatchRate: 3
durationInSec: 90
maxRPS: 40
name: Peak Load

DurationInSec - indicates how long should the step be run in seconds
NumUsers - indicates the maximum number of users to spawn
HatchRate - indicates how fast each user can be spawn per second
MaxRPS - indicates max RPS for the step
Name - a label to show which step the schedule is running
I am trying to implement a scenario where if autoRun is true then the run of locust is automatic based on the details from Step in yaml config file and if possible the run is visible on the locust web UI or at least I still get the statistics report in a presentable form .
When autoRun is false then the user gets to control the NumUsers  and HatchRate from Locust Web UI. The false flow is working very well but for autoRun true neither the task execution is happening as expected nor I see anything on locust web UI nor the execution stops after steps completion.
Below is code I used:
 if (locustConfig.getAutoRun()) {
            for (LocustConfig.Step step : locustConfig.getSteps()) {
                System.out.println("Executing step : "+ step.getName() + "\n MaxRPS: " +step.getMaxRPS()+ "\n HatchRate: " +step.getHatchRate()+ "\n Duration: " +step.getDurationInSec());
                locust.setRateLimiter(new RampUpRateLimiter(step.getMaxRPS(), step.getNumUsers(), step.getHatchRate(), TimeUnit.SECONDS, step.getDurationInSec(), TimeUnit.SECONDS));
                tasks.clear();
                tasks.add(new GetHandshakeApiTask(1, getHandshakeApi));
                tasks.add(new GetValidateUserLicenseApiTask(1, getValidateUserLicenseApi));
                locust.run(tasks);

            }
        }  else {
            locust.setMaxRPS(locustConfig.getUiMaxRPS());
            tasks.add(new GetHandshakeApiTask(1, getHandshakeApi));
            tasks.add(new GetValidateUserLicenseApiTask(1, getValidateUserLicenseApi));

            locust.run(tasks);
        }

        Runtime.getRuntime().addShutdownHook(new Thread(() -> {
            locust.stop();
            Stats.getInstance().stop();
         }));

    }
}

Please help me with this implementation.

Because locust.run will block, so only the first step works.

How will first step know which tasks to execute then?

I think you can refer to taskset to control multiple tasks, and implement your own taskset, and the taskset can change the ratelimiter at runtime.

I tried multiple ways but nothing gave the desired outcome. Can you explain bit more. My understanding is even task set needs to be run via locust

AbstractTaskSet taskset = new TaskSetThatSupportSteps()

// create a timer in another thread
timer.run(
      for (LocustConfig.Step step : locustConfig.getSteps()) {
             locust.SetRateLimiter(xxxx);
             taskset.clear()
             taskset.add(other tasks)
      }
)

locust.run(taskset)


Calculating MOD at initialization

[ ] Tests pass
[x] Appropriate changes to README are included in PR

Initializing ModInt with a negative number or a value greater than MOD does not work as expected.
I have changed it to calculate the remainder at initialization.
alias ModInt = AtCoder::ModInt1000000007

ModInt.new(-1) # => -1 (Expected:<PHONE_NUMBER>)
ModInt.new(1_000_000_007_i64 * 2) # =><PHONE_NUMBER> (Expected: 0)

Sorry, I modified it to avoid overflow.
alias Mint = AtCoder::ModInt1000000007
Mint.new(-1) * Int64::MAX # => 708828004 (before: OverflowError)


Tutorial on Docker usage

Tutorial requested by hackathon participants on common docker commands that users will need frequently
docker exec, docker cp, docker ps, etc.
Also what is dockerhub, when they would need an account, what are tags in docker

Debug app in container.
How the register app work and what is going on there.
How to log
How to get container name
How to use the docker logs --follow or --tail command
How to use docker cp


Suppress warnings for directories

If you call unlink_if_exists with some wildcard that happens to include some directory(ies) then it will give warnings about "operation not permitted" when the array_map() unlink is attempted on the directories.
Putting "@unlink" directly in the array_map() does not work.
So I have made unlink_silent() as a wrapper for @unlink().
This will fix the warnings reported in Redmine #5613 and for anything else calling unlink_if_exists() in similar circumstances.

Maybe this is the better more general solution than https://github.com/pfsense/pfsense/pull/2303
Although both together also works fine.

Done, thanks!


Improved detection of nullability for postgres

This query currently returns an Option<i64>. It would be nice to instead return i64, because COUNT is never nullable.
async fn f() -> Result<i64, sqlx::Error> {
    sqlx::query!(
        "SELECT COUNT(*) FROM crates",
    )
    .fetch_one(unimplemented!())
    .await
    .map(|rec| rec.count)
}

error[E0308]: mismatched types
  --> src/main.rs:6:5
   |
6  | /     sqlx::query!(
7  | |         "SELECT COUNT(*) FROM crates",
8  | |     )
9  | |     .fetch_one(unimplemented!())
10 | |     .await
11 | |     .map(|rec| rec.count)
   | |_________________________^ expected `i64`, found enum `std::option::Option`
   |
   = note: expected enum `std::result::Result<i64, _>`
              found enum `std::result::Result<std::option::Option<i64>, _>`

@mehcode mentioned that I can give SQLx a type hint by using r#"SELECT COUNT(*) as "count!" FROM crates;"#; this is a good workaround in the meantime but I'd love for sqlx to be able to figure it out itself. A possible implementation is to select from a temporary table or view and get the nullability from that view.

Another (related?) issue: sqlx thinks this is an Option<i32>, not an i32.
query!(
        "INSERT INTO doc_coverage (release_id, total_items, documented_items)
            VALUES ($1, $2, $3)
            ON CONFLICT (release_id) DO UPDATE
                SET
                    total_items = $2,
                    documented_items = $3
            RETURNING release_id",
        release_id,
        doc_coverage.total_items,
        doc_coverage.documented_items,
    )
    .fetch_one(conn)
    .block()?
    .release_id

@jyn514 can you provide a explain verbose for that INSERT INTO ... RETURNING release_id query? A minimal reproduction with schema would be helpful for testing.
 At the very least we would need a basic expression tree parser and to hardcode knowledge of various functions and their outputs. There's nothing in pg_proc that tells us definitively whether a function may or may not return null.

cratesfyi=# explain verbose SELECT COUNT(*) FROM crates;
                             QUERY PLAN
---------------------------------------------------------------------
 Aggregate  (cost=10.88..10.88 rows=1 width=8)
   Output: count(*)
   ->  Seq Scan on public.crates  (cost=0.00..10.70 rows=70 width=0)
(3 rows)

cratesfyi=# explain verbose INSERT INTO doc_coverage (release_id, total_items, documented_items)
cratesfyi-#             VALUES (1, 10, 7)
cratesfyi-#             ON CONFLICT (release_id) DO UPDATE
cratesfyi-#                 SET
cratesfyi-#                     total_items = 10,
cratesfyi-#                     documented_items = 7
cratesfyi-#             RETURNING release_id;
                            QUERY PLAN
------------------------------------------------------------------
 Insert on public.doc_coverage  (cost=0.00..0.01 rows=1 width=20)
   Output: doc_coverage.release_id
   Conflict Resolution: UPDATE
   Conflict Arbiter Indexes: doc_coverage_release_id_key
   ->  Result  (cost=0.00..0.01 rows=1 width=20)
         Output: 1, 10, 7, NULL::integer, NULL::integer
(6 rows)

cratesfyi=# \d doc_coverage
                       Table "public.doc_coverage"
            Column            |  Type   | Collation | Nullable | Default
------------------------------+---------+-----------+----------+---------
 release_id                   | integer |           |          |
 total_items                  | integer |           |          |
 documented_items             | integer |           |          |
 total_items_needing_examples | integer |           |          |
 items_with_examples          | integer |           |          |
Indexes:
    "doc_coverage_release_id_key" UNIQUE CONSTRAINT, btree (release_id)
Foreign-key constraints:
    "doc_coverage_release_id_fkey" FOREIGN KEY (release_id) REFERENCES releases(id)

There's nothing in pg_proc that tells us definitively whether a function may or may not return null.

That's unfortunate ... I don't want to make you add a full parser (I know one of the goals of the proc-macros is to avoid having to do that), so if this is too hard to support it's not a blocker for us.


User.language_code described at IETF but only being used as ISO-639

The bot api list the User.language_code as IETF, at https://core.telegram.org/bots/api#user, but at least from what I have seen, it is only using ISO 639.
Could someone proof that is being used as IETF and not only ISO-639, if not, can we change the description to being ISO-639?
JsonUser class:
auto user_info = client_->get_user_info(user_id_);

https://github.com/tdlib/telegram-bot-api/blob/5951bfbab8b1274437c613c1c48d91be2a050371/telegram-bot-api/Client.cpp#L406-L408
get_user_info:
https://github.com/tdlib/telegram-bot-api/blob/5951bfbab8b1274437c613c1c48d91be2a050371/telegram-bot-api/Client.cpp#L12301-L12303
Where is the users_ symbol or it's interface define? haven't found it.
https://github.com/tdlib/telegram-bot-api/blob/5951bfbab8b1274437c613c1c48d91be2a050371/telegram-bot-api/Client.cpp#L195-L198

yes, it is indeed used as IETF

Thanks! Where did you get that list from?

from the database of one of my bots


1963-minimum-number-of-swaps-to-make-the-string-balanced.java

File(s) Modified: 1963-minimum-number-of-swaps-to-make-the-string-balanced.java
Language(s) Used: java
Submission URL: _https://leetcode.com/problems/minimum-number-of-swaps-to-make-the-string-balanced/submissions/860973441/_

Thanks!


On azure, clone DT from a zip

We've had many problems in the past due to the DefinitelyTyped repository on azure getting corrupted. This would make it so that there's no longer any state that needs to be kept between types-publisher runs (though keeping the cache directory around improves performance) -- we always get a fresh ZIP file instead. On my laptop this takes under 30 seconds.

If merged will this resolve https://github.com/Microsoft/types-publisher/issues/485?
https://www.npmjs.com/package/@types/recharts is still not published; I imagine others are affected too


[CodeCompletion] Complete pattern introducer for 'for'

After for, suggest try, await, var and case. Introduced CompletionKind::ForEachPatternBeginning.
rdar://76355581

@swift-ci Please smoke test


Keyboard text alignment

Description
Add HorizontalTextAlignment="Right" in the style of KeyboardAcceleratorTextLabel for CommandBarFlyoutAppBarButtonStyleBase, CommandBarFlyoutAppBarToggleButtonStyleBase, DefaultAppBarButtonStyle, DefaultAppBarToggleButtonStyle, MediaControlAppBarButtonStyle and their uses cases.
Add HorizontalTextAlignment="Right" in the style of KeyboardAcceleratorTextBlock for DefaultMenuFlyoutItemStyle, DefaultToggleMenuFlyoutItemStyle, MenuFlyoutItemRevealStyle, ToggleMenuFlyoutItemRevealStyle, DefaultRadioMenuFlyoutItemStyle and theiruse cases.
Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

The only thing to test I can think of would be right to left flow. @karenbtlai  is there an easy way to test this in MUX solution?

The only thing to test I can think of would be right to left flow. @karenbtlai is there an easy way to test this in MUX solution?

I've only tested in the current UWP app we are working on, we basically copied the the whole styling and added the HorizontalTextAlignment="Right" which effectively made the text align to the right.

/azp run

it's actually quite easy to test right to left (RTL) by adding FlowDirection=RTL to the test panel containing the test items. I ran this change on a bunch of controls and they look good

.

/azp run

Can we please run the tests again? I had the compiler error "Member 'HorizontalTextAlignment' on type 'Windows.UI.Xaml.Controls.ITextBlock6' is defined under contract 'Windows.Foundation.UniversalApiContract' version '<IP_ADDRESS>', but the contract version for the targeted min version is '<IP_ADDRESS>'!" so I think I have to remove the HorizontalTextAlignment on Textblocks without contract6Present in my latest commit. Please tell me if there's a better way to resolve this issue so I can keep the HorizontalTextAlignment attribute.

You can add contract constraints to properties as well.  For example, you can do this:
xmlns:contract5Present="http://schemas.microsoft.com/winfx/2006/xaml/presentation?IsApiContractPresent(Windows.Foundation.UniversalApiContract,5)"

<TextBlock xmlns:contract5Present:HorizontalTextAlignment="Right" />

That should allow you to put it everywhere.

Can we please run the tests again? I had the compiler error "Member 'HorizontalTextAlignment' on type 'Windows.UI.Xaml.Controls.ITextBlock6' is defined under contract 'Windows.Foundation.UniversalApiContract' version '<IP_ADDRESS>', but the contract version for the targeted min version is '<IP_ADDRESS>'!" so I think I have to remove the HorizontalTextAlignment on Textblocks without contract6Present in my latest commit. Please tell me if there's a better way to resolve this issue so I can keep the HorizontalTextAlignment attribute.

You can add contract constraints to properties as well. For example, you can do this:
xmlns:contract5Present="http://schemas.microsoft.com/winfx/2006/xaml/presentation?IsApiContractPresent(Windows.Foundation.UniversalApiContract,5)"

<TextBlock contract5Present:HorizontalTextAlignment="Right" />

That should allow you to put it everywhere.

Thank you! I put contract5Present before every HorizontalTextAlignment I added. I'm not very familiar yet with xaml contract versions, hope this time it will build fine :)

/azp run

Any news?

Bump - is there anything holding back the merge?


Remove Hard Dependency, Expanded Dependency, Added Contact Info, Updated Mod Version Number

I had someone looking to use MC Dungeons Weapons with your mod, but there was a conflict with the Fabric versions. Upon looking at your mod, I noticed that the Fabric Dependency was hard coded, so this PR will make it so that it can accept a range of Fabric Versions. I also updated the mod version to 3.0.1 since that appeared to be the schema that you had used in the past for minor changes.
It also allows for the mod to be loaded, potentially, in 1.19.1, assuming that no code needed to be changed. I have not tested that, but it will continue to load in 1.19 without issue.
I also added contact information to the fabric.mod.json file so that if there are any issues with loading the mod, the Fabric Loader should provide the link to the end user.
Thanks, so much, for your awesome contribution to the Minecraft Modding Community! I hope you have an amazing day/night!

Hi
Thanks for the PR. I'll merge it then.


When revisiting brouter, layers et al. are stored, but not the bike type

I noticed, that layers and so on are stored, however not the type of the bicycle: the memo function (cookie? local storage?) should be harmonised and store all user-changes options/parameters.

In addition to the profile itself, any customizations are not stored in cookies either, and have to be re-done every time the page is opened, which is a little tiring.

In addition to the profile itself, any customizations are not stored in cookies either, and have to be re-done every time the page is opened, which is a little tiring.

BRouter-web doesn't store anything to cookies, that's right.
The following parameters are saved to local storage (and recalled on re-initialization) nevertheless:

active map layers
selected map layers
map view (center, zoom)
route opacity value
map overlay opacity value
active sidebar
favorited search queries
language code used for localizations

I've build support for remembering the active routing profile for my instance at bikerouter.de and will submit a pull request in the next days.


Easier argument passing

Taking an example from the readme:
trace -d -mread_b -mwrite_b -- attach $(pidof cp)

It would be much nicer if you could write this as:
trace -d -m read_b write_b -- attach $(pidof cp)

The fact is that order matters for metrics and the latter notation doesn't play nice with mixed expression types:
trace -d -mtotal_mem_kb -x'ts date +%s' -mcpu_load_perc -- run sleep 5


UDF with one more return type can't be registered to gluten

Backend
VL (Velox)
Bug description
In UDF register to gluten, using unordered_map{udf_name, udf_return_type} to register the udf to gluten, as follows:

so, in this method, udf registered to gluten can only using one return type. but if I have a UDF that int myudf(int, int) and string myudf(string, int).if so, I can't register one udf with one more return type

CC @marin-ma

done

done


Sugestion - 1)Prefixes and sulfixes,  2) a way to turn off scoreboard loading

One thing i find missing on this plugin is a way to edit the player name, but using an ingame value, like "[LV] PlayerName"
Or even something like "Playername [LV]".
Basically a way to append an title or achivement of the players (specially if using PlaceholderAPI)

Another thing i'm not sure if there's a way but i think it should... A way to completely turn off the data gathering from the scoreboard file, i tried those configurations (below) but the ip:port/stats.json still populated with data from the scoreboard file...
objectives: []

objectives:
#  - '*'

Thanks for the suggestion! Actually I'm surprised nobody suggested that before. I think the best way may be to use one of the placeholder values or scoreboard values for the display name (for instance the %player_displayname%). Would that work for you?

objectives: [] should work fine, I just tested. Make sure you either restart the server or run /webstats reload when you change the config. An empty objectives: does indeed not seem to work, probably that's just how Spigot's yml parser works.

Well, maybe... But is it possible to make something just like the placeholders? because i loved how i can configure and use as many placeholder as i want in a single variable.
Something like that:

PlayerNameOverride: '[%player_displayname%] %player_gamemode%'

I it would show up something like [Player] Survival
That way it might be easier to customize later on without relying on other plugins to do the job

So, i might have a visual bug?
i'm currently with the config

objectives: []

but running /webstats debug show up this message:
[08:41:24 INFO]: WebStats 1.9.1
[08:41:24 INFO]: Active sources: scoreboard, PlaceholderAPI

and checking the ip:port/stats.json it is still populating the "scoreboard":{"entries":[ (...) (i might be wrong, but now it seems like a normal behaviour due the PlaceholderAPI loading)

You can actually already use multiple placeholders in a single column. The following will show an extra column with the player's displayname and gamemode:
placeholders:
  '[%player_displayname%] %player_gamemode%': Player

I think it'll be an easy solution if WebStats will automatically display a column named "Player" to be the player's name.

The scoreboard key in the JSON data is actually for all data, including PlaceholderAPI. Its name is a remnant of early versions of WebStats that used to only support the ingame scoreboard. With your config, there should not be any scoreboard objectives present in the JSON, or in the displayed table.
The fact that the scoreboard source is still loaded when objectives: is commented out entirely is a small bug, but that should not matter here.

If making a new column named "Player" replace the default "Player" one, it would be perfect.
I guess the same would work for a Server table, instead of showing up just "Server" we could make up anything with Placeholders...

I've been thinking about your suggested improvements for server stats. Here's what I will probably do:

Add a separate server-name config item next to server-columns where you can use placeholders. This will replace the server's name in the first column (currently "Server").
Display the server's icon from the server list as the server's "player-head."
For tables that only display the server's stats and not also player's stats, change the first column's name from "Player" to "Name" (I'm not sure about this one).
While at it, display the server's status as "online" because it can never be afk or offline.

These improvements seems to be very nice!
Also, if the webpage is hosted somehow apart from the server itself, it is possible for the server to be offline and the page online... But just a too specific and rare possibility...

That is true, but when the server is offline there won't be a row with stats for the server to display (unless the server went offline while you have the page open :)