Explore DMA-based APIs built on top of &'static mut references

Summary of this thread as of 2018-04-02
Current status of the DMA API design:
I think we have a clear idea of what the API should provide:
It MUST

be memory safe, obviously
expose a device agnostic API, so we can use it to build DMA based HAL traits
support one-shot and circular transfers
support mem to peripheral and peripheral to mem transfers
not require dynamic memory allocation, i.e. allocator support should be optional

It SHOULD support these features:

Canceling an ongoing transfer
Memory to memory transfers
Peeking into the part of the buffer that the DMA has already written to
Data widths other than 8-bit
Using a part / range of the buffer; the API still needs to take ownership of the whole buffer in this scenario

This is out of scope:

Configuration stuff like enabling burst mode or constraining usage of a channel to certain
peripherals. The API for this functionality is up to the HAL implementer to provide / design.

What we know so far from existing implementations:

Ownership and stable addresses are a must for memory safety

(remember that mem::forget is safe in Rust)
The ongoing transfer value must take ownership of the buffer and the DMA channel
storing [T; N] in the Transfer struct doesn't work because the address is not stable -- it
changes when the Transfer struct is moved
&'static mut fulfills these two requirements but so do Box, Vec and other heap allocated
collections. See owning_ref::StableAddress for a more complete list.

These concepts need to be abstracted over (using traits or structs):

An ongoing DMA transfer
A DMA channel, singletons make sense here
Buffers compatible with DMA transfers. cf. StableAddress. Also the element type must be
restricted -- e.g. a transfer on &'static mut [Socket] doesn't make sense.

Unresolved questions:

Alignment requirements. Does a transfer on a [u16] buffer require the buffer to be 16-bit
aligned? The answer probably depends on the target device.

Attempts at the problem:

Memory safe DMA transfers, a blog post that explores using
&'static mut references to achieve memory safe DMA transfers

stm32f103xx-hal, PoC implementations of the idea present in the blog post. It contains
implementations of one-shot and circular DMA transfers

myeisha's DMA framework. Supports several heap
allocated collections like Box and Vec.

What the title says. This blog post describes the approach to building such APIs. The last part of
the post covers platform agnostic traits that could be suitable for inclusion in embedded-hal.
This issue is for collecting feedback on the proposed approach and deciding on what should land in
embedded-hal.

Great stuff. I like how you're tackling DMA. However I feel there's one very important piece missing in the embedded-hal world: The ability to tie interrupts to HAL periperals.
You nicely described how DMA gets rid of the need to wait for the transfer to complete on the spot but you still need to spend (potentially) a lot of cycles to check whether your transfer has completed at some point. Whether that is better or not depends on the application, after all you still might do other processing in interrupt handlers while busy waiting for a single transmission to complete.

I should probably add that I view interrupt support as a much more important topic for async IO and DMA mostly as a method to reduce interrupt frequency and contention. DMA without interrupts is not actually that useful.

@jamesmunns As far as I know STM32 does not support peripheral to peripheral DMA transfers. Do other microcontrollers have such feature?

@protomors based on reading this thread, I thought it might be possible, though that might be more by accident than by design. AN4031 section 1.1.4 though seems to exclude p-to-p DMA.
I can find some mentions of peripheral to peripheral DMA being used on other architectures, such as the PIC32, as well as the PSoC3 (8051), but can't really find any compelling examples for Cortex-M based devices.
I don't have a use case (or experience with this), but it looks like this is "not a problem" at the moment.

Actually, I just found the PSoC5LP (Cortex-M3 based) does support this, judging by this (app note)[http://www.cypress.com/file/137856/download] from Cypress, but this still looks like a pretty rare feature (might not be worth defining interfaces/traits for the edge cases).

@jamesmunns Plenty of Cortex-M chips have a peripheral to peripheral communication but it's often not called DMA (because it isn't necessarily just that), e.g. the Nordic NRF have it and it's called Programmable Peripheral Interconnect (PPI) there, the ATSAMD call it EVSYS. As I mentioned before an important aspect of DMA is being able to process interrupts generated by the DMA engine and "p-to-p" is about letting one peripheral know that some other peripheral has finished processing, via DMA or not is not of importance. If you have to poll for DMA completion it's pretty much just a waste of resources.

@therealprof

However I feel there's one very important piece missing in the embedded-hal world: The ability to tie interrupts to HAL periperals.

Interrupts are a topic orthogonal to the DMA that deserve their own issue / discussion thread. I have my thoughts on the topic but I won't voice them in this issue.
Also you can already use HAL abstractions with interrupts using RTFM at zero cost.

You nicely described how DMA gets rid of the need to wait for the transfer to complete on the spot but you still need to spend (potentially) a lot of cycles to check whether your transfer has completed at some point.

You don't need interrupt handlers / callbacks to avoid busy waiting for the DMA transfer to complete either (if that's what you are implicitly suggesting). If you use cooperative multitasking (i.e. generators) you can suspend (yield) the task while the transfer is in progress, go do other tasks and/or sleep, and resume the first task only when the transfer is done -- no busy waiting or interrupt handlers involved.
@jamesmunns
I don't know about peripheral to peripheral transfer but the reference implementation is missing DMA functionality like memory to memory transfers, priority levels and burst mode. However, I think that functionality could be provided as methods on the channel types; it just needs to be implemented. Mem2mem transfers sound like they may be useful to have as a trait but I'm not sure if priority and burst mode should have traits.

You don't need interrupt handlers / callbacks to avoid busy waiting

What I'm trying to get at here is that there are a few ways to avoid the busy waiting and they have more to do with the task model (reactive / callback-y / interrupt-driven, cooperative, serial / blocking, etc.) than the DMA abstraction. I'd like the DMA abstraction to be general / flexible enough to support all the task models out there with minimal code duplication (I don't know if that's possible at this point though)

@japaric

Also you can already use HAL abstractions with interrupts using RTFM at zero cost.

Well, yes, you can. But it's a completely manual and static setup and fully separate from the hal drivers.

no busy waiting or interrupt handlers involved.

That's not what I meant. Even with cooperative approaches you need to periodically check in whether the transfer has finished and depending on how slow the transfer is you might still waste lots of MCU cycles doing that.

What I'm trying to get at here is that there are a few ways to avoid the busy waiting and they have more to do with the task model (reactive / callback-y / interrupt-driven, cooperative, serial / blocking, etc.) than the DMA abstraction. I'd like the DMA abstraction to be general / flexible enough to support all the task models out there with minimal code duplication (I don't know if that's possible at this point though)

Fair enough. I still stand by my point that DMA was introduced to reduce interrupt overhead and contention and is somewhat pointless without being able to use interrupts.

Even with cooperative approaches you need to periodically check in whether the transfer has finished and depending on how slow the transfer is you might still waste lots of MCU cycles doing that.

With a naive implementation, yes. Above, I was refering to a Tokio (epoll / kqueue) like implementation; there a task is only resumed when it can make progress -- this eliminates the continuous polling you mention. I already have an implementation in my head but haven't had time to test it; it uses the interrupt mechanism (NVIC), because otherwise you can't go to sleep, but doesn't involve any (device specific) interrupt handler.

With a naive implementation, yes. Above, I was refering to a Tokio (epoll / kqueue) like implementation; there a task is only resumed when it can make progress -- this eliminates the continuous polling you mention.

So how would a task know it can make progress without polling or being notified by an interrupt handler?

it uses the interrupt mechanism (NVIC), because otherwise you can't go to sleep, but doesn't involve any (device specific) interrupt handler.

That's yet another aspect why you want to be able to use interrupts. ;)
I'll be happy to look at your ideas when you have something. Just wanted to throw in that for me personally DMA is way down on the implementation list without a better way to tie use interrupts with drivers, it's just a ton of work to implement with very little benefit over a stupid/simple register based implementation.

Current status of the DMA API design:
I think we have a clear idea of what the API should provide:
It MUST

be memory safe, obviously
expose a device agnostic API, so we can use it to build DMA based HAL traits
support one-shot and circular transfers
support mem to peripheral and peripheral to mem transfers
not require dynamic memory allocation, i.e. allocator support should be optional

It SHOULD support these features:

Canceling an ongoing transfer
Memory to memory transfers
Peeking into the part of the buffer that the DMA has already written to
Data widths other than 8-bit
Using a part / range of the buffer; the API still needs to take ownership of the whole buffer in this scenario

This is out of scope:

Configuration stuff like enabling burst mode or constraining usage of a channel to certain
peripherals. The API for this functionality is up to the HAL implementer to provide / design.

What we know so far from existing implementations:

Ownership and stable addresses are a must for memory safety

(remember that mem::forget is safe in Rust)
The ongoing transfer value must take ownership of the buffer and the DMA channel
storing [T; N] in the Transfer struct doesn't work because the address is not stable -- it
changes when the Transfer struct is moved
&'static mut fulfills these two requirements but so do Box, Vec and other heap allocated
collections. See owning_ref::StableAddress for a more complete list.

These concepts need to be abstracted over (using traits or structs):

An ongoing DMA transfer
A DMA channel, singletons make sense here
Buffers compatible with DMA transfers. cf. StableAddress. Also the element type must be
restricted -- e.g. a transfer on &'static mut [Socket] doesn't make sense.

Unresolved questions:

Alignment requirements. Does a transfer on a [u16] buffer require the buffer to be 16-bit
aligned? The answer probably depends on the target device.

Attempts at the problem:

Memory safe DMA transfers, a blog post that explores using
&'static mut references to achieve memory safe DMA transfers

stm32f103xx-hal, PoC implementations of the idea present in the blog post. It contains
implementations of one-shot and circular DMA transfers

myeisha's DMA framework. Supports several heap
allocated collections like Box and Vec.

@japaric IIRC (at least with DMA USB drivers) regardless of word size the buffer has to be 32-bit aligned on the m3's I've used. Almost definitely varies with platfom though.

I was a bit stuck trying to figure out how to use this for a while with RTFM: https://github.com/japaric/ws2812b/issues/5
I found examples/mpu9250-log.rs useful, especially this trick for what to pass around in resources rather than lots of individual items:
type TX = Option<Either<(TX_BUF, dma1::C4, Tx<USART1>), Transfer<R, TX_BUF, dma1::C4, Tx<USART1>>>>;

I was wishing for a way to reuse a Transfer repeatably (update buffer and 'go again') or get back all the channels and parts to make a new one over the same resources.  This does that, but I wonder if there's a nicer way to put more of this in the generic hal rather than user code?
However, this (in the SYS_TICK handler of that example) surprised me:
    let (buf, c, tx) = match r.TX.take().unwrap() {
        Either::Left((buf, c, tx)) => (buf, c, tx),
        Either::Right(trans) => trans.wait(),
    };

wait() is a blocking call, and I had just assumed I couldn't use it!  Perhaps it would be fine in a handler for a DMA completion interrupt, where we're already 'sure' it won't block, but this is in a timer and seems like that guarantee doesn't hold. I guess it's just being glossed over for the sake of a simple example, and that the ticks are assumed to be far enough apart that the dma will have completed?
So:

have I misunderstood, and wait() is somehow ok to use generally?
is there a non-blocking is_done() or some similar check so the tick handler could return early and come back next time? (say we have larger buffers or serial flow control)
is wait() always ok to use in the corresponding dma completion interrrupt handler? Or would it need to do something like the above in fully robust code?

I spent some time adding DMA support to the stm32l4x6_hal crate, and wanted to add some notes. I attempted to use the Transfer and CircBuffer abstractions with serial ports and RTFM.
Because the Transfer owns its Tx or Rx pin and the buffer, and wait()ing on the Transfer consumes the Transfer (returning the buffer and pin), it's not possible to use them in a task because whatever you pass to the task must be consumed to start or end the transfer. For example, a DMA1_CHANNEL5 task must move the Transfer resource to operate on the buffer contents, which it cannot do. I ended up adding a couple of functions: restart for Transfer and partial_peek for CircBuffer which return slices of the buffer. They work well enough for receiving data, but I haven't found a good solution for transmitting because of the inability to pass Transfers around.

So, the PS2, which I've been developing on, has a hardware requirement for 16-byte aligned DMA (it ignores the lower four bits of the specified memory address).
For the sake of throwing ideas out, perhaps the implementation could define a zero-sized Alignment type which is #[repr(align(T))] for T-byte minimum alignment, and then the user can either borrow from the aligned crate, or use their own.
I'd rather not reinvent the DMA interface wheel if somebody's going to change its shape.

Because the Transfer owns its Tx or Rx pin and the buffer, and wait()ing on the Transfer consumes the Transfer (returning the buffer and pin), it's not possible to use them in a task because whatever you pass to the task must be consumed to start or end the transfer. For example, a DMA1_CHANNEL5 task must move the Transfer resource to operate on the buffer contents, which it cannot do.

I think one way to do it is to declare your resource Option<Tx<...>> then in your task you .take() the resource, that gets it, and other tasks will just see a none.
the comment right above yours is actually a complicated way to do it.
The Option part is for taking the object, and the Either part is because either there is a transfer running and you need to wait() it to get the bit necessary to start a new transfer, or there is currently no transfer running and it just stores the bits and pieces to start a transfer.

On the topic of this RFC, I see a need for non-static DMA buffer for transactions that will be using the DMA but are blocking.
It might seem counterintuitive, but the byte-by-byte APIs of various MCU devices are sometimes impossible to use at the speed of the physical interface (I suspect it's because of the bus traffic in between the bytes), but there is still no interest in going full async on the API (in particular by blocking we can keep the buffer on the stack, then release it).
Here is a mention of the problem: https://javakys.wordpress.com/2014/09/04/how-to-implement-full-duplex-spi-communication-using-spi-dma-mode-on-stm32f2xx-or-stm32f4xx/
Here is some code for your consideration:
https://github.com/nraynaud/stm32f103xx-hal/blob/153b97b682790bc16169868a02c7df97c8c601c6/src/serial.rs#L508
pub trait WriteDmaImmediately<A>: DmaChannel
    where
        A: AsRef<[u8]> + ?Sized,
        Self: core::marker::Sized,
{
    fn write_immediately(self, chan: Self::Dma, buffer: &A) -> (&A, Self::Dma, Self);
}

On a slightly different subject, I have rendered the buffer + ?Sized in my DMA APIs, so that I can pass slices to the DMA transfer.

@nraynaud if the operation is blocking I think there's no need for taking the receiver, or the buffer, by value. Something like fn write_all(&mut self, buffer: &[u8]) should work fine. Note that you also need the compiler fences in blocking code.
Also, the blog post linked in the issue description is a bit dated by now. The embedonomicon has the latest information on DMA; there have been a few updates: Pin instead of &'static mut (which allows Box, Rc, etc) and the compiler fences have been softened (while preserving correctness).

Thanks, I will read all that, and try to use references it didn't occur to me.
Do you have a documentation/blog post on the compiler fences and when to use them? What about volatile?

@nraynaud the embedonomicon explains why the compiler fences are needed. volatile ops can't be reordered wrt to each other but non-volatile opss (like operations on buffers) can be reordered wrt to volatile ops; the compiler fences are there to prevent the latter, which can result in compiler misoptimizations.

I've tried to rewrite dma transfer according embedonomicon here but haven't tested it yet.

How are these efforts going?  I noticed that the last posting on the thread is from Feb 11 2019.  I am new to embedded rust and would like to play with some DMA stuff.  While I could just manipulate the chip registers directly and cobble something together, there has obviously been much more thought put into this here.  I would like to play with what you guys are suggesting.  :)

@justacec More DMA implementations keep appearing. I've personally worked on two (stm32f7xx-hal, stm32l0xx-hal) over the last weeks.
I'm not aware of any efforts for creating a platform-independent API that could be included into embedded-hal. That would be great to have, but also a significant effort, I believe.

I did submit a draft PR for the stm32f1xx-hal crate (https://github.com/stm32-rs/stm32f1xx-hal/pull/244) which allows circular DMA buffers to support reading arbitrary lengths of data as it comes in.


[BUG] - Cardano-cli creates transactions with witness-set as an array, not a map - this is invalid.

Summary
Cardano-cli generates transactions which do not meet the ledger specification. Furthermore, if you're trying to sign these transactions in a browser wallet such as Nami - it will not parse them, because serialization-lib (correctly) detects the transactions produced by Cardano-cli as invalid. This is a major stumbling block for dApps looking to integrate a Cardano-node backend with any browser wallet which uses serialization-lib - as things stand, that's just not possible - due to such a simple error:
In the case of an empty witness set, the field should be an empty map {}, not an empty array [].
Steps to reproduce
Generate a transaction with Cardano-cli and observe the witness-set field - in the case of an empty witness set, you will find that it is an empty array, when the ledger spec indicated that it should be a map.
Expected behavior
Witness set field should be an empty map: {}
System info (please complete the following information):
Latest everything
Screenshots and attachments
Example cbor:
86a600828258207a9ae75e073a713791cc3045c2c2f62b925c4d33ae670da445ec80f20c459d7f0082582093ce7d39fc764e8ec9026a50f1f41f0d18649e18378718e91c45060fead0c703010d80018382583900af035fb9a547276bbb86a1936882524463eccb82465fcdf44489013713a39f099c533eb97b848aea0ae05850b4b40ec1ec05541a3f84c9b11a0483d91a83581d70eb18d99e0c392b18a436d595b5223e8fff559c4643118c1d403dfaab821a001e8480a1581c79e78a324e27842aa2ccaa21a710198b1db2c2a835b70d663b68a9dca1534d79546573746e6574547231303030303237310158201db43f14402b617c93f860381ec852799320d35c1be86386dffa53d792d846d582583900af035fb9a547276bbb86a1936882524463eccb82465fcdf44489013713a39f099c533eb97b848aea0ae05850b4b40ec1ec05541a3f84c9b1821a01312d00a2581c6b8d07d69639e9413dd637a1a815a7323c69c86abbafb66dbfdb1aa7a14002581c79e78a324e27842aa2ccaa21a710198b1db2c2a835b70d663b68a9dca2534d79546573746e65745472313030303032363701534d79546573746e65745472313030303032363801021a0002d8a50e8007582053d1932595c943555dd467aa2d3a36efdc865caf35ad77c04c270edcfd2dc39c9fff8080f5d90103a100a119013863626172
Additional context
We're trying to complete our smart validator integration at Artifct.app - our architecture involved having cardano-cli generate transactions which were then sent to Nami wallet for signing in the browser. Nami uses serialization-lib to parse the transactions, but it fails on every transaction we generate from Cardano-cli due to this invalid witness set field. I have discussed this with an Emurgo developer on their issues tracker:
https://github.com/Emurgo/cardano-serialization-lib/issues/259

Also had this issue where we are trying to construct a tx with Plutus script in the cli and get it signed by nami wallet / Cardano-Serialization-Lib but it's just not compatible hopefully this gets resolved

Same here, I had to perform several hacks and magic tricks (thanks to the serialization lib dev guidance) to make it work. I believe this is really important, the discrepancies between the CLI and the emurgo serialization lib are hindering and slowing down a lot of dApps. Would be awesome to hear from the dev team if they are working on this or at least aware of it :).

@kieransimkin can you provide the step to reproduce this? This is what I see:
$ cardano-cli transaction assemble --tx-body-file no-wit.txb --out-file no-wit.tx
Missing: (--witness-file FILE)

So I don't know what you mean by using the cli to create a transaction with no witness set. and similarly for transaction sign.

Producing any transaction via cardano-cli using something like this:
cardano-cli transaction build-raw \              --fee 177293 \              --tx-in 172bafb41de4031be55a5227f0b4e33a55981dee1ebd623cbce7555eb62210cd#0 \              --tx-out addr1q9ry3srf6n4crvtnch432vpz8agfjhst2tl3v9kwklr2x05f7mwd3ll67jr7hs39dmdf4awgtr8ht2cp5cvqxgg2kxhqzuh6ls+1500000+"1000000 dd3a481f3e8feb0190830c4e664de69b1f819af352cb772d29c8831e.KMA" \              --tx-out addr1vx05dzjm3kjqsyz82vfl5r57hpw7axvkqseyeprkxkmfymshnauma+5147966+"0 dd3a481f3e8feb0190830c4e664de69b1f819af352cb772d29c8831e.KMA" \              --out-file rec_matx.raw
Serialization-lib fails to parse the cbor generated from this,.

Thank you @kieransimkin , now I understand.
The transaction build-raw command does not create a transaction. It creates a transaction body.
Note that you don't need to sign the transaction body, you need to sign the transaction ID, which is the hash of the body. You can get this from the cli with:
 cardano-cli transaction txid --tx-body-file <body_file>

Here's a bit more context, which you may or may not be interested in. I believe (but don't know for sure, I've not done any work myself on the cli) that the cli uses a different format for serializing the transaction body than is specified in the ledger, in order to better match how it constructs transactions. The transaction txid command, however, does use the ledger serialization in order to construct the correct ID. Moreover, the cli does put the transaction body in the ledger form when serializing a transaction.

Thank you @kieransimkin , now I understand.
The transaction build-raw command does not create a transaction. It creates a transaction body.

Thanks - this is new information to me, and very helpful, thank you. I think it would be useful to the community if someone who knows what they're doing put up some kind of guide on how to get transactions generated by cardano-cli to be correctly parsed by Serialization-lib/Nami - whether that be Emurgo, IOHK, or Nami developers - someone please help us out!

Thank you @kieransimkin , now I understand.
The transaction build-raw command does not create a transaction. It creates a transaction body.

Thanks - this is new information to me, and very helpful, thank you. I think it would be useful to the community if someone who knows what they're doing put up some kind of guide on how to get transactions generated by cardano-cli to be correctly parsed by Serialization-lib/Nami - whether that be Emurgo, IOHK, or Nami developers - someone please help us out!

Hi, I've posted solution to your issue over at cardano-serialization lib. In order to load transaction generated by cardano-cli into cardano-serialization-lib you should sign your transaction using some dummy signing keys. That way transaction witness-set should be created. Later on you just override that "dummy" transaction witness-set with real one and sign the transaction.

I believe (but don't know for sure, I've not done any work myself on the cli) that the cli uses a different format for serializing the transaction body than is specified in the ledger, in order to better match how it constructs transactions. The transaction txid command, however, does use the ledger serialization in order to construct the correct ID. Moreover, the cli does put the transaction body in the ledger form when serializing a transaction.

@JaredCorduan  It looks like this difference applies going back into the CLI too when it comes to witnesses. See Berry-Pool/nami-wallet#85 - people are reporting that the witness format the CLI takes is different from the binary spec as well. Using @pyropy 's workaround @jinglescode posted some CBOR<PHONE_NUMBER>9b8e0e4205180b4a2ac3511b39435e7e472c6cd817c1f05aaa7545c4b816601a584036b135996644821b84ccd31ad0b8ee07ef9b16eca8046aec96e8e756c0fcef293aeb0cd78866823239de10f98e5f9ddf223b814aaf2140183777d4fc742a3009) that they have confirmed worked to allow it to be accepted by the cli which I ran through a CBOR decoder and it is almost identical to the alonzo.cddl definition but with an array tag instead of a map tag - granted this is a vkey only witness so I'm not sure if the format similarity holds true for other witness types. It's still using the general key, value format as if it were a CBOR map as well. The value is as if it were an array though (2 not 1) since the keys are just any other element since it's an array.

Thanks for reporting this issue! I also saw this problem when trying to decode an unsigned transaction generated from cardano-cli. It seems like the bug is gone after the transaction is signed by cardano-cli. I pasted more evidence below. Hope it helps!
Unsigned transaction
Create an unsigned transaction
$ cardano-cli transaction build --tx-in 732bfd67e66be8e8288349fcaaa2294973ef6271cc189a239bb431275401b8e5#0 --tx-out addr_test1vrm9x2zsux7va6w892g38tvchnzahvcd9tykqf3ygnmwtaqyfg52x+100000000000 --testnet-magic 42 --change-address addr_test1vrm9x2zsux7va6w892g38tvchnzahvcd9tykqf3ygnmwtaqyfg52x --out-file tx0 --alonzo-era

$ cat tx0
{
    "type": "TxBodyAlonzo",
    "description": "",
    "cborHex": "86a50081825820732bfd67e66be8e8288349fcaaa2294973ef6271cc189a239bb431275401b8e5000d80018282581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000ba43b4b7f782581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000174876e800021a000288090e809fff8080f5f6"
}

Decode cbor hex
$ echo 86a50081825820732bfd67e66be8e8288349fcaaa2294973ef6271cc189a239bb431275401b8e5000d80018282581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000ba43b4b7f782581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000174876e800021a000288090e809fff8080f5f6 | xxd -r -p | base64 | python -m cbor2.tool -d | jq
[
  {
    "0": [
      [
        "s+\\xfdg\\xe6k\\xe8\\xe8(\\x83I\\xfc\\xaa\\xa2)Is\\xefbq\\xcc\u0018\\x9a#\\x9b\\xb41'T\u0001\\xb8\\xe5",
        0
      ]
    ],
    "13": [],
    "1": [
      [
        "`\\xf6S(P\\xe1\\xbc\\xce\\xe9\\xc7*\\x91\u0013\\xad\\x98\\xbc\\xc5۳\r*\\xc9`&$D\\xf6\\xe5\\xf4",
<PHONE_NUMBER>03
      ],
      [
        "`\\xf6S(P\\xe1\\xbc\\xce\\xe9\\xc7*\\x91\u0013\\xad\\x98\\xbc\\xc5۳\r*\\xc9`&$D\\xf6\\xe5\\xf4",
<PHONE_NUMBER>00
      ]
    ],
    "2": 165897,
    "14": []
  },
  [],  # ------\
  [],  # -------> These three arrays are problematic. They should've been an empty witness set ({}).
  [],  # ------/
  true,
  null
]

Unsigned transaction
Problem is gone after I sign the transaction.
Sign the transaction created above
$ cardano-cli transaction sign --tx-body-file tx0 --signing-key-file code/configs/local/shelley/utxo-keys/utxo1.skey --testnet-magic 42 --out-file tx.signed
$ cat tx.signed
{
    "type": "Tx AlonzoEra",
    "description": "",
    "cborHex": "84a50081825820732bfd67e66be8e8288349fcaaa2294973ef6271cc189a239bb431275401b8e5000d80018282581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000ba43b4b7f782581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000174876e800021a000288090e80a100818258208be8339e9f3addfa6810d59e2f072f85e64d4c024c087e0d24f8317c6544f62f5840e4445d095478a3e2ce3d8a81e482404bb316296694b13585b1c99e3d369548ed9927632ca798c000217d62bee24be7637fad9156b27721ed256d792dab8edf01f5f6"
}

Decode cbor hex
echo 84a50081825820732bfd67e66be8e8288349fcaaa2294973ef6271cc189a239bb431275401b8e5000d80018282581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000ba43b4b7f782581d60f6532850e1bccee9c72a9113ad98bcc5dbb30d2ac960262444f6e5f41b000000174876e800021a000288090e80a100818258208be8339e9f3addfa6810d59e2f072f85e64d4c024c087e0d24f8317c6544f62f5840e4445d095478a3e2ce3d8a81e482404bb316296694b13585b1c99e3d369548ed9927632ca798c000217d62bee24be7637fad9156b27721ed256d792dab8edf01f5f6 | xxd -r -p | base64 | python -m cbor2.tool -d | jq
[
  {
    "0": [
      [
        "s+\\xfdg\\xe6k\\xe8\\xe8(\\x83I\\xfc\\xaa\\xa2)Is\\xefbq\\xcc\u0018\\x9a#\\x9b\\xb41'T\u0001\\xb8\\xe5",
        0
      ]
    ],
    "13": [],
    "1": [
      [
        "`\\xf6S(P\\xe1\\xbc\\xce\\xe9\\xc7*\\x91\u0013\\xad\\x98\\xbc\\xc5۳\r*\\xc9`&$D\\xf6\\xe5\\xf4",
<PHONE_NUMBER>03
      ],
      [
        "`\\xf6S(P\\xe1\\xbc\\xce\\xe9\\xc7*\\x91\u0013\\xad\\x98\\xbc\\xc5۳\r*\\xc9`&$D\\xf6\\xe5\\xf4",
<PHONE_NUMBER>00
      ]
    ],
    "2": 165897,
    "14": []
  },
  {
    "0": [
      [
        "\\x8b\\xe83\\x9e\\x9f:\\xdd\\xfah\u0010՞/\u0007/\\x85\\xe6ML\u0002L\b~\r$\\xf81|eD\\xf6/",
        "\\xe4D]\tTx\\xa3\\xe2\\xce=\\x8a\\x81\\xe4\\x82@K\\xb3\u0016)f\\x94\\xb15\\x85\\xb1ɞ=6\\x95H\\xed\\x99'c,\\xa7\\x98\\xc0\u0000!}b\\xbe\\xe2K\\xe7c\u007f\\xad\\x91V\\xb2w!\\xed%my-\\xab\\x8e\\xdf\u0001"
      ]
    ]
  },  # --------> Witness set is created correctly.
  true,
  null
]

@cffls notice that cardano-cli transaction build does not create a transaction, but rather a transaction body. It is the same situation that I described above for build-raw: https://github.com/input-output-hk/cardano-node/issues/3370#issuecomment-991315090

There seems to be a lot of confusion around this - I wonder if anyone could actually post a guide - I think we're all fundamentally trying to do the same thing - make transactions that we're producing server-side somehow work with serialization-lib in the browser. Seems there are several different hurdles people are stumbling on here and a guide would really be appreciated, still not quite at the level where I understand it well enough to write that guide myself yet I'm afraid.

@cffls notice that cardano-cli transaction build does not create a transaction, but rather a transaction body. It is the same situation that I described above for build-raw: #3370 (comment)

Thanks Jared for the response. I don't think cardano-cli transaction build create a pure transaction body either. From my post above, you can see that its output is not a map, but an array that includes other fields, such as is_transaction_valid and auxiliary_data.
The confusion probably comes from the some developers' assumption that cardano-cli transaction build will create a valid and complete transaction as defined in alonzo cddl [1], which is not the case. If the command is not intended to do so, it should state very clearly in its description about what exactly it produces and the format of it. At the moment, it is producing neither a complete transaction [1] nor a transaction body [2]. I believe either scenario would work, although my preference is to have this cli produce a complete transaction so that we don't need to wrap the transaction in a transaction after. The bottom line is that we should have a clear definition for the output of this command and the command should produce what it claims to produce.
@kieransimkin A workaround solution for the use case you described (creating unsigned transactions from server and have user's wallet to sign it using the signTx API from wallet connector) is to deserialize the cbor output from cardano-cli transaction build, take the transaction body, place the body to a new transaction with a clean witness set and two other necessary fields , and serialize the transaction into a new cbor, which is to be sent to client's wallet to sign.
Nevertheless, imho, the long-term solution is for IOG to fix the behavior of cardano-cli transaction build.
[1] Transaction definition: https://github.com/input-output-hk/cardano-ledger/blob/master/eras/alonzo/test-suite/cddl-files/alonzo.cddl#L13-L18
[2] Transaction body definition: https://github.com/input-output-hk/cardano-ledger/blob/master/eras/alonzo/test-suite/cddl-files/alonzo.cddl#L50-L65
[3] SignTx: https://github.com/cardano-foundation/CIPs/tree/master/CIP-0030#apisigntxtx-cbortransaction-partialsign-bool--false-promisecbortransaction_witness_set

Just as a general tip to IOHK and IOG in general - We need documentation for this stuff! How am I supposed to know that cardano-cli generates some kind of half-baked transaction? It doesn't say that anywhere, or at least not in any of the documentation that I scoured looking for answers on this.
Why would Cardano-cli build a half-baked transaction anyway? I think I understand the issue, but not the cause of it - cardano-cli, in the way that most people are using it, generates some kind of transaction that is not compatible with other libraries. I don't know know anything of the ledger spec, I'm a PHP and Javascript programmer trying to integrate here, I shouldn't have to understand the intricacies of this - and Cardano-cli should not generate transactions that don't load into Serialization-lib - I do not care about the technical specifics of why this is, it's just wrong and needs to get fixed.
We've now solved this problem at our end, but it's frustrating for me to see other people suffering the same problem, and nothing done about it. IOHK/IOG/Emurgo (whichever part of this multiheaded beast wants to take responsibility for this mess) needs to write some docs, and fix cardano-cli - because right now you're creating a lot of very frustrated programmers out there - programmers who will quite happily drop Cardano and go to a chain that actually bothers to write documentation and fix hurdles that are tripping up 90% of the developers trying to integrate them.
I would really like to see something from IOG/IOHK/Emurgo in the way of documentation - why is the Cardano documentation so bad? Every programmer I know says it - I understand Cardano is a work in progress, but so are most software projects, and most have better documentation. If a problem is tripping up most of the people who are integrating with your system, fix it and document it!

Thanks for all the information @cffls - hopefully this will save someone else a lot of time when they're solving this problem. I would really like to see something official in the way of a fix for this - to be honest because my confidence in Cardano has gone from very high to almost zero, thanks to the issues we've had with this, and the lack of communication and general uselessness of the people who have created this mess in the first place.
I can tell you there are a lot of programmers in the community right now who really need to have their faith restored in Cardano, because it's taken quite a battering over the last few months - with the announcement of smart contracts before PAB was ready (PR disaster for Cardano), the nightmares we've had with integrations, and the general uselessness of IOHK, IOG and Emurgo. And to top it all off, turns out Emurgo have been quietly setting up in competition with us! Not good - makes me want to chuck out our Cardano integration, even though it's now working, and replace it with a Solana integration (and that's saying something!) I'm really hoping we see something of substance out of Cardano soon rather than just continued hot air from Charles.
I apologise if my messages come across as irritated - I don't mean any negativity towards any particular person - it's IOHK/IOG/Emurgo all together here that I'm annoyed with. I'm just hoping one of them might listen to the community for once rather than hiding in ivory towers.

For what it is worth, I am proud of the engineers at IO, it's a harmonious blend of academics and real world developers (and many academics turned engineers with much experience). The intermediate serialized data produced by build and build-raw is actually very helpful fol shielding users from details about the protocol that they do not need to know about, such as the redeemer pointers and other book keeping which exists as optimizations for the protocol. We now see how folks are using the CLI, in ways that we didn't predict, and we are going to do our best to clear up the confusion.

@JaredCorduan Thank you for confirming the issue! Looking forwards to a fix in the upcoming releases.
@kieransimkin I really appreciate that you found this issue, which saved me a good amount of time. Your frustration is totally understandable. However, I have to say that IOG definitely has many great engineers who are building high quality softwares. For example, the stableness of cardano-node (which can run weeks without any interruption) has impressed me a lot. The wallet web bridge was proposed long after the existence of cardano-cli, so it is understandable that IOG didn't expect the output of cardano-cli transaction build will be directly used by softwares other than itself. Needless to say, my confidence in Cardano has never changed. The more I dig into Cardano's design, the more I find the beauty and simplicity of it. Let's keep our confidence and continue building on Cardano!

For what it is worth, I am proud of the engineers at IO, it's a harmonious blend of academics and real world developers (and many academics turned engineers with much experience). The intermediate serialized data produced by build and build-raw is actually very helpful fol shielding users from details about the protocol that they do not need to know about, such as the redeemer pointers and other book keeping which exists as optimizations for the protocol. We now see how folks are using the CLI, in ways that we didn't predict, and we are going to do our best to clear up the confusion.

I agree that what has been built so far in Cardano is something to be proud of, but I really think you need to take your heads out of the sand. There are serious problems with the decision-making and management structure in IOHK and Emurgo. It's clear for everyone to see, and pretending like everything is fine will do nobody any good. There is a reason Cardano is falling down the league tables, even though progress seems to be quite good.
Acting like nobody could ever have foreseen that people would use cardano-cli and serialization-lib together just doesn't wash with me - they are the two major ways of generating and parsing transactions on Cardano - what made you think people wouldn't want to use them together?
I imagine this disconnect has something to do with Serialization-lib being maintained by Emurgo, and Cardano-cli being maintained by IOHK. It's a mess.
Rather than patting each other on the back and pretending like everything is fine, I want people at IOHK to start asking questions - "how could we have missed this?"
You released smart contracts before they were ready - knowing everyone was going to want to start using them - it would not have taken much foresight at all to think through the main process people are going to be integrating. The main way people were generating transactions is with cardano-cli in the way I've specified - there's a reason this is the main way people were creating transactions - it was pretty much the only way way detailed in any of the documentation. Did nobody at IOHK really think this through?
This is why I keep making the comment about ivory towers - it feels like nobody at IOHK or Emurgo has actually placed themselves in the shoes of a person who might be using their software. Or indeed sought and responded to feedback from those people.
Cardano is not fine - it's bleeding heavily - stop pretending like everything is okay!

@kieransimkin Ok, calm down. My reading of this is that you didn't sign the transaction, and so it didn't work. A simple mistake, hardly a devastating indictment of an entire blockchain ecosystem.
Your project is an NFT marketplace. You should be aware that the bug you are raging about is clearly insurmountable, as there are only about 15 other NFT marketplaces already launched on the Cardano blockchain. /s
If you want to contact some of those successful projects, you could join the IOHK Discord and talk to them directly, perhaps, or some of the Discords of other already launched NFT projects. I can connect you with some if you'd like. The other, easy, option for you is to use a service like Blockfrost.io, who allow API access to the Cardano system through their existing integrations.

Downplaying problems and sweeping them under the carpet again. Okay.

The plan is to make all relevant outputs from cardano-cli comply to the ledger's CDDL spec. See #3505.

Hello, I am building a tx body using cardano-cli build-raw. When I send the cbor of the tx body to cardano-serialization lib to buiild a TrasnactionBody from TransactionBody.from_byte(Buffer.from(cborHex,'hex')) I get the error that cbor is array but expected input is map. Can someone please assist on this issue?

@Fourzin what version of cardano-cli are you using?

Hi @Jimbo4350, I am using 1.35.3 version. After reading several conversations, I used the --cddl-format tag that output a tx compatible with cardano-sirialization lib (not tx body but rather tx, which is fine). I am now able to reconstruct the transaction on the web browser side. My current issue is that after successfully signing and adding the wallet as witness I get an error while submitting the tx with the error "MissingScriptWitnessesUTXOW".
I already signed the transaction with policy.skey in the cli side before sending the partially signed tx to web-browser.

@Fourzin please open a new issue and also include the commands your used to contruct the tx

Ok I will

@while0x1 There is now the option --cddl-format (soon to be deprecated again thankfully and ending this madness) that gives you just the output you need to be compatible with the specification


Made minor changes to Google cloud pipeline components automl tabular notebook

PR problem description : Extra components. Added Libraries and Deleted unnecessary code components from code now.
REQUIRED: Fill out the below checklists or remove if irrelevant

If you are opening a PR for Official Notebooks under the notebooks/official folder, follow this mandatory checklist:

[ ] Use the notebook template as a starting point.
[x] Follow the style and grammar rules outlined in the above notebook template.
[x] Verify the notebook runs successfully in Colab since the automated tests cannot guarantee this even when it passes.
[x] Passes all the required automated checks. You can locally test for formatting and linting with these instructions.
[ ] You have consulted with a tech writer to see if tech writer review is necessary. If so, the notebook has been reviewed by a tech writer, and they have approved it.
[x] This notebook has been added to the CODEOWNERS file under the Official Notebooks section, pointing to the author or the author's team.
[x] The Jupyter notebook cleans up any artifacts it has created (datasets, ML models, endpoints, etc) so as not to eat up unnecessary resources.

@bhupana if possible, can you also perform the boilerplate reduction steps? Check with your teammates for more info.

/gcbrun


ZEPPELIN-338 Add documentation for Ignite Interpreter

ZEPPELIN-338 Add documentation for Ignite Interpreter
Since the last PR that I opened few days ago has some problems, so I create new one.
I read all comments in the last one and edited below files.

looks good

I changed 'next binary release' to 'in a future binary release' : )

LGTM

Merge if there're no more discussions


Missing codecov status on PR

If you click on "show all checks" near the green CI status, the codecov check is missing. Compare this with https://github.com/cosmo-epfl/scikit-cosmo/pull/38 which still have the check.
Looking through old PR, the last one with the check was #42, and the first one without was #44.
Originally posted by @Luthaf in https://github.com/cosmo-epfl/scikit-cosmo/issues/59#issuecomment-776825832

From https://app.codecov.io/gh/cosmo-epfl/sklearn-cosmo/, it looks like coverage reports are uploaded from CI to codecov.io, but then the codecov app is not posting coverage status on the PR.

From https://community.codecov.io/t/prs-not-registering-in-codecov-pr-status/1432/11 it might be a problem with the codecov installation on the cosmo-epfl organization. @ceriottm could you try to re-install the codecov github app?

Just re-ran CI for https://github.com/cosmo-epfl/scikit-cosmo/pull/66, and it looks like re-installing the app did not solve this 😞. I'll try to look a bit more into this.

And now codecov is back, cf https://github.com/cosmo-epfl/scikit-cosmo/runs/2086686590. No idea what changed, maybe a bug on the codecov side?
Anyway, I'll close this for now, we'll re-open if needed.


Plex playlist displaying incorrect total time

When using the /plex search command, the results which return as a playlist type display the incorrect total time for the playlist. Otherwise the importing of the playlist works fine, just the total time is displayed wrong for playlists specifically.

Resolved with build v1.5 (https://github.com/ThatGuyJacobee/Elite-Music/commit/c14f8e24edf477db10f3090561afcbb52d0f7051).


Travis CI

Added Travis CI configuration file.
Updated gradle version to 2.10 to run Travis

@sagar15795  Please update the build tools version to latest 23.

Thanks!  Feel free to update this PR with a newer build version.
I merged already so I could test with Jenkins, but I think it's not very happy with not having the Android SDK:
https://builds.apache.org/job/incubator-taverna-mobile-test/4/console
Checked with the builds@ team if this is easy to achieve..
<EMAIL_ADDRESS>Anyway, more builds won't hurt - so I've requested Travis-CI integration for the Taverna Mobile repository:
https://issues.apache.org/jira/browse/INFRA-12081


False detection of CVs with blue color as Europass format

Description:
I’ve encountered an issue where the app erroneously classifies all CVs that use blue color as being in the Europass format. This behavior is incorrect and leads to false positives, as not all CVs with blue color follow the Europass standard.
Steps to reproduce:
1.	Submit any CV that uses blue as the primary color (even if it doesn’t follow the Europass format).
2.	Observe that the evaluator identifies the CV as being in the Europass format.

Expected behavior:
The evaluator should correctly assess the CV format based on the actual structure and content, not solely the color scheme. Only CVs that adhere to the Europass format should be recognized as such.
Actual behavior:
All CVs with a blue color are classified as Europass, even if they do not match the format.
Impact:
This issue results in incorrect classification and could lead to improper evaluation.

Issue was closed as requested by JessicalTV.


Create a new user

Add users to database

[x] define a new endpoint route in routes/index.js:

POST /users => UsersController.postNew

[x] define the controllers/UsersController.js that contains the endpoint

[x] To create a user, you must specify an email and a password

[x] If the email is missing, return an error Missing email with a status code 400

[x] If the password is missing, return an error Missing password with a status code 400

[ ] If the email already exists in DB, return an error Already exist with a status code 400

[x] The password must be stored after being hashed in SHA1

[ ] The endpoint is returning the new user with only the email and the id (auto generated by MongoDB) with a status code 201

[ ] The new user must be saved in the collection users:

email: same as the value received
password: SHA1 value of the value received

[ ] waiting for #12


Modal - Can't turn off fading

First, this is a great project, many thanks for the work put in!
Bug:
I noticed that I can't seem to turn off fading of modals.
I've tried fade={false} and also transitionOptions={{ duration:0}} but with no luck.
The code in Modal.svelte looks fine.
Here's a typical use in my code:
    <Modal bind:isOpen {toggle} size={"xl"} fade={false} transitionOptions={{ duration:0}}>
        <ModalHeader {toggle}>Edit News Post</ModalHeader>
        <ModalBody>
              Hello
        </ModalBody>
        <ModalFooter>
            <Button color="primary" on:click={toggle}>Save</Button>
        </ModalFooter>
    </Modal>

Sorry, this was a mistake. I used Sveltestrap 3.14 for some reason. Using 4.0.1 it seems to work. That explains why Modal.svelte looked like correct code too. Closing this.


Under windows: OSError: [WinError 126] The specified module could not be found

hi,
i installed nanomsg-python with pip install nanomsg, but when i try and import from it, i receive the following errors. my application works fine under OSX, and i'm only encountering this issue on windows.
from googling, i think the first line of the error might be diagnostic to someone who knows what it means.

could someone shed some light on this?
with thanks

ok, it looks like the nanomsg.dll is missing. i've gotten it to work by building nanomsg.dll and copying it to the Scripts folder of my virtualenv. poking around in the site-packages/nanomsg* directories, i can't find this dll anywhere, so perhaps it isn't shipped with the package at all?
this is all in python 3.5

you need build nanomsg.dll with CMAKE

You can find some binary wheels on appveyor.  Just click on the right environment, then click "Artifacts".  These were made with the  1.0.0 tag of nanomsg.  PR #54 will make this part of this repository.


I am working with the OAK-D  and trying to import this one "from depthai_helpers.version_check import check_depthai_version" and facing the Error No Module found. Can anyone guide what should I do resolve this issue

check you have the correct interpreter selected with the library installed.
https://code.visualstudio.com/docs/python/environments#_work-with-python-interpreters

Closing old issue. If this is still a problem, please reopen with the information requested. thanks


Remove rabbitmq user and permissions in the right order.

Puppet will loop forever trying to remove permissions for a user which is already gone.

Looks right to me :+1:


Bug - logic updated to change min font size to match current font siz…

logic updated to change the min font size to match the current font size if the value is greater

Before
https://github.com/pdfme/pdfme/assets/45564901/1ad03dd5-9eca-4a5a-a242-af931f4ed829
After
https://github.com/pdfme/pdfme/assets/45564901/29e834fb-c8a4-4b62-a840-ad6eec95927e

@steffancarrington Thanks! LGTM


fix generate_biom_and_metadata_release

found a small bug during today's deploy

Coverage decreased (-0.04%) to 91.553% when pulling 4a8304d031f722279dc226dee5c146ba533e8633 on antgonza:fix-generate_biom_and_metadata_release into 01c656cfa873164efa0a1798d5a48496c89c01cc on biocore:master.

Looks good, but worth adding a test?

What kind of test? The problem was that the code was using bdir_len vs os.path internal functions so if the system configuration was path vs. path/ the files will not be found.

A regression test would catch that right? Changing the config.
On (Feb-06-17|10:58), Antonio Gonzalez wrote:

What kind of test? The problem was that the code was using bdir_len vs os.path internal functions so if the system configuration was path vs. path/ the files will not be found.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/biocore/qiita/pull/2072#issuecomment-277777646

Coverage decreased (-0.03%) to 91.567% when pulling 13c4b2b4a0702fa3488a9fdf02011c93c149d644 on antgonza:fix-generate_biom_and_metadata_release into 01c656cfa873164efa0a1798d5a48496c89c01cc on biocore:master.

👍 thanks @antgonza!!


Cannot see submitted road name when roadcentrelineid and full road name do not match.

If a user selects an RCL Id and types in the road name, but the road name associated with the RCL Id does not match the road name typed in, validation rule 16 is triggered:
"RoadCentreLineID and Full Road name do not match";"Warning"
The change appears on the change queue with the warning, but Road Name components are populated based on the details of the RCL Id.
You can't see the road name the user actually entered anywhere. It does exist in the database in aims_stage.change_address.submittedvalues.
This is not a problem when we are entering in all addresses and selecting road centrelines. It will become a problem when we start doing bulk updates of addresses.
@billgeo  Are we happy to assign this as a 1.1 enhancement, or do we need to make sure the API serves up the submitted values to the UI during 1.0.2 development?

Yep. To be reviewed when SImon is designing the bulk address functionality.


Fix compile clean

Fixes #3542. I started it at the Lambda World Spree with great help from @jvican.
This is a simple ah-hoc solution, it could be improved in many ways. The main change here is that

compile:compile::clean cleans up only things in compile:classDirectory
test:compile::clean cleans up only things in test:classDirectory

Also I'm not sure how it should handle generated sources/resources.

@dwijnand @jvican
I implemented (and tested) same change for
{compile,test}:{managedSources,managedResources}::clean

to clean only generated (re)sources.
Can I add it here or should I submit a separate PR?

I'd hold fire on any additional feature for the moment (obviously hold onto those SHAs!): this is still not implemented as compile:clean and test:clean.

I'm against that. I've already explained why in this PR several times. Can you elaborate on why only class files should be cleaned by configurations?

I didn't have any strong opinion about this in the beginning, but over the couple of days at Lambda World I talked with @jvican and he basically opened my eyes on some basic things about scoping in sbt that I didn't understand (or understood wrong) before.
I think that this should be scoped by task, because it makes sense conceptually: you clean only the products of this task (in whatever configuration scope). And the source/resource-generation tasks mentioned above are another example where clean could be useful and if it's not scoped by task, how do you distinguish it from the compile task scoped one.
Another thing is the concern from https://github.com/sbt/sbt/pull/3678#discussion_r147378758:

Because compile::clean gives a false impression that it should magically work with other tasks (cleaning their specific artifacts).

but I realised that this is something general about sbt design: because of the scope delegation user should just never have such expectation and always inspect keys he uses to see if they are actually defined in the given scope or delegated to some other.

compile:clean will clean not only class files but any task result produced by the configuration Compile. The ticket I opened in https://github.com/sbt/sbt/issues/3542 mentions only compilation artifacts, the compilation artifacts are only the class files and analysis files.

I agree it's not perfect, but compilation could happen as part of run or test, so I don't think there's a point in scoping it specific to compile. If we are being super specific, I'm guessing that these files are generated in all sorts of subtasks under compile too.

See also @retronym's super simple cleanClasses impl: https://github.com/scala/scala/pull/6256#issuecomment-366142058

but compilation could happen as part of run or test

Wait, even if compilation happens because the user does run or test, how does that change the fact that compile::clean will only clean class files and analysis files? Surely we're not talking about having the same clean task for run or test, so I don't think the argument makes a difference.
Maybe someone can explain to me what's the value of having clean be scoped to the Compile configuration instead of the compile? I'm really missing the point. Also, what happens with any other configuration that triggers compile, like IntegrationTest or Test or Jmh?

Maybe someone can explain to me what's the value of having clean be scoped to the Compile configuration instead of the compile?

For me it comes down to usability and symmetry. Configuration scoping, in general I think is easier to understand for users.
If you want to add scalacOptions in Compile configuration we do:
Compile / scalacOptions += "-deprecation"

Same goes for lots of other settings related to compilation. None of them are scoped to compile task. If you want your Compile products to be cleaned it should naturally be:
> Compile/clean

If you want to compile or clean things in Test scope:
> Test/compile
> Test/clean

@eed3si9n But note that your proposal is not at odds with mine, we can still scope it in the compile task, and then delegate to it from the configuration scope. The idea is that the configuration scope, in the future, removes all products produced by other tasks scoped in Compile, whereas compile::clean is strictly charged with removing compilation products. This benefits people like me that only want the class files to be removed, and benefits other plugin authors too that may want to enrich Compile/clean with cleaning resources generated by their plugins.

Closing this in favor of https://github.com/sbt/sbt/pull/4539/commits/172c8e9a0baa90749f5ceb609a7e465c2c721d13.


Undefined for version of packages

Based on the issue: https://github.com/awes-io/demo/issues/1
Expected Behavior
Base-js doesn't return version if PHP function exec is not exists.
Current Behavior
Now the function returns undefined, should be a version of packages.
Testing
Please check on XAMPP before and after the fix.
System GA

There is an ETA about this problem?

Hey @mribera-erre2web,
we want to get the ticket for the next sprint.
It means in max. 2 weeks it will be ready to use.
At the meantime, you can use the system inside of our Docker or in the system enabled exec function.
Best,
Yevhen

➤ Galymzhan Begimov commented:
https://git.awescode.com/awes-io/base-js/merge_requests/49 ( https://git.awescode.com/awes-io/base-js/merge_requests/49 )


Html passing as a value is not rendered

i send html code as a value, but it is not rendered in the browser. what i am doing wrong?
http://plnkr.co/edit/TwMfw5LpGS2Dpqqwn5Ix?p=preview

You can't render html in that way within the grid - you're putting the string into a div, it's just rendering the string.  To render the actual html I think you'd need to make a directive, then set the cellTemplate to that directive.  Inside the directive you'd need to grab the cell value and then compile it and append it to the DOM.
Do you really want the ability to execute arbitrary html code in your cells, or do you have a limited set of different renderings that you want?  If the latter, then you can get most of what you want by using an ng-if, in some cases a filter or a cellClass may give you all you need.

depending on the user permission the backend service sends a view, edit and/or delete-button.
using ng-if is not really beautifull ;-)
do you have an example using a filter or a cellClass? i have no idea how to use...

It depended on what the html was trying to do.
If it's a button, then ng-if or ng-show is the best answer.
If it's more css stuff (making it bold or flashing or something) then cellClass would be the right answer.
If your html was just trying to format the value in the cell, then a filter can do that.
So, at the moment the best answer is an ng-if.  Is the button different at the row level, or will all rows displayed to a given user have the same button?  If so, then I'd do the configuration at the grid level using a cellTemplate.

i wrote a directive. i dont compile the html in the directive and it works as well. why?
i updated the plunk: http://plnkr.co/edit/TwMfw5LpGS2Dpqqwn5Ix?p=preview

I made a directive:
angular
   .module('theApp')
   .directive('x3mCompileHtml', x3mCompileHtml);

x3mCompileHtml.$inject = ['$compile'];

function x3mCompileHtml($compile) {

   var directive = {
      restrict: 'A',
      replace: true,
      link:     link
   };
   return directive;

   function link(scope, element, attrs) {
      attrs.$observe('x3mCompileHtml', function (val) {
         element.html(val);
         $compile(element.contents())(scope);
      });
   }
};

The directive in action:
:
columnDefs:          [{
   :
   cellTemplate:     '

'
},
:


Add support for persons-on-events

It is my understanding that since the Nov 02 update, the data model has been updated:
The Event table now holds a person_properties column that is supposed to store all the information of the person associated with the event.
This column is currently missing in the Snowflake exports. Is adding support for this column in exports as simple as adding the column into the generateCsv function or is it blocked by https://github.com/PostHog/posthog/issues/10192 ?

Closing as this plugin has been deprecated in favor of our Snowflake destination for batch exports.
Joining person properties on exported events is in our 2024 roadmap for the new batch exports system.


relayBroadcasts Spam

Having relayBroadcasts on by default, or having ircBroadcastMessage in it's current format allows anyone to spam IRC channels if they constantly use /say.

But /say is an OP-command, why would OPs use it to spam?

 It's an issue with the fact that you can join IRC channels with EiraIRC that you don't own.
It's actually useful when you set EiraIRC to join your community's IRC channel for example, but that isn't always how EiraIRC is used.

Okay, you got the core of the problem wrong but I get what you mean now. The real problem is that I forgot to check for permission on EiraIRC's relayBroadcasts hook. It's only supposed to work for server OPs (and if those were to use it to spam then you picked the wrong server OPs). Will fix.

That's not what I mean though. Since you can have EiraIRC on your server, be an op, join to an IRC channel and spam with /say.
Of course you'd probably get kicked out of IRC but it still allows this for the duration an op is not available to ban the user, and somebody might use it without knowing better.

Or you could just grab any IRC client, join to any IRC channel and spam normally.
Or you could, since you're an OP, run /irc config global relayBroadcasts true and start spamming then.
 If you do, that's your own fault.

In my opinion this is kind of like having a default alias /ANNOY in the said IRC client. Sure, people don't have to use it but it is there.
This issue for example emerged from people using /say in #rr, where you aren't in control of who is op'd in their own servers. I'll probably open an issue on Resonant Rise to change their default EiraIRC configuration since we can't see eye to eye on this, and we can agree to disagree on this matter?
At least you got something accidentally fixed :)


Should we move giter8 to sbt organization?

Not sure if it's been raised before or what is the status, but it looks like https://github.com/foundweekends/giter8/issues has been left on the sidelines and it's actually an integral part of sbt.
I was prompted by https://github.com/coursier/coursier/issues/3045#issuecomment-2536205162
It looks like there was an update to slf4j in https://github.com/foundweekends/giter8/pull/899 and I think that would at least get rid of that warning, but no one was really available to merge that.
I could at least help with some smaller maintenance, but maybe it would also be good to move it to sbt org and have more people being able to merge things?

I'm responsible for the current state of Giter8 showing some seams, but I don't know if moving to sbt org changes the situation much.

sbt new is a template resolver (https://www.scala-sbt.org/1.x/docs/sbt-new-and-Templates.html), so technically it's not limited to Giter8 templates.
running sbt new without parameter now shows .local templates built into sbt, which sbt project can keep it up to date.
foundweekends organization is managed by me and @xuwei-k, so I don't know if moving the project to sbt organization alone changes the situation significantly, just like many sbt plugins don't get maintained if people maintain them. in general, sbt project already has a wide range of projects including Zinc, so Giter8 just hasn't been on my radar.
I'd be happy to add more maintainers to Giter8 repo, but I've been careful not to add totally random person and end up introducing a "supply-chain" vulnerability intentionally or inadvertently. If you'd like to join the project to occasionally maintain it that would be awesome.
https://github.com/foundweekends/giter8/pull/899 changes the slf4j-simple version used for testing, so I don't know if bumping that is going to fix the unwanted slf4j warnings.

I'd be happy to add more maintainers to Giter8 repo, but I've been careful not to add totally random person and end up introducing a "supply-chain" vulnerability intentionally or inadvertently. If you'd like to join the project to occasionally maintain it that would be awesome.

I would be happy to help whenever needed. We use it in Metals so it might make sense to make sure it works well. We haven't had any big issues though.

https://github.com/foundweekends/giter8/pull/899 changes the slf4j-simple version used for testing, so I don't know if bumping that is going to fix the unwanted slf4j warnings.

Ach, I didn't look properly, but I see you looked into that already. Thanks!


Autorest generated C# code validate optional properties even it's null, which make validation fail

Describe the issue or request
Please describe the issue or feature request.
Storage team recently have new feature have swagger change as following.
The autorest generated C# code from it with always fail in validation, when any of the optional properties is null.
Since the properties are optional, we should skip validate if it’s null.
Swagger:
    "DateAfterModification": {
      "properties": {
        "daysAfterModificationGreaterThan": {
          "type": "number",
          "multipleOf": 1.0,
          "minimum": 0,
          "description": "Value indicating the age in days after last modification"
        },
          "daysAfterLastAccessTimeGreaterThan": {
            "type": "number",
            "multipleOf": 1.0,
            "minimum": 0,
            "description": "Value indicating the age in days after last blob access. This property can only be used in conjuction with last access time tracking policy"
        }
      },
      "description": "Object to define the number of days after object last modification Or last access. Properties daysAfterModificationGreaterThan and daysAfterLastAccessTimeGreaterThan are mutually exclusive"
    },

Generated C# code:
The generated C# code to validate this item will be following (with autorest 2.0.4413, 3.0.6306)
Please note the validation will fail when any of DaysAfterModificationGreaterThan or DaysAfterLastAccessTimeGreaterThan is null (null %1 = null, null != 0). But since both properties are optional, null should be a valid value for the 2 properties.
        public virtual void Validate()
        {
            if (DaysAfterModificationGreaterThan < 0)
            {
                throw new ValidationException(ValidationRules.InclusiveMinimum, "DaysAfterModificationGreaterThan", 0);
            }
            if (DaysAfterModificationGreaterThan % 1 != 0) // ensures that the value is a multiple of 1.0
            {
                throw new ValidationException(ValidationRules.MultipleOf, "DaysAfterModificationGreaterThan", 1);
            }
            if (DaysAfterLastAccessTimeGreaterThan < 0)
            {
                throw new ValidationException(ValidationRules.InclusiveMinimum, "DaysAfterLastAccessTimeGreaterThan", 0);
            }
            if (DaysAfterLastAccessTimeGreaterThan % 1 != 0)
            {
                throw new ValidationException(ValidationRules.MultipleOf, "DaysAfterLastAccessTimeGreaterThan", 1);
            }
        }

Describe your ideas for solutions
Please describe how you think a change or changes might work. Consider
how these changes might work for multiple services.
Suggested change:
Autorest generated code should first check if the value is null or not, and only do validation when it’s not null, like:
  if (DaysAfterModificationGreaterThan != null && DaysAfterModificationGreaterThan % 1 != 0) // ensures that the value is a multiple of 1
  {
    throw new ValidationException(ValidationRules.MultipleOf, "DaysAfterModificationGreaterThan", 1);
  }

Add labels
As appropriate, select a label to describe how hard the issue is to work
around or how hard it would be to do without a particular feature that
could help make it easier. These labels start with "workaround".
Not workaround now, until remove the validation from swagger.
Also select a label that describes how many instances of the workaround
you would have to do without the bug being fixed or feature implemented.
These labels start with "instances".
These labels will help priority bug fixes and feature requests.
If this bug or feature request is for older versions of autorest, please
remove the v3 label and add the v2 label as appropriate.

Hi @blueww, thank you so much for filing the issue. We have published a preview version here to fix this issue. Please try to generate code using autorest --csharp --use<EMAIL_ADDRESS>[...]  version and let us know if you see any issue.

@blueww Thank you so much for filing the issue!
We have published a preview version- @microsoft.azure/autorest.csharp/v/2.3.90 with the fix for this issue. Please try to generate code using autorest --csharp --use<EMAIL_ADDRESS>[...] and let us know if you see any issue.

I have tried to regerante .net SDK code with the preview version autorest, and it works well, this issue is resolve in the preview version.
Would you please help to release it, and apply it to .net SDK generated in https://github.com/Azure/azure-sdk-for-net?

It's entirely manual. So, you would need to do the following -

Reset the autorest versions by running autorest --reset  command
Run the generation

For more about versioning refer these docs:

https://github.com/Azure/autorest#generators
https://github.com/Azure/autorest/blob/master/docs/autorest-versioning.md

Hi @blueww, I'm closing this issue. Please open a new issue if you have any further questions or find any other bugs in the preview.


Different polygons on zoom level

Hi,thanks for this repository. I think it is a great start for anyone looking to start in indoor maps like myself.
How can i make to have differente polygons on differents zoom levels. For example In zoom level 1 i can see sectors and in a zoom level 2 the map renders the rooms of that sectors (polygons too)
Can you give me a hint? leaflet-indoor can manage that requirement?
Thanks again for this amazing repository!

You could use two separate geoJSON files/strings and using leaflets own way of seeing the zoomlevel to change between the two files/Strings and then re-draw the indoor map layer?
it's a bit of a late comment but nonetheless hope this helps


vscode, extension: Autogenerate generatedEs3.js on saving js.js.

This is done using Save and Run.

Thanks! It'd be good to automate this.
I am not a fan of this implementation, though.

It goes and pops up the "Terminal" side area, even though there's no useful information there. That's leaking the implementation into the UI.
I took a quick look through the code of this extension, and it is fragile and buggy -- what it actually does is to paste the text of the command into the terminal window.

This is basically equivalent to blindly typing the command into your nearest Terminal.app window at random times! If you're lucky, or perhaps if you never use the terminal for anything else, this will work... but if there's anything else going on, it could have absolutely arbitrary effects.

VS Code has a built-in feature intended for this use case, called "Tasks". Want to try using that instead?

A good step 1 would be to just make a "task" that runs tools/generate-webview-js. It won't yet run automatically on save, but you'll have the keyboard shortcut Ctrl+Shift+B (or Mac equivalent; "Run Build Task") for it.
To make it run automatically on save, the recommended thing is to have the task itself watch for changes. We can handle this after step 1 is done.


Any way to perhaps customize the sounds?

Lets say I want to change the death sound to the gmod death sound, what files should I change or configure? Or is there no way to do it yet?

Unfortunately this isn't possible with the mod as it currently is. Sorry!


Develop

ENH: merging work of yqiao
This contains the PhD research by Yuchuan Qiao.
Part of this commit came from mstaring/performance_ITK4 and part from YuchuanQiao/performance_ITK4
I took it manually and committed it.

@N-Dekker @YuchuanQiao This commit contains work of Yuchuan from his PhD thesis. Could you please check i) if this code is itk5 compliant, ii) if I copied it correctly from https://github.com/YuchuanQiao/elastix/tree/performance_ITK4, because I did that manually.

@mstaring The "azure" CI has accepted the work of @YuchuanQiao for all tested platforms, including ITK5. Is it included (for compilation and test) "by default", or it there a CMake flag to "enable" his work?

I updated some refs. I think more can be improved, many seem a copy-paste from a previous file. But that can be done after the merge in a new PR.


Fix cy.DoesNotContain

While beginning work on another test case I noticed that this method looked wrong. Because this method passes when an element does not exist on the page, it is easy to get it wrong and not notice. So I created a test case to verify it, and sure enough the code in it was bad (my bad). Now we can keep an eye on it if we ever have to change it and make sure it keeps working.

:tada: This PR is included in version 0.341.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Added new term 'lysophospholipids' under module 'metabolomics'

Updated AD.model.csv to include new terms 'lysophospholipids' and 'Leiden Oxylipins'

Approved and merging my two commits to add 'lysophospholipids' and 'Leiden Oxipilins,' respectively into the AD data model

@MasterPastures Sorry to do this, I was looking to see why the automated checks hadn't run and I realized that you edited the AD.model.csv directly. Can you edit the module csvs instead? E.g. go to modules -> experimentalData -> assay.csv and edit there. Then the script that rolls up the changes into the full csv and generates the json-ld will run!

Merged in #60 and #61, closing this.


"/:pathMatch()" seems not to work when regex contains group

Version
4.0.2
Reproduction link
https://jsfiddle.net/b69otxkj/1/
Steps to reproduce
Go to the minimal reproduction page and click run. Open console.
What is expected?
Can correctly parse below regex:
^(?!saml.$).
What is actually happening?
In console, you can see error
vue-router@next:879 Uncaught Error: Invalid custom RegExp for param "pathMatch" (^(?!saml.): Invalid regular expression: /(^(?!saml.)/: Unterminated group
at tokensToParser (vue-router@next:879)
at createRouteRecordMatcher (vue-router@next:1185)
at addRoute (vue-router@next:1273)
at vue-router@next:1423
at Array.forEach ()
at createRouterMatcher (vue-router@next:1423)
at createRouter (vue-router@next:2654)
at ?editor_console=true:167

What I would like to achieve is to catch any no-matching URL except whose starting with "saml", and redirect to 404. But it cannot correctly parse my regex. Looking at the error message, seems the extract regex is missing anything after the first closing parenthesis.

You can use the path ranker to test out the generation. I also added what seems to be what you want to do. If you try the regex you were trying you will see what RegExp is Vue Router trying to create and why it fails: the closing ) has to be escapeed inside custom regexp

You can use the path ranker to test out the generation. I also added what seems to be what you want to do. If you try the regex you were trying you will see what RegExp is Vue Router trying to create and why it fails: the closing ) has to be escapeed inside custom regexp

A bad design


ci: disable x3 publish

:tada: This PR is included in version 5.0.0-beta.30 :tada:
The release is available on:

v5.0.0-beta.30
GitHub release

Your semantic-release bot :package::rocket:


custom messages not working for input type='text'

Setting title attribute but not working for input type='text'. It does work for input type='checkbox' however.
{{input type="text" placeholder="Email" valueBinding="email" required='required' title='custom error message'}}

Hi,
As per spec, the "title" attribute is only used when the "pattern" validation is failing. If you set a pattern attribute it will display the title. However, as you have "required", the first validation rule will be the required.


CUDA-11.6-fix

Fix a cuda cfg parsing problem for CUDA >= 11.5: target labels are changed from .L_ to .L_x_, so new code is added to extract name using the new pattern.
Interestingly, using cuda-11.6 collects 13% less samples than cuda-11.4 for quicksilver, but it has nothing to do with postmortem analysis. I confirmed this issue using hpctoolkit's log file.

@jmellorcrummey Thanks for the suggestion. Please check the simplified code.


Facet panel for mobile

Objective
What:
👉 Facets panel for mobile & tablet
Why:
👉 Close the responsive issue
Type

[x] Bug Fix
[ ] New Feature
[ ] Performance Tweaks
[x] Style Tweaks
[ ] Code Refactoring
[ ] Documentation
[ ] Tests

Tested
https://user-images.githubusercontent.com/47173348/189872795-e62131f5-ea53-4996-a016-8dfda83a31dc.mov
Documented

[ ] Have you documented in changed file using comments
[ ] Have you added instructions to the README if needed?

Task linked: CU-2t2zmg6 Add a slider panel for filter on mobile


Fix get-deps tool to handle non LinuxKit containers

The tool assumed that all lfedge/... docker images are LinuxKit images and there must be pkg/ directory which it not true for e.g. eve-rust container lfedge/eve-rust:1.80.1 Improve version parsing algorithm
@jsfakian I think you also need this PR

The tool assumed that all lfedge/... docker images are LinuxKit images and there must be pkg/ directory which it not true for e.g. eve-rust container lfedge/eve-rust:1.80.1 Improve version parsing algorithm
@jsfakian I think you also need this PR

I realized that it is easier to check that the pkg/<image> directory exists...

The tool assumed that all lfedge/... docker images are LinuxKit images and there must be pkg/ directory which it not true for e.g. eve-rust container lfedge/eve-rust:1.80.1 Improve version parsing algorithm
@jsfakian I think you also need this PR

I realized that it is easier to check that the pkg/<image> directory exists...

Yeah, it should be easier, FWIW (for other reviewers), the main goal of the tool is to scan Dockerfile files looking for dependencies of all eve-* packages that we built inside eve, so they will be under pkg/. With the introduction of the eve-rust (which AFAIK is now the only one that we build "outside" EVE) the parser wasn't prepared for this new use case and will consider a non-existent pkg/eve-rust, so it must be fixed....

The tool assumed that all lfedge/... docker images are LinuxKit images and there must be pkg/ directory which it not true for e.g. eve-rust container lfedge/eve-rust:1.80.1 Improve version parsing algorithm
@jsfakian I think you also need this PR

I realized that it is easier to check that the pkg/<image> directory exists...

Yeah, it should be easier, FWIW (for other reviewers), the main goal of the tool is to scan Dockerfile files looking for dependencies of all eve-* packages that we built inside eve, so they will be under pkg/. With the introduction of the eve-rust (which AFAIK is now the only one that we build "outside" EVE) the parser wasn't prepared for this new use case and will consider a non-existent pkg/eve-rust, so it must be fixed....

there is another broken case pkg/external-boot-image: pkg/kernel whatever external-boot-image is. I do not think we even use it

which AFAIK is now the only one that we build "outside" EVE

For now 😁

The tool assumed that all lfedge/... docker images are LinuxKit images and there must be pkg/ directory which it not true for e.g. eve-rust container lfedge/eve-rust:1.80.1 Improve version parsing algorithm
@jsfakian I think you also need this PR

I realized that it is easier to check that the pkg/<image> directory exists...

Yeah, it should be easier, FWIW (for other reviewers), the main goal of the tool is to scan Dockerfile files looking for dependencies of all eve-* packages that we built inside eve, so they will be under pkg/. With the introduction of the eve-rust (which AFAIK is now the only one that we build "outside" EVE) the parser wasn't prepared for this new use case and will consider a non-existent pkg/eve-rust, so it must be fixed....

there is another broken case pkg/external-boot-image: pkg/kernel whatever external-boot-image is. I do not think we even use it

pkg/external-boot-image (kubevirt related) is still built by our Makefile, so isl valid to check this dependency... pkg/kernel was the old kernel package, but I see that a kind of workaround was introduced here: 6cff366e9d90f4ca81db9c799e1c89edb2e70fe6
maybe we could change get-deps to not consider eve-kernel, I think it makes more sense once the kernel is not built on pkg/ anymore.....

The tool assumed that all lfedge/... docker images are LinuxKit images and there must be pkg/ directory which it not true for e.g. eve-rust container lfedge/eve-rust:1.80.1 Improve version parsing algorithm
@jsfakian I think you also need this PR

I realized that it is easier to check that the pkg/<image> directory exists...

Yeah, it should be easier, FWIW (for other reviewers), the main goal of the tool is to scan Dockerfile files looking for dependencies of all eve-* packages that we built inside eve, so they will be under pkg/. With the introduction of the eve-rust (which AFAIK is now the only one that we build "outside" EVE) the parser wasn't prepared for this new use case and will consider a non-existent pkg/eve-rust, so it must be fixed....

there is another broken case pkg/external-boot-image: pkg/kernel whatever external-boot-image is. I do not think we even use it

pkg/external-boot-image (kubevirt related) is still built by our Makefile (and has a directory under pkg/), so is valid to check this dependency... pkg/kernel was the old kernel package, but I see that a kind of workaround was introduced here: 6cff366 maybe we could change get-deps to not consider eve-kernel, I think it makes more sense once the kernel is not built under pkg/ anymore.....

my fix fixes this incorrect dependency

Merging, although there is one test failing. I don't see how it can be related to the PR.


Snarky support for plookup

This PR adds lookup constraints support for Snarky. The feature is currently contingent on the design of the proper Snarky interface that would enable the passing of the static scalar field element constants table from the application layer down to the constructor of the constraint system. Currently, the lookup table is temporarily initialized from bindings at https://github.com/MinaProtocol/mina/blob/3274a08de957859edf5d3fb4b0035fae611f142d/src/lib/marlin_plonk_bindings/stubs/src/pasta_fp_plonk_plookup_index.rs#L132.
The sample application provided for testing the feature can be run as
dune runtest src/lib/ocaml_plonk_constraints_app

Very old, and very stale. Closing.


Fix external containers

The changes in this PR allow connection to external containers and selection of the external containers by an orchestrator more efficiently.  Limit of 1 request per container at a time initially.  A 1 request queue would be a nice addition but I think is not needed right now.
Note: the go-livepeer/ai-video for selection updates relies on this update to ai-worker.

Dependent on https://github.com/livepeer/go-livepeer/pull/3033.

Closing this. The multiple containers per pipeline/model is merged separately as well.


Add Hybrid Search

A new search endpoint /hybrid-search and ui section that combines full text search with vector search using reciprocal rank fusion (as seen in our docs).

Hybrid search can be more effective in situations where there is a strong need for domain-specific language. Hybrid search allows the use of text search scores, which capture the nuances of the words, to enhance vector search (semantic) results.

Hi Luiz, This looks like it's in draft. Can you please update it.


Release 1.0 required changes

https://github.com/airalab/robonomics_comm/milestone/1 changes

@strdn could you add issue references to commit message please, like (fix #issue)


Add icon to mines

there is an icon inside the rmkit/src/minesweeper directory from linus, so that should be reasonably easy to use, just need to package it and add it to the draft file


Account settings button

These are some adjustments in favour of #1323 and its proposal to make it more clear for the end-user.
Its output is tested in IE9, IE10, IE11, Edge, Chrome, Firefox ESR (v45), iOS9.
A few functionalities to recall:

Account settings only shows the first name and never the full name (Tom is short, Janneke is medium, Hendrik-Jan is large).
If a name is longer than intended, it will be adding …
If a translation of ‘Account Settings’ or ‘Login’ is longer than intended it will be adding…

This is mainly we have a fixed height footer in the navigation in order to maintain a predictable scrolling area for the upper items of this list is longer than the screen.
What it looks like
If no gravatar is set:

If a gravatar is set:
￼
On mobile:

Open issue
The adding of the Wagtail version number in the settings menu enables the following bug (#2802)
This applies to the current master branch, but also to this feature.

Thanks, @jjanssen, looks good! Are we dropping the circular mask on the gravatar?

Thanks @tomdyson! I wouldn't say ditching the circular mask on the gravatar. For the sidemenu the square size makes more sense to be eye-catching. Making it round while attached to the side looks a bit silly. For displaying within the content-area I would still go with the circular mask.
But I am happy to change it if we decide it's a breach of style :-)

Hi @jjanssen, could you please add a larger screenshot showing the menu in context? Ideally with mask / no-mask options :)

Rough sketch ;-)
Without the mask

With mask

@jjanssen thanks! I think I still prefer the masked version, but will defer to experts. @tmsndrs / @JoshBarr / @jonnyscholes / @ollywillans?

I think the first (square) option is probably going to be easier for people to use. Circles work well when people have a photo with the right composition but the square will be more forgiving.
Can I be really picky and suggest a little more margin above 'account settings' when the menu is open? It feels a bit tight.

Sure @tmsndrs
I'll wait for some feedback from the others, so I can pick it in one batch.

I like it, i like it!

Applied it?

Nope, was cleaning up branches a little too enthousiastic

Hi @jjanssen - just a couple of issues I've spotted:

if the user has no email (e.g. they were created using ./manage.py createsuperuser and no email was provided at that point) and the avatar element is missing, the alignment is broken:

on the settings -> users -> edit page, clicking anywhere in the 'Account' tab of the form causes the popup to toggle up and down due to the re-use of the 'account' ID.

Other than those, looks good!

Hi @gasman, thanks for the feedback! I have processed the issues and updated the PR.
Also made a little change in the base colouring of the element, based on the PR for #2985.

Looks good to me! Merged in 2992d90e97a9eb776d45c09b4bd50499ced555eb.


VS2017 implicit to std::string conversion fix.

Hi! Great work so far. I had to patch the library to allow implicit to-string-conversions in VS2017.
    template < typename ValueType, typename std::enable_if <
                   not std::is_pointer<ValueType>::value and
                   not std::is_same<ValueType, typename string_t::value_type>::value
//#ifndef _MSC_VER  // fix for issue #167 operator<< ambiguity under VS2015
                   and not std::is_same<ValueType, std::initializer_list<typename string_t::value_type>>::value
                   and not std::is_same<ValueType, typename std::string_view>::value
//#endif
                   , int >::type = 0 >
    operator ValueType() const
    {
        // delegate the call to get<>() const
        return get<ValueType>();
    }

Hope this helps.

@qis Could you please try the most recent version (f2dfa09aecf672a3b9980613bde196a5cb1ef994)?

My bad - I misunderstood your comment, so I reverted the last commit.
I cannot add the line with std::string_view to the code, because this would require C++17.
Could you please post the compiler error for the original code together with the options you are using with VS2017?

Here are my tests with 1a6d7f5bca9baec12fe6065c6009903c24ea533f:
#include <json.hpp>
#include <string>

int main(int argc, char* argv[]) {
  using json = nlohmann::json;
  json v  = "test";
  std::string test;
  test = v;
  return 0;
}

Compiled with /permissive- /std:c++latest /utf-8 it yields this error:
1>main.cpp
1>C:\Workspace\test\src\main.cpp(8): error C2593: 'operator =' is ambiguous
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.24930\include\xstring(2254): note: could be 'std::basic_string<char,std::char_traits<char>,std::allocator<char>> &std::basic_string<char,std::char_traits<char>,std::allocator<char>>::operator =(const _Elem)'
1>        with
1>        [
1>            _Elem=char
1>        ]
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.24930\include\xstring(2249): note: or       'std::basic_string<char,std::char_traits<char>,std::allocator<char>> &std::basic_string<char,std::char_traits<char>,std::allocator<char>>::operator =(const _Elem *const )'
1>        with
1>        [
1>            _Elem=char
1>        ]
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.24930\include\xstring(2243): note: or       'std::basic_string<char,std::char_traits<char>,std::allocator<char>> &std::basic_string<char,std::char_traits<char>,std::allocator<char>>::operator =(const std::basic_string_view<char,std::char_traits<char>>)'
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.24930\include\xstring(2221): note: or       'std::basic_string<char,std::char_traits<char>,std::allocator<char>> &std::basic_string<char,std::char_traits<char>,std::allocator<char>>::operator =(const std::basic_string<char,std::char_traits<char>,std::allocator<char>> &)'
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.24930\include\xstring(2184): note: or       'std::basic_string<char,std::char_traits<char>,std::allocator<char>> &std::basic_string<char,std::char_traits<char>,std::allocator<char>>::operator =(std::initializer_list<_Elem>)'
1>        with
1>        [
1>            _Elem=char
1>        ]
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.24930\include\xstring(2072): note: or       'std::basic_string<char,std::char_traits<char>,std::allocator<char>> &std::basic_string<char,std::char_traits<char>,std::allocator<char>>::operator =(std::basic_string<char,std::char_traits<char>,std::allocator<char>> &&) noexcept(<expr>)'
1>C:\Workspace\test\src\main.cpp(8): note: while trying to match the argument list '(std::string, json)'

I propose this change:
    template < typename ValueType, typename std::enable_if <
                   not std::is_pointer<ValueType>::value and
                   not std::is_same<ValueType, typename string_t::value_type>::value
#if !defined(_MSC_VER) || _MSC_VER > 1900  // fix for issue #167 operator<< ambiguity under VS2015
                   and not std::is_same<ValueType, std::initializer_list<typename string_t::value_type>>::value
#endif
#if defined(_MSC_VER) && defined(_HAS_CXX17)
                   and not std::is_same<ValueType, typename std::string_view>::value
#endif
                   , int >::type = 0 >
    operator ValueType() const
    {
        // delegate the call to get<>() const
        return get<ValueType>();
    }

The _HAS_CXX17 preprocessor definition is used in <string_view> from Microsoft.

I shall install VS2017 and investigate that bug, we cannot put string_view in the library unfortunately.
Thanks for the report!

Thanks for the feedback. This seems to be an MSVC bug, since the code compiles with earlier versions and C++17 should not break C++11 code.

I'm not sure there is a good way to fix this problem, #144 fixed a similar issue, by removing the possibility to convert basic_json to char (#276).
If I understood correctly the other issues, this problem will arise whenever a class with several operator= is used on the left-hand side of operator= (I don't think this is a bug of the library, if I'm wrong, please tell me)
To me, #144 was not a good fix, for the reason mentioned above, thus I don't want to do the same thing.
However, to workaround this problem, you can still use:

implicit conversion by calling constructor (std::string s = j;)
get: s = j.get<std::string>();
static_cast the right-hand side of operator=

That might not suit your needs, but I really don't know if there's a way to workaround that without removing features like in #144
We should mention this problem in the README, since this might come up again, even with user-defined types.

FYI: The library is now also checked with MSVC 19.10.25017.0 / Build Engine version 15.1.548.43366 at AppVeyor, see https://ci.appveyor.com/project/nlohmann/json.

@qis I added the code you posted in https://github.com/nlohmann/json/issues/464#issuecomment-281160071 as regression test (https://github.com/nlohmann/json/commit/b4dbebffcdb98c91130133adb6583a1b5dafe6a6). It succeeds with MSVC 2015 and 2017 in AppVeyor.
Can I close this ticket?

@nlohmann Great! May I have until evening (UTC) to test it? Just in case I find a related problem.

@nlohmann Great! May I have until evening (UTC) to test it? Just in case I find a related problem.

@nlohmann Great! May I have until evening (UTC) to test it? Just in case I find a related problem.

Of course!

@nlohmann Ok, it works fine (like before) if there is no /std:c++latest compiler option present. It still requires my changes for a C++17 enabled build.
Here is a complete project for the minimal example: https://github.com/qis/json
P.S.: Just noticed that _HAS_CXX17 is always defined as 0 if C++17 is disabled or 1 if it is enabled. Updated json-c2e80a7-fixed.hpp.

Sorry, I misunderstood. I shall add your fix. I also need to add a c++latest build to AppVeyor.

@qis I am neither expert with MSVC not with CMake. Could you give me a hint how to pass /std:c++latest to the build via cmake? Then I can add it to the AppVeyor build matrix.

@nlohmann You can just add it to CMAKE_CXX_FLAGS: https://github.com/qis/json/blob/master/CMakeLists.txt#L11

Ok, got it. AppVeyor now has a configuration where MSVC 2017 is executed with the mentioned flags: https://ci.appveyor.com/project/nlohmann/json/build/1882/job/xmj9j1tlga6eeh46
Next stop: adding the fix ;)

Unfortunately, the proposed fix does not work, see https://github.com/nlohmann/json/commit/628be157a27f71b4f045fc687d723f9e1bbdfc51 and https://ci.appveyor.com/project/nlohmann/json/build/1884. It seems as if _HAS_CXX17 is also defined with MSVC 2015 and also with MSVC 2017 without defining /std:c++latest. Consequently, we get the error
C:\projects\json\src\json.hpp(3734): error C2039: 'string_view': is not a member of 'std' [C:\projects\json\test\json_unit.vcxproj]

I have to find a better way to check whether std::string_view exists.

Ok, it's insufficient to just check whether _HAS_CXX17 is defined: it also has to have the value 1. This works now: https://github.com/nlohmann/json/commit/e48114bbd6625e6199baa10cbfe783bb95c32604.
@qis, could you check for yourself whether this works? Then I can merge the feature branch.

@nlohmann Seems to work fine in both, VS2017 and clang 4.0.0 in C++17 mode. Thank you very much for this change!

Merged the feature branch. Thanks for reporting and helping out so patiently!

doesn't work in VS2015.3 ,with /std:c++latest

Could you provide an error message and the exact version?

json.hpp(3734): error C2039: 'string_view': is not a member of 'std'
in VS2015  with /std:c++latest ,_HAS_CXX17 has value of 1

@lethe555 The _MSC_VER > 1900 code you pasted fixes the problem, correct?

@qis YES .

I'll have a look.

@lethe555 I added your proposed fix. Could you please check if it works for you?

@nlohmann It works well now.

Cool, thanks!