ENH: adding a configure ,method to SynSignal

I have added the configure method to SynSignal, so that each of the simulated detectors in hw have this method.
This method is required so that we can deploy a 'plan' that includes  yield from bluesky.plan_stubs.configure. This allows for the testing of callbacks (like the suitcase Serializers) with plans that have mulitple descriptors per stream.

I always thought of configure as a Device thing, like stage and unstage, not a Signal thing. I would like to hear from other contributors how they think about it before deciding this either way.
Another way to achieve what you need on the suitcase side would be to make a Device that contains these Signals and adding a configure method to that.

Codecov Report

Merging #682 into master will increase coverage by 0.07%.
The diff coverage is 40%.

@@            Coverage Diff             @@
##           master     #682      +/-   ##
==========================================
+ Coverage   90.96%   91.03%   +0.07%
==========================================
  Files          62       62
  Lines        9481     9486       +5
==========================================
+ Hits         8624     8636      +12
+ Misses        857      850       -7

Impacted Files
Coverage Δ

ophyd/sim.py
77.89% <40%> (-0.33%)
:arrow_down:

ophyd/signal.py
93.62% <0%> (+0.17%)
:arrow_up:

ophyd/log.py
63.15% <0%> (+11.84%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 442271c...eded43b. Read the comment docs.

I don't disagree with you @danielballan, but currently all of our simulated detectors inherit from SynSignal, not Device and this is the easiest path to get the required outcome. I thought I would implement this, and open the PR to generate discussion on how best to solve this, so I am open to other options.

This ties into https://github.com/NSLS-II/ophyd/issues/678 about the distinction between Signal and Device.
In the tests we are using SynSignal as a stand-in for when we would have a Device on the floor (because from bluesky's point of view they are 'the same'), but in this case there is something bluesky knows about that is a way to tell Devices and Signals apart.
Given that it all synthetic we can not apply the teddy criteria to sort out if this should be a device or a signal, this is the path of least resistance.

In addition to adding the empty dict warning as requested I have also made some changes to pre-exisitng flake8 issues.

configure is not part of the BlueskyInterface and appears to lie in somewhat of a grey area.
We should review the methods that bluesky is expecting and make them a part of the BlueskyInterface as either required (or optional) members. If configure is deemed an optional one, then it's OK - albeit strange - that SynSignal implements it whereas all of the other Signals classes in the entirety of ophyd do not.
My opinion: If this work is blocking progress, merge away. The above should be addressed sooner rather than later, though.

I have set up suitcase-utils to use this branch as a test dependency so this is not blocking progress but we should aim to resolve it in the next couple weeks at most. Also, be sure not to delete this branch @awalter-bnl until suitcase-utils has been corresponding updated to rely on ophyd master or the next ophyd release.

thanks for the heads up @danielballan I will try to remember not to delete the branch.

Thanks @mrakitin for the update, this new device will be useful to add to the detector_list pytest.fixture in suitcase-utils.tests.conftest.py

I think there are two issues here

we probably should have included configure in the bluesky interface (at least as an optional method and may provide a reasonable way to squeeze setting the EGU and PREC into EpicsSignal API) as the RunEnigne has special behavior responding to the configure message (discarding the descriptor cache)
in practice we have ended up with very few 'bare' Signals being used in plans (the only one I can think of that is used commonly is the ring current, everything else (AD, scalar, quadem, any motor, ...) have some sort of user configurable behavior and hence end up being a Device.  We (at least I) have been thinking of the thing is hw as a "toy beamline" but most of the things in there are actually Signal subclasses rather Device subclasses so the issue this is fixing is coming from that impedance miss-match.

We could re-do the contents of hw to 'promote' SynGauss and friends to be devices (at the cost of changing their types, confident we could do this without changing anything else about the API around them) or add a bunch more objects which are the Device version of them (but then we have 2 for not great reasons) or make a new hw fixture in suitcase-utils that has just Device subclasses, however none of those options are great.
There is already some really wacky stuff in here (such as collect_asset_docs on a Signal subclass) which is a legacy of the fact that much of this code was copied from bluesky where we were just duck-typing ophyd for testing.
I think this is the most pragmatic path forward.   I think of this module as a 'for-testing-only' space (hence being pretty cavalier about it), are these classes being used in production?

@tacaswell I don't believe you can set EGU and PREC except through accessing the corresponding record fields. That is to say, if you have EpicsSignal('PV.VAL') there is no caput that you could do to change precision or units, outside of making an additional channel/signal EpicsSignal('PV.EGU').
Regardless of the above, I personally think the right path is making them into Devices.
We use the simulated devices fairly extensively here at SLAC. @teddyrendahl and @ZLLentz would know better if such a change would break things for us.

I think this is the most pragmatic path forward. I think of this module as a 'for-testing-only' space (hence being pretty cavalier about it), are these classes being used in production?

Not used in production anywhere but we do use them in a few of our other test suites.

Regardless of the above, I personally think the right path is making them into Devices.

I agree with @klauer. I am fine making them into full devices and am willing to pay the cost of updating our test suites if it ends up making a difference. If we go this route I think it would be interesting to have all of the parameters of SynGauss as Signal components in spirit of https://github.com/NSLS-II/ophyd/pull/655 as well. (largely so it will make a pretty typhon screen :smile:)

@tacaswell I don't believe you can set EGU and PREC except through accessing the corresponding record fields. That is to say, if you have EpicsSignal('PV.VAL') there is no caput that you could do to change precision or units, outside of making an additional channel/signal EpicsSignal('PV.EGU').

EpicsSignal already holds 1 or 2 channels (when there are separate write / readback PVs) so I don't think it is too much of a stretch to do something like
def configure(self, egu=None, prec=None):
    if prec is not None:
         prec_pv = self.pvname.partition('.')[0] + '.PREC'
         self.cl.get_pv(prec_pv).put(prec)
    ...

modulo any caching / error handling logic.
I'm going to start pushing commits at this branch to do the Device conversion.

I think that the recent changes to this branch have broken all suitcases which (against our usual practice...) depend on this branch. https://travis-ci.org/NSLS-II/suitcase-cookiecutter/builds/512748508#L1970

What's the status of this work? Reviewing the comments above, it looks like we're all on board with "rewrite test objects as Devices" but that there is more to do.

It would be great if this PR could be revived. It would make developing suitcases easier.

Sorry @danielballan it appears I missed your comment on this from May. If the solution is to "rewrite test objects as Devices"  I don't have the time to undertake that amount of work, and history shows that having me attempt such wide-ranging PR's into Bluesky related repo's is a sure way to kill them. If someone else would like to take over that task I would be happy with that, or if there is a preference to just adds this with minimal changes I am happy to work through the minimal changes.

Closing in favor of #762 which has a rebased and slightly extended version of this code and https://github.com/bluesky/suitcase-utils/pull/29 which de-couples suitcase-utils from the exact ophyd version.


Get rid of stdin for deis config:push

From @pfeodrippe on May 23, 2017 18:50
The implementation of deis config:push here  uses stdin to fetch the buffer, but it was causing me problems as you can see at this issue. Running the command inside a script which gets its inputs from stdin, I had to pipe it with < dev/null to get the correct result or it would give me Creating config... ...Error: Unknown Error (409): {"detail":"user changed nothing - release stopped"}.
Can something be made to correct it?
Copied from original issue: deis/workflow-cli#305

From @Joshua-Anderson on May 24, 2017 23:10
Looks like we need to switch code to check if a  filename is provided before attempting to use stdin.


Fix #1151 'None' should not be substituted as nan when calling execute_mdx_dataframe

Closed in favor of #1159


Update CI script

Run lint/test/WASM-test/i386-test in parallel.
Node is upgraded to v12 as v11 is EOL-ed.

Codecov Report

Merging #954 into master will decrease coverage by 0.55%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #954      +/-   ##
==========================================
- Coverage   76.94%   76.39%   -0.56%
==========================================
  Files          59       59
  Lines        4350     4350
==========================================
- Hits         3347     3323      -24
- Misses        729      744      +15
- Partials      274      283       +9

Impacted Files
Coverage Δ

internal/util/util.go
80% <0%> (-20%)
:arrow_down:

internal/mux/mux.go
84.61% <0%> (-6.16%)
:arrow_down:

sctptransport.go
75.38% <0%> (-4.62%)
:arrow_down:

dtlstransport.go
71.72% <0%> (-3.15%)
:arrow_down:

icetransport.go
76.55% <0%> (-1.38%)
:arrow_down:

peerconnection.go
73.09% <0%> (-0.18%)
:arrow_down:

stats.go
82.05% <0%> (+5.12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d69a90...280604a. Read the comment docs.

They run in parallel.

@at-wat feel free to merge w/e you are happy with this!


Feature Request: Add Height/Bump Mapping to Principled Material

I am a heavy user/developer of OSPRay on Paraview and one feature that would be great to have is have principled materials support height/bump mapping as it would greatly add to the realism we could achieve.
As far as I am aware there is no way to do this with the current architecture, but if I am wrong please let me know.
https://en.wikipedia.org/wiki/Heightmap
Thanks,
Marston

Hi Marston,
Principled already supports bump mapping (normal mapping), e.g. using a (three channel) texture map_baseNormal, for multiple layers (e.g. also map_coatNormal). But you are specifically referring to (Blinn) height mapping (single channel texture)? Easiest way would then be to convert the height map into a normal map (there are many tools for that).
Cheers,
Johannes

Please re-open if there are further questions.


Removed io.anserini.{embeddings, qa, and py4j}

Per #239

Primary goal is to remove embeddings package, but qa and py4j depend  on it also. Both are also not needed anymore since we've decided to move pyjnius and perform all reranking/scoring on the Python end.

@Peilin-Yang this is code by @rosequ so I explicitly want a sign off from him. Let's not merge yet.

I have no objections, this can be removed.


Build error in Xcode10 beta

On Xcode 10.0 beta4 I've got two errors Invalid redeclaration of 'countableRange(from:relativeTo:)' for internal func countableRange(from range: Range<Int>, relativeTo collection: CountableRange<Int>) -> CountableRange<Int> and internal func countableRange(from range: ClosedRange<Int>, relativeTo collection: CountableRange<Int>) -> CountableRange<Int> in Util.swift
With the implementation of SE-0143 Conditional conformances in Swift 4.2, CountableRange and CountableClosedRange are not separate types relative to Range, ClosedRange, but their type aliases.

I actually don't get what for are those last 4 internal func countableRange(from range:... functions in Util.swift
Since after their deletion it compiles ok in both XCode 9 and 10

@hardW @johndpope
Thanks for reporting! I'll fix it.

hack
github "johndpope/EasyImagy" "0.5.1"

@hardW @johndpope
Hi. I checked it and found that the problems had been fixed in 69adfd044e8d6c1e6543705d21e6559ae4c82c3f #25 by @Dschee. Please use the swift-4.2 branch for Xcode 10 beta.
I checked that swift test and "Product > Test" for iOS with Xcode 10.0 beta 6 passed.

@hardW

Since after their deletion it compiles ok in both XCode 9 and 10

They were unused and I removed them. Thanks for pointing it out.

Now 0.5.0 also supports Swift 4.2.


[Usage]: The issue of memory usage with quantized models.

Your current environment
My environment:
vllm: 0.6.2
transformers: 4.45.2
auto_gptq: 0.7.1

How would you like to use vllm
Hello, I have a question. I am deploying the Qwen2.5-7B-Instruct and Qwen2.5-7B-Instruct-GPTQ-Int4 models using vllm, and I notice that the GPTQ-Int4 model consumes a lot of video memory. I expected it to consume only a quarter of the video memory compared to the non-quantized model. Is there any issue or parameter that I should pay attention to?

Here is my deployment command:
# Qwen2.5-7B-Instruct
CUDA_VISIBLE_DEVICES=0 python -m vllm.entrypoints.openai.api_server --port=8000 --model=/models/qwen2.5/models--Qwen--Qwen2.5-7B-Instruct/snapshots/acbd96531cda22292a3ceaa67e984955d3965282

# Qwen2.5-7B-Instruct-GPTQ-Int4
CUDA_VISIBLE_DEVICES=1 python -m vllm.entrypoints.openai.api_server --port=8001 --model=/models/qwen2.5/models--Qwen--Qwen2.5-7B-Instruct-GPTQ-Int4/snapshots/e9c932ac1893a49ae0fc497ad6e1e86e2e39af20

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Regardless of model size, vLLM allocates 90% of your GPU memory by default (you can change that via --gpu-memory-utilization argument). For smaller model, the extra memory can be used for other features such as KV cache.


EEPROM Programming

Is it possible to add eeprom read/write to pyupdi? By my understanding, the NVM Controller also allows read/writing to the EEPROM. So it should be just the same approach as flash programming with just different address right?

I recommend you look at pymcuprog - using the "-t uart" tool, you can access pyupdi.
EEPROM is of course supported.
Latest on pypi test server: pip install -i https://test.pypi.org/simple/ pymcuprog

Got it to work. Thanks!

Good to hear it 👍
(The version from test.pypi will be promoted to pypi production server in the coming weeks.)


[Feature] RTMP URLs

I would like to have a way to play RTMP & RTMPS urls.

Any examples of these links?

Any examples of these links?

rtmp://<IP_ADDRESS>/path
you can use the rtmp module for nginx and obs to experiment with this.
You'll need to set up something yourself because usually these URLs are used for streaming to a platform (and the key is suffixed onto the path)


plutus-tx: improve some errors and handle an odd case

Improve the errors seen in https://github.com/input-output-hk/plutus/issues/1495 (I can't actually fix the issues directly, I don't think), and handle the odd case expressions introduced by bang patterns.

This is all green, but Hydra is just refusing to do the aggregate job for some reason.


Copula model underfitting

Hello there,
I'm trying to use scDesign3 to simulate single-cell ATAC-seq data, and I like your cool results shown in FigureS6 and S7 of the paper. But so far I cannot repeat that on another dataset, because the copula model is always underfitting. I attached my code and some results here. Do you have any idea why the model is not working?
My code:
simu <- scdesign3(   sce = sce,   assay_use = "counts",   celltype = "cell_type",   pseudotime = NULL,   spatial = NULL,   other_covariates = NULL,   mu_formula = "cell_type",   sigma_formula = "1",   family_use = "zip",   n_cores = 2,   usebam = FALSE,   corr_formula = "cell_type",   copula = "gaussian",   DT = TRUE,   pseudo_obs = FALSE,   return_model = TRUE,   nonzerovar = FALSE )
My results:

The BIC and AIC for the copula model are Inf
The marginal BIC and AIC is also very large (with aic.marginal=4431524, bic.marginal=4819040)
The similarity of peak-peak correlation matrices between the real (training) data and the synthetic data is low (see below attachment).

selected.pdf
My training data:
They're two cell groups from a public sci-ATAC-seq atlas, where I select about 7000 peaks and 1000 cells.
Could you let me know for simulating ATAC-seq data, how many peaks you usually use/ would recommend to use?

Hi Siyuan,
Thank you for your interest in our work! The correlation does look unsatisfying. In our study, we use 1133 peaks/3836 peaks for ATAC/SCIATAC, respectively (see Table S2). Ideally, your feature should be smaller than the cell number (due to the curse of dimensionality for correlation estimation), but we can not always guarantee this.
For your question:

Since you use Gaussian copula and your feature number is larger than cell number, AIC/BIC will always be Inf. Using vine coupla can give you AIC/BIC but it will also be very slow for > 1000 features.
Marginal AIC/BIC seems Ok to me.
Yes, it looks weird. Some things you may want to try: (a) Start from a smaller set of peaks and check the result (e.g., < 1000 peaks); (b) Try NB instead of ZIP.

Would you mind sharing your sce data with me (I guess it is Ok since it comes from public data)? I can do a very quick check. My email is<EMAIL_ADDRESS>Best,
Dongyuan


Fix: Map relations to snapshot physical tables in the dbt adapter

Sometimes ref are passed into the dbt adapter while constructing a model query. This means that the production table name will be used since jinja rendering happens before we get a chance to resolve table references.
This update ensures that dbt's ref returns resolved table references when using adapter.

@crericha thanks for suggesting it in the first place!


(SERVER-1006) Unregister jruby instances on flush

Prior to this commit, when a JRuby instance was added to the pool,
it was also added to the list of "registered" instances for the pool
so that we could find an authoritative list of all of the instances
even when some were out on loan.
However, we did not have the corresponding logic to unregister
instances from this list when an instance was flushed via
max-requests-per-instance.  This resulted in a reference being
kept to the flushed instances even after they were flushed,
meaning that we could not reclaim all of the memory during
garbage collection.  This caused memory to leak over time.
This commit adds code to unregister instances when they are
flushed, so that we do not retain the extra reference to the
flushed instances, and we are able to garbage collect them
and free up the memory.

:+1:

Going to add one more commit to this to null out the http client references in the jrubies after a terminate.

:+1:, will give this another review once @cprice404's new commit is pushed up

@fpringvaldsen whoops, as @camlow325 pointed out, that change re: the http client is not relevant in the 1.x branch.  So I'm done with this PR, and we will need to add the commit that I hinted at in the 2.x branch once this is merged up

@KevinCorcoran I'm going to go ahead and merge this.  We can follow-up in later PRs if you find any issues from your review.

retroactive :+1:


JAVA-679: Support bind marker in QueryBuilder DELETE's list index.

:+1:

Should we also add a setElt for deleting from set?  It could be an alias of listElt and still work I believe.


KAFKA-6207 : Include start of record when RecordIsTooLarge

When a message is too large to be sent (at org.apache.kafka.clients.producer.KafkaProducer#doSend), the RecordTooLargeException should carry the start of the record (for example, the first 1KB) so that the calling application can debug which message caused the error.
To resolve this, I created a new function which will get the first 1KB of the message (if the RecordTooLargeException is occurring), and the same will be logged...
Thanks,
Sumit Jawale
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
No test results found.
--none--

FAILURE
7975 tests run, 5 skipped, 1 failed.
--none--

FAILURE
7975 tests run, 5 skipped, 1 failed.
--none--

SUCCESS
8083 tests run, 5 skipped, 0 failed.
--none--


[cork] Failed to run create with UTF-8 user.name in git config

If you have an UTF-8 user.name value in git config then cork create command fails with this:
Traceback (most recent call last):
  File "/mnt/host/source/.repo/repo/main.py", line 541, in <module>
    _Main(sys.argv[1:])
  File "/mnt/host/source/.repo/repo/main.py", line 516, in _Main
    result = run()
  File "/mnt/host/source/.repo/repo/main.py", line 509, in <lambda>
    run = lambda: repo._Run(name, gopts, argv) or 0
  File "/mnt/host/source/.repo/repo/main.py", line 212, in _Run
    result = cmd.Execute(copts, cargs)
  File "/mnt/host/source/.repo/repo/subcmds/init.py", line 460, in Execute
    if opt.config_name or self._ShouldConfigureUser():
  File "/mnt/host/source/.repo/repo/subcmds/init.py", line 343, in _ShouldConfigureUser
    mp.config.GetString('user.email')))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc1' in position 18: ordinal not in range(128)
2021-09-13T09:56:26Z cork: repo init failed: exit status

After changing the value to an ASCII-only then it will work as expected.
Please not that the stacktrace mentions only user.email but the whole line looks like this:
    print('Your identity is: %s <%s>' % (mp.config.GetString('user.name'),
                                         mp.config.GetString('user.email')))

This is not working:
user.name=Árpád Kunszt

This is working:
user.name=Arpad Kunszt

I think this is a bug in gerrit/git-repo but I'm not familiar with mantle/cork enough to open a ticket there. Until that it worth to mention this limitation in the documentation. If it's already there and I didn't find then then please excuse me about the noise.

Should be resolved with the new containerised SDK which does not use cork or repo.
I did some cursory testing by pasting the UTF8 name provided in the issue report into my local .git/config and the container starts as expected.
@akunszt I'll close this issue, please feel free to re-open if you encounter similar issues with the new containerised SDK.


Add a relationship to a span part-way through

Current APIs only seem to support adding relationships to a span at construction time.
However, it would be useful to add relationships part-way through a span.
For example, communication between two spans, such as receiving an RPC response, or streaming packets.
This would also enable baggage to be propagated inside communication between two long-running spans.

For example, communication between two spans, such as receiving an RPC response, or streaming packets.

Could you elaborate when receiving an RPC response requires establishing a new relationship?

@JonathanMace

receiving an RPC response

Why receiving need a new span? When A -> B, A's client span can deal with the response, if you want.

In short, if a parent span ever blocks on a child completing, then that is a causal relationship from child to parent.  Currently, we do not record this relationship (either explicitly, or even as an annotation).  At best, it is done with some ad-hoc annotation.
It is necessary to record all causal relationships for two reasons: first is for dependency and critical path analysis; second is for correct back-propagation of baggage.
Dependency and critical path analysis
Currently if a parent blocks on a child we do not capture that information.  We do not record in a principled way whether the parent spun off the child asynchronously or if the parent blocked on the child.  This is an extremely important causal relationship to record if you care about critical path latency or understanding the dependencies between children within a single span.
For example, Google encountered this in the following two papers:
Modeling the Parallel Execution of Black-Box Services, Mann et. al., HotCloud ‘11
and
Diagnosing Latency in Multi-Tier Black-Box Services, Ostrowski et. al., LADIS ’11
The figure below, taken from Figure 1 of "Diagnosing Latency in Multi-Tier Black-Box Services", illustrates how if you don't record the child response you get the top; if you were to record the child response you get the bottom.

There are further examples I can happily share if you want to discuss this one in more detail.
Back-propagation of baggage
OpenTracing only currently supports forward-propagation of Baggage.  Forward-propagation is when baggage is copied to child spans.  However, if there is a reverse causal dependency from the child back to the parent (ie, the parent blocks on the child), baggage is not propagated back to the parent.  Baggage is supposed to follow the causal execution path of requests, so this is a missing relationship.
This means that if you set some baggage in a child, you won't be able to see it in the parent.  For example, if a child blocks for a long time on a database, you might store this statistic in the baggage.  The parent will block, because the child took a long time, but will not be able to find out why they blocked.

Also see HTrace equivalent: https://issues.apache.org/jira/browse/HTRACE-118


File/API structure planning steps by default

It is crucial to have a planning step for changes in file structure, and it should have mandatory approval by users by default.
Before we even begin to make changes to files that are bigger than half the context (32k tokens / 2) we should be able to plan changes in each file's public API with mandatory approval by user by default.
Today, with so much limitation, it is almost impossible to make it autonomous.
Every change or plan can be perfected by GPT-4 many times giving it prompt (Can you do better? Can you improve your plan? And so on).
On the other hand, user should be in control of spending via GPT-4 API, and it should be configurable all guided to have that many second thoughts.
Anyway, it will be completely impossible without https://github.com/irgolic/AutoPR/issues/31 and https://github.com/irgolic/AutoPR/issues/85
Related to https://github.com/irgolic/AutoPR/issues/94

It already plans changes while it looks at files, so this is a duplicate of #87.
What would you think of having two separate triggers, one to iterate the plan in the issue, and one to trigger code generation?


Disable broken tests to ensure CI & other pipelines run

For #8797

Codecov Report

Merging #8798 into master will decrease coverage by 0.37%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #8798      +/-   ##
==========================================
- Coverage   58.67%   58.29%   -0.38%
==========================================
  Files         526      526
  Lines       27155    27155
  Branches     4040     4035       -5
==========================================
- Hits        15932    15830     -102
- Misses      10366    10467     +101
- Partials      857      858       +1

Impacted Files
Coverage Δ

src/client/testing/serviceRegistry.ts
49.09% <0%> (-47.28%)
:arrow_down:

src/client/testing/codeLenses/main.ts
40% <0%> (-30%)
:arrow_down:

src/client/datascience/jupyter/jupyterUtils.ts
64.7% <0%> (-29.42%)
:arrow_down:

...t/datascience/jupyter/jupyterDataRateLimitError.ts
50% <0%> (-25%)
:arrow_down:

src/client/formatters/serviceRegistry.ts
75% <0%> (-25%)
:arrow_down:

...ience/jupyter/jupyterDebuggerRemoteNotSupported.ts
60% <0%> (-20%)
:arrow_down:

...cience/jupyter/jupyterDebuggerNotInstalledError.ts
50% <0%> (-16.67%)
:arrow_down:

src/client/common/utils/icons.ts
83.33% <0%> (-16.67%)
:arrow_down:

src/client/api.ts
80% <0%> (-13.34%)
:arrow_down:

src/client/datascience/cellFactory.ts
58.75% <0%> (-12.5%)
:arrow_down:

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9c64b9...025411f. Read the comment docs.


fix(#patch); aave forks; update directory structure

Update aave amm, rwa, arc into their own protocol under aave-v2-forks. This is to reflect our current FE configuration on the subgraph side.
Testing:

https://okgraph.xyz/?q=dmelotik%2Faave-v2-ethereum-amm
https://okgraph.xyz/?q=dmelotik%2Faave-v2-arc
https://okgraph.xyz/?q=dmelotik%2Faave-v2-rwa

Note: going to fix lint errors in a separate PR after this is merged


[3.2.2 beta 1] Sliders apply incorrect hover mask

Godot version:
3.2.2 beta 1
OS/device including version:
Win10
Issue description:
In 3.2 when you hover a mouse on the stylized Slider it processes correctly:

In 3.2.2 however, it turns dark:

Steps to reproduce:
Create Theme for Sliders and play with it on 3.2 and 3.2.2, but better download the demo project down below.
Minimal reproduction project:
SliderBug.zip

This is not a bug. This is because an hover style for the Slider "filled" area was added in #37517, and the default fallback texture is used since you didn't define one.

I may be missing something, but from your MRP I'm not seeing either result from the Beta 1.  😕

Thanks for the notice @Calinou! Indeed, this should be mentioned in the release notes.
@avencherus The problem is: because project is created in older version of engine the style in theme does not automatically updated to add support for a new style.

@akien-mga Should I open a pull request to update the changelog and close this issue?

A changelog for 3.2.1 and 3.2.2 would be great to have, but in the meantime this can be mentioned in the Known incompatibilities in the Godot 3.2.2 release blog post: https://godotengine.org/article/maintenance-release-godot-3-2-2

in the meantime this can be mentioned in the Known incompatibilities in the Godot 3.2.2 release blog post: https://godotengine.org/article/maintenance-release-godot-3-2-2

Done.


Can't write point cloud data to a PCD file

I'm using an ifm O3D303 ToF camera, I can ping the device and I can see the point-cloud with the ifm3d-pcl-viewer.
I'm trying to save grabbed point cloud to a pcd file following the given ex-file_io.cpp example, but I get the following error when i run it:
terminate called after throwing an instance of 'pcl::IOException'
  what():  : [pcl::PCDWriter::writeASCII] Number of points different than width * height!
Annullato (core dump creato)

The code:
#include <iostream>
#include <memory>
#include <opencv2/opencv.hpp>
#include <pcl/io/pcd_io.h>
#include <pcl/point_cloud.h>
#include "o3d3xx_camera.h"
#include "o3d3xx_framegrabber.h"
#include "o3d3xx_image.h"

int main(int argc, const char **argv)
{
  o3d3xx::Logging::Init();

  o3d3xx::Camera::Ptr cam = std::make_shared<o3d3xx::Camera>();
  o3d3xx::ImageBuffer::Ptr img = std::make_shared<o3d3xx::ImageBuffer>();
  o3d3xx::FrameGrabber::Ptr fg =
    std::make_shared<o3d3xx::FrameGrabber>(
      cam, o3d3xx::IMG_AMP|o3d3xx::IMG_RDIS|o3d3xx::IMG_CART);

  if (! fg->WaitForFrame(img.get(), 1000))
    {
      std::cerr << "Timeout waiting for camera!" << std::endl;
      return -1;
    }

  pcl::io::savePCDFileASCII("point_cloud.pcd", *(img->Cloud()));
  imwrite("amplitude.png", img->AmplitudeImage());
  imwrite("radial_distance.png", img->DepthImage());

  return 0;
}

How can I solve it?
Thanks.

@MYSystemLogistics Thanks very much for the code example -- nothing stands out on a first glance. I'll have a look in the next day or two.

Hi @MYSystemLogistics -- I've had time to investigate and I'm unable to reproduce the issue using the exact code you've pasted along with my o3d303.
First a note: the code shared above is from the libo3d3xx library which is the predecessor of ifm3d and has limited support and no new feature development at this point. Do you have strong need to use libo3d3xx instead of ifm3d? For reference, here is the equivalent sample to what you posted using ifm3d instead (you'll see the usage pattern is nearly identical and should be an easy port):
#include <iostream>
#include <memory>
#include <opencv2/opencv.hpp>
#include <pcl/point_cloud.h>
#include <pcl/io/pcd_io.h>
#include <ifm3d/camera.h>
#include <ifm3d/fg.h>
#include <ifm3d/image.h>

int main(int argc, const char **argv)
{
  auto cam = ifm3d::Camera::MakeShared();
  auto fg = std::make_shared<ifm3d::FrameGrabber>(
      cam, ifm3d::IMG_AMP|ifm3d::IMG_RDIS|ifm3d::IMG_CART);
  auto img = std::make_shared<ifm3d::ImageBuffer>();

  if (! fg->WaitForFrame(img.get(), 1000))
    {
      std::cerr << "Timeout waiting for camera!" << std::endl;
      return -1;
    }

  pcl::io::savePCDFileASCII("point_cloud.pcd", *(img->Cloud()));
  imwrite("amplitude.png", img->AmplitudeImage());
  imwrite("radial_distance.png", img->DistanceImage());

  return 0;
}

That said, I did try to reproduce the issue using your example code based on libo3d3xx and was unable to reproduce the failure. I receive a .pcd file of size 709k which renders correctly in pcl_viewer. Best I can tell, your code looks correct to me.
Could you do a couple things for me:

Output your camera configuration by running ifm3d dump and pasting the output to this issue?
Dump some basic information about the img->Cloud() you're receiving, like so:

std::cout << "Cloud size: ("
               << img->Cloud()->width << ", "
               << img->Cloud()->height << ")" << std::endl;
std::cout << "isOrganized: " << img->Cloud()->isOrganized() << std::endl;

Hi @theseankelly , thank you for your reply.

This is what I see with ifm3d dump:

$ ifm3d dump
{
  "ifm3d": {
    "Apps": [
      {
        "Description": "Run 3D camera at 10Hz without overheating",
        "Id": "476707713",
        "Imager": {
          "AutoExposureMaxExposureTime": "600",
          "AutoExposureReferencePointX": "88",
          "AutoExposureReferencePointY": "66",
          "AutoExposureReferenceROI": "{\"ROIs\":[{\"id\":0,\"group\":0,\"type\":\"Rect\",\"width\":130,\"height\":100,\"angle\":0,\"center_x\":88,\"center_y\":66}]}",
          "AutoExposureReferenceType": "0",
          "Channel": "0",
          "ClippingBottom": "131",
          "ClippingCuboid": "{\"XMin\": -3.402823e+38, \"XMax\": 3.402823e+38, \"YMin\": -3.402823e+38, \"YMax\": 3.402823e+38, \"ZMin\": -3.402823e+38, \"ZMax\": 3.402823e+38}",
          "ClippingLeft": "0",
          "ClippingRight": "175",
          "ClippingTop": "0",
          "ContinuousAutoExposure": "true",
          "ContinuousUserFrameCalibration": "false",
          "EnableAmplitudeCorrection": "true",
          "EnableFastFrequency": "true",
          "EnableFilterAmplitudeImage": "true",
          "EnableFilterDistanceImage": "true",
          "EnableRectificationAmplitudeImage": "false",
          "EnableRectificationDistanceImage": "false",
          "ExposureTime": "7700",
          "ExposureTimeList": "1100;7700",
          "ExposureTimeRatio": "7",
          "FlyingPixelEraser": "2",
          "FrameRate": "10",
          "MaxAllowedLEDFrameRate": "13.8",
          "MinimumAmplitude": "42",
          "Resolution": "0",
          "SpatialFilter": {},
          "SpatialFilterType": "0",
          "SymmetryThreshold": "0.4",
          "TemporalFilter": {},
          "TemporalFilterType": "0",
          "ThreeFreqMax2FLineDistPercentage": "80",
          "ThreeFreqMax3FLineDistPercentage": "80",
          "TwoFreqMaxLineDistPercentage": "80",
          "Type": "upto30m_moderate",
          "UseSimpleBinning": "false"
        },
        "Index": "1",
        "LogicGraph": "{\"IOMap\": {\"OUT1\": \"RFT\",\"OUT2\": \"AQUFIN\"},\"blocks\": {\"B00001\": {\"pos\": {\"x\": 200,\"y\": 200},\"properties\": {},\"type\": \"PIN_EVENT_IMAGE_ACQUISITION_FINISHED\"},\"B00002\": {\"pos\": {\"x\": 200,\"y\": 75},\"properties\": {},\"type\": \"PIN_EVENT_READY_FOR_TRIGGER\"},\"B00003\": {\"pos\": {\"x\": 600,\"y\": 75},\"properties\": {\"pulse_duration\": 0},\"type\": \"DIGITAL_OUT1\"},\"B00005\": {\"pos\": {\"x\": 600,\"y\": 200},\"properties\": {\"pulse_duration\": 0},\"type\": \"DIGITAL_OUT2\"}},\"connectors\": {\"C00000\": {\"dst\": \"B00003\",\"dstEP\": 0,\"src\": \"B00002\",\"srcEP\": 0},\"C00001\": {\"dst\": \"B00005\",\"dstEP\": 0,\"src\": \"B00001\",\"srcEP\": 0}}}",
        "Name": "Test_10Hz",
        "PcicEipResultSchema": "{\"layouter\" : \"flexible\", \"format\": { \"dataencoding\": \"binary\", \"order\": \"big\" }, \"elements\" : [ { \"type\": \"string\", \"value\": \"star\", \"id\": \"start_string\" }, { \"type\": \"records\", \"id\": \"models\", \"elements\": [ { \"type\": \"int16\", \"id\": \"boxFound\" }, { \"type\": \"int16\", \"id\": \"width\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"height\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"length\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"xMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"zMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yawAngle\" }, { \"type\": \"int16\", \"id\": \"qualityLength\" }, { \"type\": \"int16\", \"id\": \"qualityWidth\" }, { \"type\": \"int16\", \"id\": \"qualityHeight\" } ] }, { \"type\": \"string\", \"value\": \"stop\", \"id\": \"end_string\" } ] }",
        "PcicPnioResultSchema": "{\"layouter\" : \"flexible\", \"format\": { \"dataencoding\": \"binary\", \"order\": \"big\" }, \"elements\" : [ { \"type\": \"string\", \"value\": \"star\", \"id\": \"start_string\" }, { \"type\": \"records\", \"id\": \"models\", \"elements\": [ { \"type\": \"int16\", \"id\": \"boxFound\" }, { \"type\": \"int16\", \"id\": \"width\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"height\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"length\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"xMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"zMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yawAngle\" }, { \"type\": \"int16\", \"id\": \"qualityLength\" }, { \"type\": \"int16\", \"id\": \"qualityWidth\" }, { \"type\": \"int16\", \"id\": \"qualityHeight\" } ] }, { \"type\": \"string\", \"value\": \"stop\", \"id\": \"end_string\" } ] }",
        "PcicTcpResultSchema": "{ \"layouter\": \"flexible\", \"format\": { \"dataencoding\": \"ascii\" }, \"elements\": [ { \"type\": \"string\", \"value\": \"star\", \"id\": \"start_string\" }, { \"type\": \"blob\", \"id\": \"normalized_amplitude_image\" }, { \"type\": \"blob\", \"id\": \"distance_image\" }, { \"type\": \"blob\", \"id\": \"x_image\" }, { \"type\": \"blob\", \"id\": \"y_image\" }, { \"type\": \"blob\", \"id\": \"z_image\" }, { \"type\": \"blob\", \"id\": \"confidence_image\" }, { \"type\": \"blob\", \"id\": \"diagnostic_data\" }, { \"type\": \"string\", \"value\": \"stop\", \"id\": \"end_string\" } ] }",
        "RtspOverlayStyle": "{\"ROI\": {\"default\": {\"visible\": true, \"filled\": false, \"use_symbol\": false, \"label_alignment\": \"top\", \"label_content\": \"\", \"label_background\": \"black\", \"label_fontsize\": 8, \"label_failonly\": false}, \"model_defaults\": {}, \"specific\": {} } }",
        "TemplateInfo": "",
        "TriggerMode": "1",
        "Type": "Camera"
      }
    ],
    "Device": {
      "ActiveApplication": "1",
      "ArticleNumber": "O3D303",
      "ArticleStatus": "AD",
      "Description": "",
      "DeviceType": "1:2",
      "EIPConsumingSize": "8",
      "EIPProducingSize": "450",
      "EnableAcquisitionFinishedPCIC": "false",
      "EthernetFieldBus": "0",
      "EthernetFieldBusEndianness": "0",
      "EvaluationFinishedMinHoldTime": "10",
      "ExtrinsicCalibRotX": "0",
      "ExtrinsicCalibRotY": "0",
      "ExtrinsicCalibRotZ": "0",
      "ExtrinsicCalibTransX": "0",
      "ExtrinsicCalibTransY": "0",
      "ExtrinsicCalibTransZ": "0",
      "IODebouncing": "true",
      "IOExternApplicationSwitch": "0",
      "IOLogicType": "1",
      "IPAddressConfig": "0",
      "ImageTimestampReference": "1554451147",
      "Name": "O3D303_test",
      "OperatingMode": "0",
      "PNIODeviceName": "",
      "PasswordActivated": "false",
      "PcicProtocolVersion": "3",
      "PcicTcpPort": "50010",
      "PcicTcpSchemaAutoUpdate": "false",
      "SaveRestoreStatsOnApplSwitch": "true",
      "ServiceReportFailedBuffer": "15",
      "ServiceReportPassedBuffer": "15",
      "SessionTimeout": "30",
      "TemperatureFront1": "3276.7",
      "TemperatureFront2": "3276.7",
      "TemperatureIMX6": "37.984001159668",
      "TemperatureIllu": "42.3",
      "UpTime": "0.998333333333333"
    },
    "Net": {
      "MACAddress": "00:02:01:40:93:AD",
      "NetworkSpeed": "0",
      "StaticIPv4Address": "<IP_ADDRESS>",
      "StaticIPv4Gateway": "<IP_ADDRESS>",
      "StaticIPv4SubNetMask": "<IP_ADDRESS>",
      "UseDHCP": "false"
    },
    "Time": {
      "CurrentTime": "1554451146",
      "MaxNumberOfNTPServers": "5",
      "NTPServers": "",
      "StartingSynchronization": "false",
      "Stats": "",
      "SynchronizationActivated": "false",
      "Syncing": "false",
      "WaitSyncTries": "2"
    },
    "_": {
      "Date": "Wed Mar 18 11:49:49 2020",
      "HWInfo": {
        "Connector": "#!02_A300_C40_03315945_008023176",
        "Diagnose": "#!02_D322_C34_03252158_008026824",
        "Frontend": "#!02_F342_C34_18_00003_008023607",
        "Illumination": "#!02_I300_001_03389895_008001175",
        "MACAddress": "00:02:01:40:93:AD",
        "Mainboard": "#!02_M381_003_03318647_008022954",
        "MiraSerial": "0430-5909-00af-0654"
      },
      "SWVersion": {
        "Algorithm_Version": "2.2.6",
        "Calibration_Device": "00:02:01:40:93:ad",
        "Calibration_Version": "0.9.0",
        "Diagnostic_Controller": "v1.0.30-73a66baf54-dirty",
        "IFM_Recovery": "unversioned",
        "IFM_Software": "1.30.4123",
        "Linux": "Linux version 3.14.34-rt31-yocto-standard-00048-gc8112a91aeb2-dirty (jenkins@dettlx190) (gcc version 4.9.2 (GCC) ) #1 SMP PREEMPT RT Wed Mar 27 18:25:10 CET 2019",
        "Main_Application": "1.0.30"
      },
      "ifm3d_version": 1700
    }
  }
}

Here the basic information of XYZImage().size() and Cloud():

XYZ image width: 176
XYZ image height: 132
Cloud size:<PHONE_NUMBER>, 32703)
isOrganized: 1

I tried to use the ifm3d library instead of the libo3d3xx as you told me, but I have the same error.
What I'm able to do is to create two PNG files of amplitude and radial distance images.

Is it possible that there is an error in my CMakeLists.txt file? This is mine:

project(File_IO CXX)
cmake_minimum_required(VERSION 3.5)
cmake_policy(SET CMP0048 NEW)

# Global compiler flags
set(CMAKE_BUILD_TYPE Release) # Release or Debug
set(CMAKE_CXX_EXTENSIONS OFF) # OFF -> -std=c++14, ON -> -std=gnu++14
set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED true)

# Finding the PCL library
find_package(PCL 1.2 REQUIRED)
include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})

# Finding the OpenCV library
if(FORCE_OPENCV3)
  find_package(OpenCV 3 REQUIRED)
elseif(FORCE_OPENCV2)
  find_package(OpenCV 2.4 REQUIRED)
else()
  find_package(OpenCV REQUIRED)
endif()

# Finding the ifm3d library
find_package(ifm3d 0.11.0 CONFIG
  REQUIRED COMPONENTS camera framegrabber image
  )

# Create executable
add_executable(File_IO File_IO.cpp)
target_link_libraries(File_IO
		      ifm3d::camera
                      ifm3d::framegrabber
                      ifm3d::image
                      ${OpenCV_LIBRARIES}
		      ${PCL_LIBRARIES}
                      ${PCL_COMMON_LIBRARIES}
                      ${PCL_IO_LIBRARIES}
                      )

Can I somehow exploit XYZImage() or Cloud() to separately save into vectors/arrays the X, Y and Z coordinates of the point-cloud?

I see no issue with that config. I've loaded it to my camera and the code is still successful.

Those values look correct to me.

Maybe. Here's my CMakeLists.txt:

cmake_minimum_required(VERSION 3.10)

find_package(ifm3d CONFIG REQUIRED
  COMPONENTS
    camera
    framegrabber
    image)
find_package(PCL 1.7.1 REQUIRED COMPONENTS common io)
find_package(OpenCV REQUIRED)

include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})

add_executable(example main.cpp)
target_link_libraries(
  example
  ifm3d::camera
  ifm3d::framegrabber
  ifm3d::image
  ${PCL_LIBRARIES}
  ${OpenCV_LIBRARIES}
  )

The most obvious difference is the version of PCL being used -- are you actually using PCL 1.2 (from 2011) or is there a more recent version installed?

Sure, that's a good idea. XYZImage() returns a 3-channel cv::Mat of type 16S. You can use cv::split to separate the channels into individual arrays like this:

  cv::Mat xyz[3];
  cv::split(img->XYZImage(), xyz);

And then I used the example here to convert those individual cv::Mats into vectors (but you will loose the organization of rows/cols):
  std::vector<std::int16_t> x;
  if (xyz[0].isContinuous())
  {
    x.assign(xyz[0].data, xyz[0].data + xyz[0].total());
  }
  else
  {
    for (int i = 0; i < xyz[0].rows; ++i)
    {
      x.insert(
          x.end(), xyz[0].ptr<std::int16_t>(i),
          xyz[0].ptr<std::int16_t>(i) + xyz[0].cols);
    }
  }

(Repeat for y and z using xyz[1] and xyz[2].

& 2. I tried with your code and with your CMakeLists.txt file, but again the same error. It seems there is a problem with *(img->Cloud()).
Is it normal that the cloud width is a number around 10^6 and the cloud height is around 32k?
I think savePCDFileASCII is expecting a smaller cloud size, like 176 * 132 ≈ 23k points, which is my camera resolution.

Regarding the version of PCL being used, that 1.2 was just a copy&paste typo. The version installed on my Ubuntu 18.04 is the PCL 1.8.1.

I split the coordinates into vectors as you suggested. I noticed that the new vectors containing the xyz coordinates are full of values like 0 and 255. Is it normal? Which is the unit of measure of the returned XYZImage() channels? I expect them to be in millimeters.
I also tried to create a PCD file following the documentation on the pointcloud website, but when I try to visualize it the pcl_viewer shows me a messy point cloud cube, which is far away from what I see with the ifm3d-pcl-viewer.
Is this due to the lost of the rows/cols organization?

@MYSystemLogistics sorry, I misread your comment and saw the valid dimensions for the XYZImage and missed the fact that the dimensions of the img->Cloud() object were invalid. In my not-repro, the dimensions are as expected (176x132).
Can you grab logs per the guidance here:
https://github.com/ifm/ifm3d/blob/master/doc/troubleshoot.md#ifm3d-logging

@theseankelly no problem.
Since I have no foo.json file, I created a json file saving my ifm3d dump sensor state.
So I made the logging with the IFM3D_VLOG=15 ifm3d config < TestSensorState.json command.
The output of the ifm3d.INFO file is:
Log file created at: 2020/03/23 12:19:55
Running on machine: Arcanite
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0323 12:19:55.346953 19022 frame_grabber_impl.hpp:178] Camera connection info: ip=<IP_ADDRESS>, port=50010
I0323 12:19:56.341259 19023 frame_grabber_impl.hpp:596] FrameGrabber thread done.

There isn't any .ERROR or .WARNING file in the tmp folder.

@MYSystemLogistics sorry, I've given you poor instructions. I didn't re-read them in detail before sending the link and I thought they were more clear. The whole "config < foo.json" part of the instruction is just an example.
The key is, can you set that environment variable IFM3D_VLOG=15 to enable verbose logging, and then reproduce the scenario. For example, you can run IFM3D_VLOG=15 ./example where 'example' is to be replaced by the name of your binary.

@theseankelly Ok, maybe I was taking the example way too literally.
This is the new output:
Log file created at: 2020/03/23 15:57:00
Running on machine: Arcanite
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0323 15:57:00.051069 22814 camera_impl.hpp:406] Increasing XML-RPC response size limit...
I0323 15:57:00.051697 22814 camera_impl.hpp:410] Initializing Camera: ip=<IP_ADDRESS>, xmlrpc_port=80, password=
I0323 15:57:00.051728 22814 camera_impl.hpp:414] XMLRPC URL Prefix=http://<IP_ADDRESS>:80
I0323 15:57:00.126024 22814 camera.cpp:207] Instantiating O3D...
I0323 15:57:00.126368 22814 camera_impl.hpp:406] Increasing XML-RPC response size limit...
I0323 15:57:00.126395 22814 camera_impl.hpp:410] Initializing Camera: ip=<IP_ADDRESS>, xmlrpc_port=80, password=
I0323 15:57:00.126413 22814 camera_impl.hpp:414] XMLRPC URL Prefix=http://<IP_ADDRESS>:80
I0323 15:57:00.126435 22814 camera_impl.hpp:419] Dtor...
I0323 15:57:00.126796 22814 frame_grabber_impl.hpp:474] c_command: 1000L000000644
1000c000000628
      {
        "layouter": "flexible",
        "format"  : {"dataencoding":"ascii"},
        "elements":
         [
           {"type":"string", "value":"star", "id":"start_string"},
           {"type":"blob", "id":"distance_image"},
           {"type":"blob", "id":"normalized_amplitude_image"},
           {"type":"blob", "id":"x_image"},
           {"type":"blob", "id":"y_image"},
           {"type":"blob", "id":"z_image"},
           {"type":"blob", "id":"confidence_image"},
           {"type":"blob", "id":"extrinsic_calibration"},
           {"type":"string", "value":"stop", "id":"end_string"}
         ]
      }

I0323 15:57:00.126845 22814 frame_grabber_impl.hpp:496] t_command: 1001L000000007
1001t
I0323 15:57:00.141258 22814 frame_grabber_impl.hpp:178] Camera connection info: ip=<IP_ADDRESS>, port=50010
I0323 15:57:00.141641 22815 frame_grabber_impl.hpp:514] Framegrabber thread running...
I0323 15:57:00.141701 22815 frame_grabber_impl.hpp:542] Connecting to PCIC...
I0323 15:57:00.157835 22815 frame_grabber_impl.hpp:655] Full ticket: '1000L000000007
1000*
'
I0323 15:57:00.191690 22815 frame_grabber_impl.hpp:655] Full ticket: '0000L000255926
'
I0323 15:57:00.214082 22814 byte_buffer.hpp:275] xyzidx=18446744073709551615, xidx=93032, yidx=139544, zidx=186056, aidx=46520, raw_aidx=18446744073709551615, cidx=232568, didx=8, uidx=18446744073709551615, extidx=255848, gidx=18446744073709551615, intridx=18446744073709551615, invintridx=18446744073709551615
I0323 15:57:00.214159 22814 byte_buffer.hpp:377] xfmt=3, yfmt=3, zfmt=3, afmt=2, raw_afmt=4294967295, cfmt=0, dfmt=2, ufmt=4294967295, extfmt=6, gfmt=4294967295, intrfmt=<PHONE_NUMBER>, invintrfmt=<PHONE_NUMBER>
I0323 15:57:00.214184 22814 byte_buffer.hpp:397] npts=23232, width x height=176 x 132
I0323 15:57:00.274408 22815 frame_grabber_impl.hpp:655] Full ticket: '0000L000255926
'
I0323 15:57:00.371397 22815 frame_grabber_impl.hpp:655] Full ticket: '0000L000255926
'

Again no .ERROR or .WARNING file in the tmp folder.

Hey @MYSystemLogistics -- I'm struggling to get to the bottom of this one without a solid repro -- there's clearly something different between your setup and mine. I wonder if ABI compatibility is coming into play.

For PCL: did you get the library from apt ('libpcl-dev, right?) or build from source?
For ifm3d: Did you build from source or pull from the apt repository?

If apt, could you confirm whether you've used the ROS or non-ROS version of the repository listed on the README?

Which version of ifm3d do you have? (ifm3d --version)

Hi @theseankelly, sorry for my late reply but I think I made a mess trying to fix the problem by my self:

I have a PCL library 1.8 installed from libpcl-dev (usr/include directory) and a PCL library 1.10 installed from source (usr/local/include directory).
I installed the ifm3d library following the README guide, so first I configured the ifm apt server (ROS version), then I Installed the software from apt-get install ifm3d-...
ifm3d --version gives me ifm3d: version=0.17.0

Recently I updated my CMake version from 3.10 to 3.17 and the problem now is that I can't even build the source code: when I launch the cmake .. command from the build folder I get these errors:
CMake Error at CMakeLists.txt:22 (add_executable):
 Target "Save2PCD" links to target "ifm3d::camera" but the target was not
 found.  Perhaps a find_package() call is missing for an IMPORTED target, or
 an ALIAS target is missing?

CMake Error at CMakeLists.txt:22 (add_executable):
 Target "Save2PCD" links to target "ifm3d::framegrabber" but the target was
 not found.  Perhaps a find_package() call is missing for an IMPORTED
 target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:22 (add_executable):
 Target "Save2PCD" links to target "ifm3d::image" but the target was not
 found.  Perhaps a find_package() call is missing for an IMPORTED target, or
 an ALIAS target is missing?

Do you have and idea why?

Small remark: is it possible that the firmware version of my camera is different from yours?

Can you run ldd on your executable to determine which of your two PCL installations is being linked?
Which version of ROS do you have installed (should be melodic for ROS1 or one of the ROS2 variants)?
Camera firmware is not relevant at this layer. That said, I did test with the same version you have installed (it's 1.30.4123, found in the output of the ifm3d dump command you posted above).

Also, since you're using a highly customized setup I might suggest you build ifm3d from source instead. That way you can link against the proper versions of PCL and other libraries you're trying to use. Be sure to source your ROS environment before building.
Instructions are pretty straightforward and can be found here:
https://github.com/ifm/ifm3d/blob/master/doc/source_build.md
And prior to building, sudo apt remove the currently installed ifm3d packages, remove the repository from your apt sources.list, and sudo apt-get update.

Hi @MYSystemLogistics -- I had one more thought yesterday. If you don't actually need a custom version of PCL, I recommend using the version supplied via ROS. Additionally, we recently released ifm3d (including headers/libs and the command line tool) into the ROS distribution for Kinetic/Melodic. You can get them via sudo apt-get install ros-kinetic-ifm3d or sudo apt-get install ros-melodic-ifm3d. Again be sure to remove what you currently have installed and clear the ifm3d PPA from your sources.list.

Hey @MYSystemLogistics just wanted to follow up after a couple weeks.

Hi @theseankelly sorry if I haven't replied, but lately I have dedicated myself to something else. As soon as I get back to PCL I will let you know (I hope to give you some news by Monday).

HI @MYSystemLogistics it's been a while -- just wanted to follow up on this and see if I can help in any way.

Hi @theseankelly, I'm sorry for my extremely late reply.
I "solved" the problem running the code on a different environment. Basically I started from scratch. I tried it both on Nvidia Jetson Nano and on a PC with Ubuntu 20, and it worked.
I think that my Ubuntu 18 PC is totally corrupted, maybe to many wrong installation and libraries conflicts that I'm not able to solve.
I forgot to update you, so sorry again.
Anyway, thank you very much for your support.

Interesting -- too bad I wasn't able to get to the root cause on your Ubuntu 18 system, but it's good to hear the issue was solved!


Permission node context

Is there a way to check use permission nodes in context. something that will only continue if they have the permission. I am to promote user only if they have a permission node. So the /lp user name promote command with a permission context.

https://github.com/lucko/LuckPerms/wiki/Argument-based-command-permissions


WIP: Fix 47627

TBD

Pending testing on container scope

/test images

Pending testing on container scope

Test passed on both tmscopes.

some nits and minor comments, but nothing that should prevent us to merge this PR.
Good work here!


Improve dataset .skip() speed in streaming mode

Feature request
Add extra information to the dataset_infos.json file to include the number of samples/examples in each shard, for example in a new field num_examples alongside num_bytes. The .skip() function could use this information to ignore the download of a shard when in streaming mode, which AFAICT it should speed up the skipping process.
Motivation
When resuming from a checkpoint after a crashed run, using dataset.skip() is very convenient to recover the exact state of the data and to not train again over the same examples (assuming same seed, no shuffling). However, I have noticed that for audio datasets in streaming mode this is very costly in terms of time, as shards need to be downloaded every time before skipping the right number of examples.
Your contribution
I took a look already at the code, but it seems a change like this is way deeper than I am able to manage, as it touches the library in several parts. I could give it a try but might need some guidance on the internals.

Hi! I agree skip can be inefficient to use in the current state.
To make it fast, we could use "statistics" stored in Parquet metadata and read only the chunks needed to form a dataset.
And thanks to the "datasets-server" project, which aims to store the Parquet versions of the Hub datasets (only the smaller datasets are covered currently), this solution can also be applied to datasets stored in formats other than Parquet. (cc @severo)

@mariosasko do the current parquet files created by the datasets-server already have the required "statistics"? If not, please open an issue on https://github.com/huggingface/datasets-server with some details to make sure we implement it.

Yes, nothing has to be changed on the datasets-server side. What I mean by "statistics" is that we can use the "row_group" metadata embedded in a Parquet file (by default) to fetch the requested rows more efficiently.

Glad to see the feature could be of interest.
I'm sure there are many possible ways to implement this feature. I don't know enough about the datasets-server, but I guess that it is not instantaneous, in the sense that user-owned private datasets might need hours or days until they are ported to the datasets-server (if at all), which could be cumbersome. Having optionally that information in the dataset_infos.json file would make it easier for users to control the skip process a bit.

re: statistics:

https://arrow.apache.org/docs/python/generated/pyarrow.parquet.FileMetaData.html
https://arrow.apache.org/docs/python/generated/pyarrow.parquet.RowGroupMetaData.html

>>> import pyarrow.parquet as pq
>>> import hffs
>>> fs = hffs.HfFileSystem("glue", repo_type="dataset", revision="refs/convert/parquet")
>>> metadata = pq.read_metadata("ax/glue-test.parquet", filesystem=fs)
>>> metadata
<pyarrow._parquet.FileMetaData object at 0x7f4537cec400>
  created_by: parquet-cpp-arrow version 7.0.0
  num_columns: 4
  num_rows: 1104
  num_row_groups: 2
  format_version: 1.0
  serialized_size: 2902
>>> metadata.row_group(0)
<pyarrow._parquet.RowGroupMetaData object at 0x7f45564bcbd0>
  num_columns: 4
  num_rows: 1000
  total_byte_size: 164474
>>> metadata.row_group(1)
<pyarrow._parquet.RowGroupMetaData object at 0x7f455005c400>
  num_columns: 4
  num_rows: 104
  total_byte_size: 13064

user-owned private datasets might need hours or days until they are ported to the datasets-server (if at all)

private datasets are not supported yet (https://github.com/huggingface/datasets-server/issues/39)

@versae Dataset.push_to_hub writes shards in Parquet, so this solution would also work for such datasets (immediately after the push).

@mariosasko that is right. However, there are still a good amount of datasets for which the shards are created manually. In our very specific case, we create medium-sized datasets (rarely over 100-200GB) of both text and audio, we prepare the shards by hand and then upload then. It would be great to have immediate access to this download skipping feature for them too.

From looking at Arrow's source, it seems Parquet store metadata at the end, which means one needs to iterate over a Parquet file's data before accessing its metadata. We could mimic Dask to address this "limitation" and write metadata in a _metadata/_common_metadata file in to_parquet/push_to_hub, which we could then use to optimize reads (if present). Plus, it's handy that PyArrow can also parse these metadata files.

So if Parquet metadata needs to be in its own file anyway, why not implement this skipping feature storing the example counts in dataset_infos.json? That would allow:

Support both private and public datasets
Immediate access to the feature upon uploading of shards
Use any dataset, not only those uploaded using .push_to_hub()
A proper Parquet metadata file could still be created and "overwrite" the dataset_infos.json info in the datasets-server.


Add all of the participants in the D&I session at CHAOSSCon EU 2019

In the past, we had one pull request per contributor and asked for their individual consent before merging. Should we do this again?

@GeorgLink that's a good point.
In this case what we did was to ask the people to add themselves during the conference, so they're aware of this.
With this in mind, do you still think we should reach them out with this respect?

Thank's for clarifying. Now that we documented the consent here, we should
be good.


Add metadata about determinism to the planning pipeline

This pull request adds three new features to the PrPy planning pipeline:

DETERMINISTIC_TRAJECTORY Trajectory Tag: Indicates whether repeating the same planning query will produce the same trajectory
DETERMINSTIC_ENDPOINT Trajectory Tag: Indicates whether repeating the same planning query will result in a trajectory with the same endpoint as this trajectory.
determinstic flag on PlanningError: Indicates whether repeating the same planning query could result in success.

The trajectory tags default to being absent and the deterministic flag defaults to None. I strongly suggest logging a warning and not silently defaulting to False if they are not set. It is possible that I missed a few calls in the PrPy motion planners that we need to annotate.
This data metadata is correctly propagated by the Sequence, MethodMask, and FirstSupported MetaPlanners. Note that the behavior of Sequence is a bit unintuitive: it will flag a trajectory as non-deterministic if any previous planner in the sequence was non-determinstic.
There are a few additional points of subtly:

Seeds. I consider the value of a random number generator seed, even if it is passed explicitly as an argument, to be non-deterministic. This is the only useful way to model randomness: almost everything on a computer deterministic if we consider PRNGs to be deterministic.
DETERMINSTIC_ENDPOINT on Sequence. I conservatively flag the endpoint of a trajectory as non-deterministic if a previous failure was non-deterministic. This is overly conservative for PlanToConfiguration, where the endpoint is always deterministic.
Trajopt. We need to port these changes over to or_trajopt separately since it lives in a separate repository.

This now needs to be merged after #334. I had to fix that regression to test this code.


Maven import

@mariobodemann I can't still import in my project the repository. #4

Hey,
that is odd. Can you share your complete pom.xml or build.gradle file with me then? Additionally do you have any more sample to reproduce your issue?
Thanks,
Mario

this is my pom.xml file
pom.txt.txt

Any updates on this? I am also not able to do an import...

Hello @matteo5883
Jitpack doesn’t support repositories with a “dot” in the name, so Jitpack changes it to "~", which is not supported by Maven.
https://github.com/jitpack/jitpack.io/issues/2372
This repo has been renamed from rich-text-renderer.java to rich-text-renderer-java
Changing the artifactId in your pom.xml should solve the problem.


[Chaos] IP address appearing as 'null' in the WARN when the session invalidated at SP creation

Environment
IS 2 node cluster setup fronted by Nginx
Mysql 5.6
JDK 1.8
OS - Ubuntu-14.04-LTS
Browser Chrome 58.0.3029.96
Steps to reproduce the issue

Register SP
Provide SP config params
Click the update button after 20 mins (session timeout)

Observations
The error message does not reflect the session timeout issue, rather then it says "Error while uploading the service provider".

Following WARN appeared in the backend that does not have the IP address in it (IP address appeared as null)
TID: [-1234] [] [2017-12-19 06:44:40,043]  WARN {org.wso2.carbon.server.admin.module.handler.AuthenticationHandler} -  Illegal access attempt at [2017-12-19 06:44:40,0043] from IP address null while trying to authenticate access to service IdentitySAMLSSOConfigService

Expected Results
The error message should indicate the failure in Updating SP has occurred because of the session timeout issue.
In the backend WARN IP address should not be 'Null'

Thank you for your contribution!
We are closing this issue since it has not been prioritized for a long time. Chances are that it has already been solved in more recent versions. If not, we will be re-evaluating this when it becomes a priority.


Question - Ability to auto expand operations by clicking endpoint name in "focused" mode?

I wanted to see if I could start a discussion regarding the new "focused" render style. We really love this style as it is perfect for our relatively large spec (on "read" the page scrolls for quite a long time). I was curious if there are any plans in the future to ever give the option in "focused" mode to default to the first operation in a route when clicking an endpoint, and expanding all of its operations?
To further explain what I am talking about, here's a screenshot of what currently happens when you click an endpoint name in "read" mode (in this case I clicked "Accounts").

It takes you to the first operation, with the endpoint name at the top and its description below it.
Right now when you click an endpoint name in "focused" mode, it takes you only to the endpoint's description.

In our case we aren't even providing descriptions for the endpoints so it just takes the user to a blank screen.
I'm aware that most specs do include descriptions, but I'm just curious if you all would ever consider having some type of option to have clicking an endpoint auto expand its operations (instead of having to click only the arrow), and then it would take you to the first operation if it is there similar to the "read" mode.

We do get 2 polarising request on this.

There are users who would write multipage descriptions for a Tag
And there are some who just use it to group the operation with no description or a one line description. (you fall in this category)

Those who write long descriptions for Tags, request to convert their markdown headings into links on the navigation bar.
your suggestion of clicking Tags should expand the endpoint wont solve your problem coz, focused mode is not a continuous rendering like a read mode and therefore  expanding endpoints wont show up the very next item below it in focused mode. (you can check by manually expanding it using the expand/collapse arrow)
We have received request such as directly take me to the first operation under the tag from those who are'nt heavy user of Tag description.
At this point we are little undecided on how to approach this situation as there are many users who wont use Tag descriptions. and for them the focused rendering of tags looks little incomplete.
we will refine this in future but at present we dont have anything on our design board
I will keep this issue open for others (or you) to comment on the approach they would like to see considering all the aspects

That all makes sense. I figured what I was asking for wasn't possible since it is a bit of a hybrid of both focused mode and read mode.
I do really like this idea.

We have received request such as directly take me to the first operation under the tag from those who are'nt heavy user of Tag description.

I assume this would mean then you could never see the tag description for that operation? I think that would be useful for teams like us that do not utilize the tag descriptions.

I assume this would mean then you could never see the tag description for that operation? I think that would be useful for teams like us that do not utilize the tag descriptions.

yes thats true, however there are other options  we have used to address those who like to avoid tag descriptions. In one of our RapiDoc fork we have introduced an attribute on-nav-tag-click which can take values expand-collapse or show-description
after discussion with the extended team  it was decided to extend this option to the open-source version of RapiDoc. I have updated the code and API documentation
for an example implementation checkout https://mrin9.github.io/RapiDoc/examples/tag-focused-mode.html
we do have plans to revisit this later to bring in more functionality around it, possibly without introducing any breaking change. I think this should address your particular concern.
I will close this issue  as we are tracking this feature internally as a separate one. feel free to reopen or create a new one in case you find something odd

@mrin9 that feature is exactly what we are looking for. Thanks so much you guys are awesome!


Small workflow/getting-started.md fix

Small workflow/getting-started.md fix to make it understandable

Thank you for your pull request.
It looks like this may be your first contribution to an Oro, Inc. open source project, in which case you'll need to sign a Contributor License Agreement (CLA) at https://oroinc.com/b2b-ecommerce/contributor-license-agreement/
If you've already signed the CLA, it's possible we don't have your GitHub username or you're using a different email address. GitHub uses the email address you set in your local Git configuration to associate commits with your GitHub account. Please sign the CLA again using the correct GitHub username and email address or see this help article on setting the email on your git commits.
Once you've signed the CLA, please allow for some time for the submission to be processed.

Thank you for your contribution, @campovski!
We reviewed and refactored the whole workflow bundle documentation. The updated version is available at our new documentation portal https://doc.oroinc.com/backend/entities-data-management/workflows/


Replace OpenTracing with OpenTelemetry

Replace OpenTracing with OpenTelemetry. This will unlock custom metrics to track the performance of flows and actions in core (which we have to figure out about which would be useful for the core team).

Closing as it's tracked in Linear.


Add option to plot residuals

Summary
This PR will give the user the option to pass plot_residual=True to the analysis. This will add a residual plot to the figure.
Details and comments
Some details that should be in this section include:

Came from an issue that was open #1169 .
What tests and documentation have been added/updated
What do users and developers need to know about this change

What needed to do in this PR:

[x] Add support for different subplot sizes.
[x] Add automatic residual calculation.
[x] Plot residuals.
[x] Style the plot's color, legend, limits, etc.

When setting plot_residuals=True in analysis option, a residual plot will be added. currently work only for experiments with 1 plot in its figure.
Example for output with plot_residuals=True for RamseyXY experiment:

Can the residual plot example be moved from the Ramsey manual to the Visualization tutorial? I think that's a place where users are more likely to lok for the information.


Remove pre-commit as a dependency and update docs to suggest brew install

Description
Try to reduce the weight of the project.

I changed my mind.


Nullify damage events while in god mode (fixes #652)

Contains
Nullify damage events while in god mode (fixes #652)
How to test
-Activate "Breathing" module
-Begin play (either new or saved)
-Use pickaxe on blocks, verify they can still be destroyed and picked up
-Find deep water, go swimming
-Verify you begin to lose health after being underwater for the standard period of time
-Surface, enter ghost mode ('F1' then type "ghost")
-Dive back in
-Verify that the oxygen meter goes down, but no health is lost.
Outstanding before merging
I'm new to this project, and just assumed that god mode and ghosting were synonymous, since I couldn't find any references to "god mode" in the code. If I got this wrong, hey, sorry.

Can one of the admins please verify this patch?

@Axydlbaaxr: "ghost" would be the correct current state, yeah. Used to be that in the very old days double-space would enable no-clip and flying, which became known as god mode :-)
Thanks! Will review and test when able.
@GooeyHub: ok to test

Hooray Jenkins reported success with all tests good!

Alright, did some testing, after clearing out my workspace of something weird keeping it from updating to the newest - looking good! :-)
While I can confirm that this works it brings up an interesting question of exactly how we should approach this and similar functionality. I added some notes at https://github.com/MovingBlocks/Terasology/issues/2158#issuecomment-222381471 as I think we need to improve our understanding of what might be a "better" approach
Will add you to the credits shortly - thanks again! And please keep the code coming if you're interested in doing more. If you have/make a forum account we'll set you up with a contributor badge there :-)

Thanks! I am definitely looking to hopefully make larger contributions to the project as my understanding of the code base increases.

Awesome! Just let us know if you have questions or would like some direction on what to do next :-)
You can write a brief intro about yourself here if you like, if you make a forum account you get a contributor badge there as well: http://forum.terasology.org/forum/contributor-introductions.7/


AzureFileCopy@4 does not behave like AzureFileCopy@2/AzureFileCopy@3 when copying files.

Question, Bug, or Feature?
Type: Bug
Enter Task Name: AzureFileCopy@4
Environment

Server - Azure Pipelines

Agent - Hosted:

Issue Description
This is my yaml definition:
- task: AzureFileCopy@2    displayName: 'AzureBlob File Copy'   condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))     inputs:     SourcePath: '$(build.artifactstagingdirectory)/Gaine.Utilities.zip'     azureSubscription: 'Gainecorp'     Destination: 'AzureBlob'     storage: 'gainestorage'     ContainerName: 'software'     BlobPrefix: 'utilities'
When this runs, it copies the Gaine.Utilities.zip file to a "folder" called utilities in blob storage, i.e. /utilities/Gaine.Utilities.zip.
When I run the identical task using v4 (i.e. AzureFileCopy@4), it doesn't act the same.  It creates a file called 'utilities' that has no extension, but it has the expected contents.  Meaning that if I download it and manually add a .zip extension, I can open it and find the expected contents.
The bottom entry is from a V4 task:

Task logs
AzureFileCopy@2 content:
Uploading files from source path: 'D:\a\1\a\Gaine.Utilities.zip' to storage account: 'gainestorage' in container: 'software' with blob prefix: 'utilities'  & "AzCopy\AzCopy.exe" /Source:"D:\a\1\a" /Dest:"https://gainestorage.blob.core.windows.net/software/utilities" /@:"D:\a\_temp\gqnikfaxypsjfwesmu606n7b9" /XO /Y /SetContentType /Z:"AzCopy" /V:"AzCopy\AzCopyVerbose_e0f10ef1-7060-455e-966b-d9d873c5e2df.log" /S /Pattern:"Gaine.Utilities.zip"
AzureFileCopy@4 content:
Uploading files from source path: 'd:\a\1\a\Gaine.Utilities.zip' to storage account: 'gainestorage' in container: 'software' with blob prefix: 'utilities'  & "AzCopy\AzCopy.exe" copy "d:\a\1\a\Gaine.Utilities.zip" "https://gainestorage.blob.core.windows.net/software/utilities"  --log-level=INFO --recursive
The V2 behavior is expected; the V4 behavior is not.  What configuration changes are required to get V4 operating, or should I just stay with V2?  By the way, V3 performs like V2, thankfully.

#13159 seems to be a dupe of this report.

@chriswill  Are you giving the inputs as a yaml file or in the azure task itself ?

@chriswill Please refer to this issue : https://github.com/microsoft/azure-pipelines-tasks/issues/13159
It will resolve your query

closing this as dup with https://github.com/microsoft/azure-pipelines-tasks/issues/13159

I ran into the same problem as described in the original post by Chris and I don't believe it is a permission issue or related to #13159
In the v4 version of the task if you explicitly specify a single file in the SourcePath in combination with a BlobPrefix, the format of the AZ Copy command is ultimately missing a trailing slash. This causes the file that is uploaded to be created with the name of the Blob Prefix.
steps:
- task: AzureFileCopy@4
  displayName: 'AzureBlob File Copy'
  inputs:
    SourcePath: '$(System.DefaultWorkingDirectory)/directory/testFile.txt'
    azureSubscription: subscriptionName
    Destination: AzureBlob
    storage: storageaccountname
    ContainerName: downloadscontainer
    BlobPrefix: blobprefixname

AzCopy command that runs:
"AzCopy\AzCopy.exe" copy "D:\AgentA\_work\r193\a\directory\testFile.txt" "https://storageaccountname.blob.core.windows.net/downloadscontainer/blobprefixname"  --log-level=INFO --recursive

Note that the trailing slash is missing after blobprefixname/. You cannot add the trailing slash yourself as it is stripped out when the task executes.
As a work around -If you specify a star pattern in the SourcePath the file is uploaded as expected to the virtual directory.

I ran into the same problem as described in the original post by Chris and I don't believe it is a permission issue or related to #13159
In the v4 version of the task if you explicitly specify a single file in the SourcePath in combination with a BlobPrefix, the format of the AZ Copy command is ultimately missing a trailing slash. This causes the file that is uploaded to be created with the name of the Blob Prefix.
steps:
- task: AzureFileCopy@4
  displayName: 'AzureBlob File Copy'
  inputs:
    SourcePath: '$(System.DefaultWorkingDirectory)/directory/testFile.txt'
    azureSubscription: subscriptionName
    Destination: AzureBlob
    storage: storageaccountname
    ContainerName: downloadscontainer
    BlobPrefix: blobprefixname

AzCopy command that runs:
"AzCopy\AzCopy.exe" copy "D:\AgentA\_work\r193\a\directory\testFile.txt" "https://storageaccountname.blob.core.windows.net/downloadscontainer/blobprefixname"  --log-level=INFO --recursive

Note that the trailing slash is missing after blobprefixname/. You cannot add the trailing slash yourself as it is stripped out when the task executes.
As a work around -If you specify a star pattern in the SourcePath the file is uploaded as expected to the virtual directory.

I observed the same behaviour. By changing 👍
SourcePath: '$(System.DefaultWorkingDirectory)/directory/testFile.txt'
to:
SourcePath: '$(System.DefaultWorkingDirectory)/directory/testFile.*' the target file was copied properly to the correct folder.

closing this as dup with #13159

This is not a dup I'm afraid - the issue is still unresolved for single files being copied to a folder.

@chshrikh  / @nadesu

This is known issue
https://developercommunity.visualstudio.com/content/problem/1109500/azure-file-copy-task-drops-file-name-when-using-pr.html
and adding trailing slash will break most of the customers check this issue. https://github.com/microsoft/azure-pipelines-tasks/issues/13935
Till AzureFileCopy V3 you should use like below
Source Path : Gaine.Utilities.zip
Blob prefix : utilities
File Path : utilities/Gaine.Utilities.zip

For AzureFileCopyV4 use like this.
Source Path : Gaine.Utilities.zip
Blob prefix : utilities/Gaine.Utilities.zip
File Path : utilities/Gaine.Utilities.zip

I hope it helps


markdown-to-json refuses to work on some aspects

My issue is pretty self-explanatory.
I just have a piece of markdown coming from a file and I pass it through the <Markdown></Markdown> component and it just works... more or less.
If I pass something like _Hello World_ it converts it to italics. However, using # to specify headings won't work and neither do tables.
I don't get any console warning or error, and I just can't find what's wrong

Try wrapping your content with String.raw, often processing React templates perform on strings messes them up

Try wrapping your content with String.raw, often processing React templates perform on strings messes them up

Not working, I keep getting the same error, the content is showed and italics, bold... is formatted correctly, however, I keep getting titles (using #) as normal text and tables are not shown

Gotcha. Any chance you could put together a minimal reproduction? It's hard to help without sample code.

Yeah, sure.
My markdown file looks something like this:
---
title: "Some title"
subtitle: "Some subtitle"
date: "some date"
author: "author"
---

Markdown content

### Title

(just a normal markdown file)
then I receive that file in /[slug].tsx in which slug is the filename. I pass that slug to a function that selects the data of the file just like this:
import matter from 'gray-matter' // This is used to separate the data between --- and the content itself
import fs from 'fs'

const getPostContent = (slug: string) => {
	const folder = 'src/posts/'
	const file = `${folder}/${slug}.md`
	const content = fs.readFileSync(file, 'utf-8')
	const matterResult = matter(content)
	return matterResult
}

and then on my component I just pass to the <Markdown> component the content which is not data between ---
const PostPage = (props: any) => {
	const slug = props.params.slug
	const post = getPostContent(slug)

	return (

				<Markdown>{post.content}</Markdown>

	)
}

The weird thing about it is that some weeks ago it worked absolutedly perfect and then all of a sudden it stopped formatting what I said above.
These are some screenshots:
Markdown code
| Team              | Driver 1         | Driver 2          |
| ----------------- | ---------------- | ----------------- |
| Red Bull Racing   | Verstappen       | **Ricciardo**     |
| Ferrari           | Leclerc          | **Hamilton**      |
| Aston Martin      | **Sainz**        | Stroll            |
| Williams          | _Kimi Antonelli_ | Albon             |
| Stake Kick Sauber | Bottas           | **Hulkenberg**    |
| Mercedes          | Russell          | **Alonso**        |
| McLaren           | Norris           | Piastri           |
| Alpine            | Gasly            | **_??? Unknown_** |
| Visa CashApp RB   | _Liam Lawson_    | Tsunoda           |
| Haas              | _Oliver Bearman_ | Magnussen         |

As you can see, bold and italics are working just fine, but the table isn't

At least visually from that screenshot it looks like the table is intact, are you referring to missing styling or something? The content is aligned into columns as you would expect.

I'll try to reproduce this in a sandbox, thanks for the sample code

Hmmm, that's true.
I thought it wasn't correct because before it was showed with paddings, little separators and all of that, and most importantly, centered.
Now that I think of it, it might not be a problem of the <Markdown> component. It could be all happening because of the Tailwind styles I'm applying in the
 parent element.
I'll try to investigate this and let you know if this was the problem

Turns out it was just a Tailwind problem because I accidentally removed the plugin which allowed me to use the prose class.
Thanks for the help anyways.

Ah ok, thanks for using the library :-)


windows 11 behavior

heya,
win11 is not officially up, but i decided to report the issue, because i loved the idea behind your tool. Even bumped that payment link ;) anyhow. installer works, program works whatsoever. The game i've tested this was with World of Warcraft. Behavior is as follows:

was able to choose from only 1 type of other - adding new cursors seems to makes no effect
same goes for the advanced cursors with animation - only per type (no colors) were available
resizing works pretty fine

that's it, hope to see at least initial update of this lovely shit ;)

Hi.
Sorry for late reply. Planning to give Win11 a look over this weekend.
Thanks for the report and will note that in my test notes :)
Note, this email is not often checked. Where did you see it listed? ;)
Cheers
Alex
Dragonrise Games
On Thu, Sep 23, 2021 at 9:18 PM k1ck3r @.***> wrote:

heya,
win11 is not officially up, but i decided to report the issue, because i
loved the idea behind your tool. Even bumped that payment link ;) anyhow.
installer works, program works whatsoever. The game i've tested this was
with World of Warcraft. Behavior is as follows:

was able to choose from only 1 type of other - adding new cursors
seems to makes no effect
same goes for the advanced cursors with animation - only per type
(no colors) were available
resizing works pretty fine

that's it, hope to see at least initial update of this lovely shit ;)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/PandaTeemo/YoloMouse/issues/22, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABW4KMNJOGHULJEEWNUWLPDUDP3Z7ANCNFSM5EVCVKJQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Allow For Links In Tables That Open New Windows/Tabs

Is your feature request related to a problem? Please describe
When a row in a table is clicked leading to a trigger the only way to link to an external page is if the name for the row is set as the link.
Describe the solution you'd like
It would be pretty useful to be able to embed links within tables so that when the cell is clicked, a new tab or window is opened.
Describe alternatives you've considered
Setting the row name as the link address but this is pretty limiting and prevents more than one link per row.

Thanks for the issue @TravisCouture. Would #428 resolve the issue for you? I am not sure I really understood what you are asking in connection with trigger though.


test_pynnio failing on CircleCI for Python 2.7

After merging #565 into master (3694c07):
======================================================================
FAIL: test_write_segment (neo.test.iotest.test_pynnio.TestPyNNNumpyIO_Spikes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/circleci/repo/neo/test/iotest/test_pynnio.py", line 60, in test_write_segment
    assert_file_contents_equal(self.test_file, write_test_file)
  File "/home/circleci/repo/neo/test/tools.py", line 96, in assert_file_contents_equal
    assert file_digest(a) == file_digest(b), generate_error_message(a, b)
AssertionError: Files have the same size but different contents

The test is failing on CircleCI for Python 2.7. It's passing on CircleCI for Python 3.6, and on Travis.

This test has been failing intermittently for months now, and I've been planning to remove the PyNNIO classes anyway, since PyNN now uses Neo directly rather than writing to its own format. I'll try to find time this week to take care of it.

for months now

years, in fact


Updates documentation to reflect discovery made in #59

This project uses semver, please check the scope of this pr:

[x] #none# - documentation fixes and/or test additions
[ ] #patch# - backwards-compatible bug fix
[ ] #minor# - adding functionality in a backwards-compatible manner
[ ] #major# - incompatible API change

CHANGELOG

Updates documentation to reflect discovery made in #59

👍


Add tests

It's time. Too easy to regress stuff now.
Thinking as a start I'll use WebApplicationFactory to test all the endpoints in the sample app. This will be in-process and thus not native AOT or trimmed. I can then have another test that publishes the app and tests the endpoints for real. Ideally would do it for both PublishTrimmed and PublishAot which will likely require manipulating the project file as I've hit issues passing -p:PublishAot=true/-p:PublishTrimmed=true at the command line as it ends up applying to the source generator project too which causes errors as it targets .NET Standard which isn't compatible with trimming or native AOT.

Added WebApplicationFactory based tests for the sample webapp.


Office 365: better, but...

I use Little Snitch on MacOS.  What a great product.  As a result, I'm very aware, and can fine-control, connections in or out.
Several months ago, merely starting Word from an Office 365 subscription triggered 19 attempts to connect to various servers.
This appears much better now, but as of today, starting Word and Excel (Office 365 subscription) on MacOS still triggers the following connection attempts.
omextemplates.content.office.net
config.edge.skype.com
ocos-office365-s2s.msedge.net
officeclient.microsoft.com
nexus.officeapps.live.com
d.docs.live.net
client-office365-tas.msedge.net

Among other things above, I consider Word and Excel's attempts to connect to Skype are abhorrent.  What do you think?

Maybe we could get a business/corporate extension to handle Microsoft products and  IoT devices (https://github.com/StevenBlack/hosts/issues/493), as it's definitely extremely useful for users of related products but completely irrelevant to everyone else, which is the majority of users.

Hello Steven @StevenBlack and @ScriptTiger,
As I'm actually rebuilding the system behind Ultimate.Hosts.Blacklist I also had to review our whitelist list.
So maybe the following whitelists may help build what @ScriptTiger mentioned as it's one among others used by @pi-hole  ...
https://github.com/deathbybandaid/piholeparser/tree/master/ScriptLsts/Whitelsts/WhiteDomains
Have a nice day/night.

Nice, @funilrys. After looking through it, I think I'd like to revise the terminology of the target software to "bloatware" and call it a bloatware extension. I think that better describes what we're trying to squelch here.

@funilrys If you're looking for whitelists, take a look at my work https://github.com/anudeepND/whitelist

Hey, @anudeepND to be sincere about that, I already look at that and once I saw adf.ly I got away from that list ... Sorry but I do prefer to be sincere, I do think that adf.ly should be whitelisted locally not by a global system like the one I'm building.
Thanks anyway :+1:

I would propose that, going forward, for for anyone who wants this level of "bells and whistles" from their document editor, that this hosts file is definitely and categorically NOT for them.
Somebody convince me otherwise?  I'm interested in the counterpoint.
We had a similar issue awhile ago, wherein the default Windows text editor is unable to distinguish line endings used in our hosts files.  My feeling then, as now, remains: anyone who is such a mass-market-grade computing user, these hosts are not made especially for you.
And just this week, somebody complaining that occasional areas (ads) on msn.com don't show anymore.  Really?  Let me debug that for you.
I'd like to propose that this hosts collection is for smart people who "get" it.  For the others, well, send them to Symantec or McAfee and wish them, good luck and happy computing!

Microsoft and security you say?
Never heard of it


fixed https://github.com/kentcdodds/react-workshop/issues/8 issue

Also added myself to contributers and changed Component to React.Component in ClickCounter - in comments for StopWatch example.

done!

Thanks a ton!


adding additional options to avoid errors when reading options from c…

…onfig file
Pre-review checklist

[ ] I've confirmed that instructions included in README.md are still correct after my changes in the codebase.
[ ] I've added new automated tests to verify correctness of my new code.
[ x] I've confirmed that my changes are up-to-date with the target branch.
[ x] I've described my changes in the section below.

Changes description
Currently when running the snow CLI with some snowsql configurations with commands like
snow streamlit list I get errors when options like warehouse / shcema / role / db are set

Hello, I think I don't understand what and how this PR fixes.
The changes are only in ConnectionParams class in snowcli/connection_params.py. This class is an unused leftover from the merge of snowpark containers commands into the main branch (1295af519d31cecfaf9a703d0ac4a0bd939a70f0). I've just removed it in https://github.com/Snowflake-Labs/snowcli/pull/250.
Additionally, not all github checks have passed and there is no test added to verify that the solution really works.
If you see any errors, can you paste what command you're executing and what is the CLI's output (please include --debug option, for example snow --debug streamlit list).

Ok. So If I update I should not see this class anymore. The reason I posted the change is that when using containers / streamlits and reading my settings from the .snowsql/config file I was getting exceptions because some properties were not recognized. Maybe I was just running an old version. I will update and re-check and you can dis-regard this PR then.


A dot in repository name breaks elm-make

Hey,
not exactly sure where to file this. It appears that if the repository field in the elm-package.json has a dot (other than that in the .git extension), the compiled JS code is broken. You end up with a dot in a variable name, which results in an elm.js:7203 Uncaught SyntaxError: Unexpected token ..

Steps to reproduce
git clone git@github.com:parametric-svg/parametric-svg.surge.sh
cd parametric-svg
git checkout 7d240b3
npm install; node_modules/.bin/elm-make source/Main.elm --output=public/elm.js
  # Or just `elm-make source/…` if you have elm-platform 0.17.1 installed globally
less +7203 public/elm.js

I am not sure there is a totally unambiguous way to handle dots, so the solution may be to disallow dots.

A dot in repository name is necessary when using Github Pages.

I know this is closed but since it's just machine generated code, how about hashing the user name and repository name? Then we could have dots in the user name or repository name.
var _hachibu$hachibu.github.io$Main$renderJs = function (code) {}

var _72ce4c6f002d700abe7c44bb082d929bfddaf1bddb0efed9d27a175d71d880c1$Main$renderJs = function (code) {}

I just ran into this issue as well.
Changing the "repository" field was my solution, but it's not a great solution.
"https://github.com/elam03/elam03.github.io.git" -> "https://github.com/elam03/elam03_github_io.git"
Disallowing dots is not a good solution for repos since they are supported and required by github.io pages (as michallepicki has mentioned).
Hash the name or replace the dots with underscores in the generated code?

replace the dots with underscores

One of our goals here is to stay unambiguous. https://github.com/elam03/elam03_github_io.git is a valid repo name. Hashing seems like a great idea :)

https://github.com/elam03/elam03_github_io.git is a valid repo name, but that repo name does not satisfy the requirements to use Github's webpage hosting.
Github's web-hosting service is free and convenient, so repo names with dots are a common occurrence.

I just hit this today, as a noob. Was completely stumped for hours, until I joined slack and asked someone. Spent about a day debugging my first project with a repo with a dot in it. It's been around for a while. Time to fix this -- and get rid of the "github" requirement. We use aws codecommit for "reasons.".

Dots in project names will be disallowed in an upcoming release
https://github.com/elm-lang/elm-compiler/blob/dev/src/Elm/Package.hs#L128-L129

The issue wasn't about a missing explanation. Do we have a fix for the Github's web-hosting service case that was mentioned several times?

Open an issue with a http://sscce.org demonstrating your concern. When an issue gets long, it becomes about 10 different things, and not all can be addressed. New issues, specific description, good examples.

@evancz where is this on being fixed? Github pages repos still need dots and elm now doesn't allow dots or underscores so it is getting hard to work out how to resolve this.
Can we get elm to escape repo names (or hash them)?
Thanks

For those who have doubts about this being fixed by explicitly disallowing it: the repository field only makes sense for libraries, not applications. I believe the field will be removed altogether for applications in 0.19. As such, this only means one cannot publish a package on package.elm-lang.org where the package-name contains a ..
So github pages repositories aren't really relevant to this discussion :)

If that's the case can we have elm make ignore it? It's really handy to be able to track github urls in the package so that devs can get the issue tracker url etc. from that (rather than relying on their own remote settings  which get distorted by forks etc).
It's okay if not but it would be handy.
Thanks


Hotkey option for show original text

Option which require press hot keys to show original text by hover

Also allow to copy text from this popup