[Feature] how to get the token score(logprob) of  greedy decoder？

Motivation
I want to get the token score(logprob) of output tokens when use greedy search decode
Related resources
If you want to get the score(logprob) of the output token, you can only use tok sampling. If you set k=1, the output score will be 1. If it is greater than 1, topk sampling will output different results, which is not in line with expectations. Is there any way to get the score of the output token by greedy decode?
Additional context
none

Without considering efficiency, you can inference it twice and compute it manually with existing apis.
import torch
from lmdeploy import pipeline, GenerationConfig
pipe = pipeline('/nvme/shared/vicuna-7b-v1.5/', log_level='INFO')
messages = 'hello' # or openai format

output = pipe(messages, gen_config=GenerationConfig(top_k=1))

decorated = pipe.chat_template.messages2prompt(messages)
prompt_tokens = pipe.tokenizer.encode(decorated)
all_tokens = prompt_tokens + output.token_ids
all_logits = pipe.get_logits(all_tokens)
all_scores = torch.softmax(all_logits, dim=-1)
gen_scores = all_scores[0, len(prompt_tokens) - 1:-1]
for i, tk in enumerate(output.token_ids):
    print(i, tk, gen_scores[i, tk])


Add additional AWS Neptune Domains

The following domains are for Neptune services and should be added to the default allow list.
"api.aws", "on.aws", "aws.dev"

Added in https://github.com/aws/graph-notebook/pull/483.


Root component disappears in specific situation

repro: http://codepen.io/posva/pen/MpYVGJ
debug view: http://s.codepen.io/posva/debug/MpYVGJ/wQrPoZYxYJYM
Root instance (vm = new Vue) created with:

string template or render fuction
template depending on data
Using one component

If we modify a reactive property of the vm, the Root component disappears from the devtools:

Commenting that line fixes it:

have same problem.
Then i try use this computed methods, i lost
  computed: {
    getAlbums () {
      return this.$root.base.albums
    },
    getPosts () {
      return this.$root.base.posts
    }
  }

I loose  too, but not sure what triggers it. All I know is:

I load my page, and there's no .
I navigate to other page (I'm using vue-router), and then everything dissapears from the vue-devtools.
I press the vue-devtools refresh button and  is correctly displayed for the rest of the session.

I'm seeing similar issue as well when I set some properties. Haven't been able to exactly pin down the exact trigger conditions.

Same issue here

Since the latest version of vue-devtools I haven't experienced this anymore. It might be worth double checking it's still an issue for you.

Yes! I can see Root again, sweet!

With vue-devtools 3.1.4 and Vue 2.3.4 I keep experiencing the error in my setup.

I accidentally deleted my old repro but I cannot reproduce it anymore 🙁
@skyrpex If you manage to reproduce it in a boiled down codepen (https://codepen.io/pen?template=PWbYJO) or a simple html file, please, could you open a new issue with it?

I did it: https://codepen.io/skyrpex/pen/mwmJRe?editors=1010 (here the debug view).
You can see the Root component if you load the devtools before the timeout in the code happens. Then, if you press the reload button, it'll be gone.
Seems related to:

Altering a data variable in created() using a timeout (haven't tried changing the variable using a button, though).
Using the data mentioned before to display or not an element (v-if or ternary operator).

Thanks a lot!

I can verify that it is the same behaviour for me. As soon as I am modifying a data property on the root component in a callback from either created() or mounted() AND using that property in a v-if to display/hide component => Root is removed from devtools.

same issue here
main.js
import App from "./app.vue";
new Vue({
    el: '#main-app',
    store,
    data: {},
    components: {App},
    render: (h) => h(App)
});

app.vue
<template>
    <app-layout>
        ...
    </app-layout>
</template>
<script>
 export default {
        name: 'MainApp',
        components: {
            AppLayout
        },
       ...
}
</script>

with such configuration i can't see Root and MainApp in the devtools
i tried to remove el: '#main-app' with new Vue({...}).$mount(element)
i tried to replace render: (h) => h(App) with template: '<App/>' but nothing changed

This was happening to me and the problem was due to an invalid component in my router/index.js. Commented out the import and the path reference and problem solved.

I have the exact same problem, with vue-devtools 3.1.6, vue 2.5.2 with a very similar setup to @senid231 above.  No Root entry ever appears.  Even the main layout component doesn't show up.  The root node in my tree is the VApp component from Vuetify.

This is happening to me whenever I load data in the created hook. Would be very grateful if this issue could receive some attention.

Did anyone get a solution to this ?


Problem calling script in _host.cshtml

When a call the script      on _host.cshtm Blazor .NET 6 the following problem happens:
2022-08-18T16:51:01.222Z] Error: Microsoft.JSInterop.JSException: Could not find 'loadScript' ('loadScript' was undefined).
Error: Could not find 'loadScript' ('loadScript' was undefined).
at http://localhost:5251/_framework/blazor.server.js:1:497
at Array.forEach ()
at i.findFunction (http://localhost:5251/_framework/blazor.server.js:1:465)
at E (http://localhost:5251/_framework/blazor.server.js:1:2606)
at http://localhost:5251/_framework/blazor.server.js:1:3494
at new Promise ()
at kt.beginInvokeJSFromDotNet (http://localhost:5251/_framework/blazor.server.js:1:3475)
at http://localhost:5251/_framework/blazor.server.js:1:72001
at Array.forEach ()
at kt._invokeClientMethod (http://localhost:5251/_framework/blazor.server.js:1:71987)
at Microsoft.JSInterop.JSRuntime.InvokeAsync[TValue](Int64 targetInstanceId, String identifier, Object[] args)
at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync(IJSRuntime jsRuntime, String identifier, Object[] args)
at GoogleCaptchaComponent.Components.GoogleRecaptcha.OnAfterRenderAsync(Boolean firstRender)
at Microsoft.AspNetCore.Components.RenderTree.Renderer.GetErrorHandledTask(Task taskToHandle, ComponentState owningComponentState)
It doesn't seem to be finding the script in the lifecycle, what could I do to fix it?

Hi
Working on an update
Thank you for mentioning the issue

Please update the package to version 2.0.0  and put the
<script src="_content/GoogleCaptchaComponent/Scripts/ScriptLoader.js"></script>  in _Layout.cshtml file before
<script src="_framework/blazor.server.js"></script>
DON'T put any script code in _Host.cshtml file


[SPARK-49964][Core] Remove ws-rs-api package

What changes were proposed in this pull request?

To Remove the dependency of javax.ws.rs.ws-rs-api as it's no longer required.

Prior discussion can be found on:

https://github.com/apache/spark/pull/41340
https://github.com/apache/spark/pull/45154

Why are the changes needed?
In the past, the codebase used to have a few .scala classes referencing and using the ws-rs-api, such as https://github.com/apache/spark/commit/b7fdc23ccc5967de5799d8cf6f14289e71f29a1e#diff-9c5fb3d1b7e3b0f54bc5c4182965c4fe1f9023d449017cece3005d3f90e8e4d8R624-R627
However as the time passed by, all usages of ws-rs-api are either got removed / refactored. Hence there is no need to have it import on root POM as now and we can always re-introduce it later, if the usage can be justified again.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Unit-test, to make sure the codebase is not impacted by the removal of the dependency.
Was this patch authored or co-authored using generative AI tooling?
No

@pan3793 Can you have a look on this, since this is a follow up on SPARK-47118


Assert that a class has exactly (declared) fields

Summary
Currently AbstractClassAssert#hasFields() and AbstractClassAssert#hasDeclaredFields() check only that the passed fields exist. It would be good to have an assertion that asserts that a class has exactly the specified (declared) fields.
The signature can of course be discussed, I am more interested in the functionality :)
Example
class MyClass {
   public String fieldOne;
   public String fieldTwo;
   private String fieldThree;
}

//pass
assertThat(MyClass.class).hasFieldsExactly("fieldOne", "fieldTwo");
assertThat(MyClass.class).hasDeclaredFieldsExactly("fieldOne", "fieldTwo", "fieldThree");

//fail
assertThat(MyClass.class).hasFieldsExactly("fieldOne");
assertThat(MyClass.class).hasDeclaredFieldsExactly("fieldOne", "fieldThree");

Java 8 specific ?

NO  : create Pull Request from the 2.x branch

Ok with that but just putting Exactly after has, ex hasExactlyFields

I'm currently integrating the PR but I'm gonna rename the assertions hasOnlyFields and hasOnlyDeclaredFields to be consistent with iterable assertions where containsExactly checks the order of the given elements whereas containsOnly does not.

I also a minor bug in the error message use (shouldHaveExactlyDeclaredFields) where actual fields where passed instead of expected ones. I discovered that is by adding some examples in assert-examples, making them fail and looking at the error message. Adding examples puts you in the user's shoes - always a good thing.

@joel-costigliola thanks for integrating it. I have no problem with the renaming of the assertions. I can do that as well if you want to. I have no problem in updating my PR.
Sorry for the bug...

@filiphr I will finish the work, it's not much, thanks for the PR, always appreciated.

@joel-costigliola no problem, glad I could help :)


fix: cache auto retry on MOVED / ASK / ...

try fix #700
DoCache, DoMultiCache end out with one or some redis transaction like:
MULTI, PTTL {key}, GET {key}, EXEC
When we scale a redis-cluster, PTTL {key} may return a MOVED / ASK error because slot have mark migrate / migrating / migrated to another node, then the transaction may fail.
We hope driver can handle this case automaticlly by just retry the transaction on node which error said.
EDIT:
We tested in our real test environment and it works.

BTW, happened to find out testcase TestClusterClient_SendToOnlyPrimaryNodeWhenPrimaryNodeSelected Do with no slot may fail sometimes. Seems like driver will pick any conn to do initSlot commands, but testcase assume it only use primary node conn

Thank you for catching that. We can just delete the test case because we don't guarantee using primaries for those commands. In the future, nodes may have mixed roles of primaries and replicas.

@rueian all review discussions resolved, please check this out.


File IO issues on Windows 11

Lapce Version
0.2.8
System information
Windows 11
Describe the bug

Files will not open, nor will folders.

On the first click, file open operation will create a ghost tab without any title, on the second click the file name will appear but still no content, further clicks does not show content.
Folders simply does not expand at all. (no response on click)

Creating new files/folders does not work.
File operations do not work.
File content may appear after a long wait.

Additional information

try setting Terminal: Shell in Settings to cmd.exe or powershell.exe or pwsh.exe depending on the terminal you normally use?

Everything seems to be working now!

The option seems to be missing in the new version 0.3.1 and everything breaks again.


Error when having enabled-op false

https://pastebin.com/raw/tMwaw83Z when you have enabled op set to false and then attempt to op in console you get the error and also a way to bypass this in console you can do minecraft:op i have my reason why i want this op thing fully disabled so you can please patch this also the version i am running on is LuckPerms v4.2.7

Did you install/update LP by reloading your server?

no i restarted the server

Please can you try:

Completely stopping your server
Replacing the LP jar you have currently with the one available here: https://luckperms.github.io/
Starting your server

and then trying to reproduce the issue again.

https://pastebin.com/gtSTrQGb
same and also i am using Thermos i do have the special source update it one and also use the latest LuckPerms-Bukkit-Legacy on https://luckperms.github.io/

Error is being thrown on this line
https://github.com/lucko/LuckPerms/blob/b7de0e6901cb1b800b6f13ae7f577b5c8c49c38b/bukkit/src/main/java/me/lucko/luckperms/bukkit/listeners/BukkitPlatformListener.java#L83
Most likely a Thermos issue, not LuckPerms.
I suggest reporting the error to the Thermos issue tracker.


Hardcoded python location in kernel.json breaks after a homebrew upgrade of Jupyter

I had jupyter 4.2.3 installed on MacOS via Homebrew. jjava is installed outside of Jupyter under ~/Library/Jupyter/kernels/java.
Recently upgraded Jupyter to 4.2.5 via brew. Now the kernel fails to start (see the attached screenshot) , because kernel.json has version-specific Python location of "/opt/homebrew/Cellar/jupyterlab/4.2.3/libexec/bin/python". Fixing it required kernel reinstall.

Hopefully this issue will go away on its own after #35 is released in M4, that feature a pure Java, Python-free kernel startup script

1.0-a4 release is out, so no more install.py or any other dependency on a Python


Performance impact of mi_collect

Greetings,
I have exact case described in mi_collect documentation: one long-running thread allocates enormous amount (from tens of thousands to hundreds of thousands per second) of small (1Kb-64Kb) blocks  which are then processed/released by dozens of other worker threads.
What is the performance impact of calling mi_collect every 60 seconds?
What is the difference between force and non-force modes and their relative performance?
Is there a difference in performance that depends on thread calling mi_collect (thread that originally allocated memory vs unrelated housekeeping thread)?

Also curious about these questions. Is the answer from the developers that we should just measure it ourselves?


Composable SoftAssertions entry-point classes

Summary
Soft assertions are good.
I like being able to use soft assertions with my custom exceptions in my code.
One of the selling points of AssertJ is the single entry point, which makes it easy to simply type assertThat() in your IDE and have it automatically give you the list of applicable assertions.
I'd like to be able to easily combine soft assertions entry classes from multiple sources. This is not currently possible as Java doesn't support multiple inheritance for classes. The workaround is clunky - each project must maintain its own soft assertions entry point with all of the assertThats in it that they want.
Switching to a model based on interfaces and Java 8 default methods would make multiple inheritance possible.
Example
Current situation
public class Project1SoftAssertions extends SoftAssertions {
    public MyCustomAssert assertThat(MyCustom a) {
        return proxy(MyCustomAssert.class, MyCustom.class, a);
    }
}

public class Project2SoftAssertions extends SoftAssertions {
    public MyCustom2Assert assertThat(MyCustom2 a) {
        return proxy(MyCustom2Assert.class, MyCustom2.class, a);
    }
}

// Can't do this - multiple inheritance not permitted for classes
public class UberSoftAssertions extends Project1SoftAssertions, Project2SoftAssertions {
}

// Have to do this - lots of repetition if you've got lots of assertions!
public class UberSoftAssertions extends SoftAssertions {
    public MyCustomAssert assertThat(MyCustom a) {
        return proxy(MyCustomAssert.class, MyCustom.class, a);
    }
    public MyCustom2Assert assertThat(MyCustom2 a) {
        return proxy(MyCustom2Assert.class, MyCustom2.class, a);
    }
}

// Also can't do this:
public class JUnitJupiterUberSoftAssertions
    extends JUnitJupiterSoftAssertions,
        Project1SoftAssertions, Project2SoftAssertions {}

Proposed
There will be a single functional interface that all soft assertions classes must implement. This has one default method that is used to generate the proxy classes. Then subinterfaces of this interface can call this method in their default assertThat() implementations. Because the soft assertion entry points can be interfaces, you can combine several of them into a single class.

public interface SoftAssertionsBase {
    <V extends Assert<V, T>, T> V proxy(Class<V> assertClass, Class<T> actualClass, T actual);
}

// Existing implementation with the SoftAssertionsBase implementation added
public class AbstractSoftAssertions implements SoftAssertionsBase {
    // Note that there is also a TODO in the source for proxy() that adds the bound on V,
    // as per below.
    public <V extends Assert<V, T>, T> V proxy(Class<V> assertClass, Class<T> actualClass, T actual) {
        return proxies.createSoftAssertionProxy(assertClass, actualClass, actual);
    }
    ....
}

public interface StandardSoftAssertions extends SoftAssertionsBase {
    // All of the standard assertThat assertions go in here as default methods.
}

public interface Project1SoftAssertions extends SoftAssertionsBase {
    default MyCustomAssert assertThat(MyCustom a) {
        return proxy(MyCustomAssert.class, MyCustom.class, a);
    }
}

public class Project2SoftAssertions extends SoftAssertionsBase {
    public MyCustom2Assert assertThat(MyCustom2 a) {
        return proxy(MyCustom2Assert.class, MyCustom2.class, a);
    }
}

// Now we can do this
public class UberSoftAssertions extends AbstractStandardSoftAssertions
    implements Project1SoftAssertions, Project2SoftAssertions {
}

// And this
public class JUnitJupiterUberSoftAssertions
    extends JUnitJupiterSoftAssertions
    implements Project1SoftAssertions, Project2SoftAssertions {}

// And this, which allows us to get rid of that annoying "softly." prefix on all assertions:
public class MyTest extends SoftAssertions implements Project1SoftAssertions, Project2SoftAssertions {

    @AfterAll
    public void after() {
        assertAll();
    }
}

Please note that, although JDK < 1.8 is dying rapidly, nonetheless the above is still JDK <1.8 compatible because it's only the custom extension entry points that need to be interfaces with default methods. If you're pre JDK 1.8, simply define your custom extension entry points as classes (as you always have) and you're back where we started.

Proposed
There will be a single functional interface that all soft assertions classes must implement. This has one default method that is used to generate the proxy classes. Then subinterfaces of this interface can call this method in their default assertThat() implementations. Because the soft assertion entry points can be interfaces, you can combine several of them into a single class.

Sorry, that should have read "one abstract method" or "one non-default method", not "one default method". Apologies for any confusion.

:eyes:

@kriegfrj sorry for the late feedback, I totally agree having one single entry point for all soft assertions would be great!
One challenge I'm not sure is addressed by the proposed  approach is to collect assertion errors from all different soft assertions types into one place, at the moment that place is the SoftProxies.errorCollector instance built when soft assertions are created).
Said otherwise if you have 2 errors from Project1SoftAssertions and one from Project2SoftAssertions, how do make sure the 3 errors with one call to assertAll()?
Either assertAll() called in the test is aware of all implemented soft assertions and is able to get all errors for each of them (I'm not sure this technically feasible) or the SoftProxies.errorCollector instance is shared between all soft assertions instances (but again, not sure it is possible).
I think the way to go is to create a spike and see if we can make it work.

@kriegfrj sorry for the late feedback, I totally agree having one single entry point for all soft assertions would be great!

No problems @joel-costigliola !
SA Health COVID19 - Update 5 - 250320.pdf

One challenge I'm not sure is addressed by the proposed approach is to collect assertion errors from all different soft assertions types into one place, at the moment that place is the SoftProxies.errorCollector instance built when soft assertions are created).
Said otherwise if you have 2 errors from Project1SoftAssertions and one from Project2SoftAssertions, how do make sure the 3 errors with one call to assertAll()?
Either assertAll() called in the test is aware of all implemented soft assertions and is able to get all errors for each of them (I'm not sure this technically feasible) or the SoftProxies.errorCollector instance is shared between all soft assertions instances (but again, not sure it is possible).

I think the way I put it together takes care of that, because there is only one concrete implementation of the proxy() method (belonging to AbstractSoftAssertions), which points all of the generated assertion classes to the same SoftProxies.errorCollector instance. This must be the case, because only classes can actually have concrete instances, so the n SoftAssertion interfaces can't have their own error collector instances.
The only disadvantage of this is that proxy() must become a public method.

I think the way to go is to create a spike and see if we can make it work.

:+1: If I get some time I'll give it a try; otherwise feel free!

alright I missed that point :+1:

I just realised that proxy() is already a public method, so no problems there!

Hi @joel-costigliola , I've just submitted a "spike" as a PR (#1817). There is an example test case which illustrates combining two soft assertion entry points into one (org.assertj.core.api.abstract_.SoftAssertionsMultipleProjectsTest). Have a look and see if you like it (or not...). :smile:


builders: stop trying subrepos for nacl builds?

The nacl builder fails almost all subrepos.

Stop trying?

We at least need a --subrepos=false flag so we can run "make check" on the
nacl Docker images and verify the nacl environment is correct. Right now it tries to
build the subrepos and fails, so the Makefile would like to skip them.

But perhaps the builders as a whole should skip them.

I guess this was done at some point. I think @adg did it.


doc update release process related to maven

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Using razor engine dynamically

Hey guys,
I'm trying to build an odata endpoint behind which I am sitting the razor engine to allow me to template lots of different things for different scenarios across my organisation.
as part of the endpoint I have defined (on top of the usual CRUD functionality for maintaining template info) a render method.
Here's what I'm trying to do ...
client request:
POST "~/Template(1)/Render()"
Content: { ... any valid lump of json ... }

on the server:
var modelData = await Request.Content.ReadAsStringAsync();
return service.Render(templateId, modelData) {
      var template = db.Get<Template>(templateId);
      var model = ????;
      return RazorEngine.Engine.IsolatedRazor.RunCompile(
              template.Content,
              template.Name,
              null,
              model
      );
}

In order to ensure that the model data I have been given conforms to that which is needed by my template I want to about ask the template what type of model it wants ...
In an ideal world I want to do is something like ...
template.Content = "@model Foo ..."
var modelType = RazorEngine.Engine.IsolatedRazor.GetModelTypeFrom(template.Content);
var model = ParseModelAs(modelType, modelData);
return RazorEngine.Engine.IsolatedRazor.RunCompile(
              template.Content,
              template.Name,
              null,
              model
      );

Is this possible as I can't seem to find anything for this?

No this is not possible as templates can work without any model type as well. As a heuristic you can try to parse and resolve the type in the @model or @inherits line yourself (which don't need to exist at all and can even be overwritten by using a custom template-base-class).

Yes that's what i'm talking about ... if the model directive is in the template I should be able to detect it and then ensure the data I have can be cast to that type of data, but also knowing the type will help with deserialisation of my odata requested model json string

With the implementation on GetModelTypeFrom you are on your own. But feel free to contribute back with a PR :)


MTV.com broken

Checklist

[X] I'm reporting a broken site support
[X] I've verified that I'm running youtube-dl version 2021.12.17
[X] I've checked that all provided URLs are alive and playable in a browser
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.mtv.com/video-clips/s5iqyc/mtv-cribs-dj-khaled']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 2dc375acc
[debug] Python version 3.10.2 (CPython) - macOS-12.2.1-arm64-arm-64bit
[debug] exe versions: ffmpeg 5.0, ffprobe 5.0
[debug] Proxy map: {}
[mtv] mtv-cribs-dj-khaled: Downloading webpage
Traceback (most recent call last):
File "/opt/homebrew/bin/youtube-dl", line 33, in
sys.exit(load_entry_point('youtube-dl==2021.12.17', 'console_scripts', 'youtube-dl')())
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/init.py", line 475, in main
_real_main(argv)
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/init.py", line 465, in _real_main
retcode = ydl.download(all_urls)
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 2068, in download
res = self.extract_info(
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 808, in extract_info
return self.__extract_info(url, ie, download, extra_info, process)
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
return func(self, *args, **kwargs)
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
ie_result = ie.extract(url)
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/common.py", line 534, in extract
ie_result = self._real_extract(url)
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/mtv.py", line 300, in _real_extract
mgid = self._extract_mgid(webpage)
File "/opt/homebrew/Cellar/youtube-dl/2021.12.17/libexec/lib/python3.10/site-packages/youtube_dl/extractor/mtv.py", line 293, in _extract_mgid
mgid = video_player['props']['media']['video']['config']['uri']
TypeError: 'NoneType' object is not subscriptable
Description

Using M1 macOS, able to properly download on other sites. It appears all video clips are no longer downloadable when they had been on prior releases a few months ago. It's unclear whether the site changed.
Example URLs:
https://www.mtv.com/episodes/scyb0g/aeon-flux-utopia-or-deuteranopia-season-1-ep-1
https://www.mtv.com/video-clips/s5iqyc/mtv-cribs-dj-khaled

I expect this is the same problem as found for related sites that use the same technology and should be fixed by PR #30149 when it's revived and merged.

Hey, look what I found:

https://neutron-api.viacom.tech/api/2.9/property?brand=mtv&platform=web&region=US-PHASE1&type=episode&shortId=scyb0g
https://neutron-api.viacom.tech/api/2.9/property?brand=mtv&platform=web&region=US-PHASE1&type=showvideo&shortId=s5iqyc

If it helps, I think I have a working implementation of this now:
PS C:\> mtv -i -a mtv.com/episodes/scyb0g
GET https://neutron-api.viacom.tech/api/2.9/property?brand=mtv&platform=web&region=US-PHASE1&shortId=scyb0g&type=episode
GET https://topaz.viacomcbs.digital/topaz/api/mgid:arc:episode:mtv.com:96defb28-d238-11e1-a549-0026b9414f30/mica.json?clientPlatform=android
GET https://topaz.viacomcbs.digital/h/a/dG9wYXoxYDk2ZGVmYjI4LWQyMzgtMTFlMS1hNTQ5LTAwMjZiOTQxNGYzMGA3YzMwNTVhM2QzZjQ1ZGY0ZjUyZDkzZGZjZTE0NGQwMTQ4NTZjYzUyYFNrZkhuTWZPS1pXR1dHSnM4MFJQSFMxTGZ0VV9yTzBhdWtQU3hrTTRKc2c/master.m3u8?mgid=mgid:arc:episode:mtv.com:96defb28-d238-11e1-a549-0026b9414f30&ts=1633969951-0&pthash=7c3055a3d3f45df4f52d93dfce144d014856cc52&hdnts=exp=1646347175~acl=/h/a/dG9wYXoxYDk2ZGVmYjI4LWQyMzgtMTFlMS1hNTQ5LTAwMjZiOTQxNGYzMGA3YzMwNTVhM2QzZjQ1ZGY0ZjUyZDkzZGZjZTE0NGQwMTQ4NTZjYzUyYFNrZkhuTWZPS1pXR1dHSnM4MFJQSFMxTGZ0VV9yTzBhdWtQU3hrTTRKc2c/*~hmac=cf1875f91a6f4f35eddd27ceb7a9023d83429d02b47c52486430cf1cf2ded01c
Resolution:320x240 Bandwidth:268860 Codecs:avc1.4d401e,mp4a.40.2 Audio:audio0
Resolution:480x360 Bandwidth:410154 Codecs:avc1.4d401f,mp4a.40.2 Audio:audio0
Resolution:576x432 Bandwidth:659235 Codecs:avc1.4d401f,mp4a.40.2 Audio:audio0
Resolution:640x480 Bandwidth:910392 Codecs:avc1.4d401f,mp4a.40.2 Audio:audio0
Resolution:768x576 Bandwidth:1287890 Codecs:avc1.4d401f,mp4a.40.2 Audio:audio0

Its challenging, as MTV uses separate video and audio tracks, and they also encrypt both, with separate keys. In addition, the keys use relative paths, so you have to make sure to resolve them.
https://github.com/89z/mech/blob/v1.48.9/cmd/mtv/mtv.go#L66-L96

FYI I discovered that the Topaz API has been blocked for some episodes of Aeon Flux. For example, 1x1 works fine:
https://topaz.viacomcbs.digital/topaz/api/mgid:arc:episode:mtv.com:96defb28-d238-11e1-a549-0026b9414f30/mica.json?clientPlatform=desktop
supposedly 1x3 should work, but if you try it on the website, you get:

Sorry, this video is not available from your location

and the API fails with desktop or android:
https://topaz.viacomcbs.digital/topaz/api/mgid:arc:episode:mtv.com:96defbbe-d238-11e1-a549-0026b9414f30/mica.json?clientPlatform=desktop
https://topaz.viacomcbs.digital/topaz/api/mgid:arc:episode:mtv.com:96defbbe-d238-11e1-a549-0026b9414f30/mica.json?clientPlatform=android
YT-DLP has a workaround using the MediaGenerator API, but its slow, and it pulls the episode in chapters, rather than one long file. Personally I am switching to Paramount for this show:
https://paramountplus.com/shows/aeon-flux
as the extractor doesnt have these issues.

Maybe off topic, but I just noticed that both MTV and Paramount only offer season one of Aeon Flux:

https://www.mtv.com/shows/aeon-flux
https://www.paramountplus.com/shows/aeon-flux

I'm also having a problem with mtv "Http Internal Server Error 500" Has anyone managed to get it working yet?

See https://github.com/ytdl-org/youtube-dl/issues/31056#issuecomment-1351661662.


MultiPolygon formed not correctly?

MultyPolygon coordinates are represented by array of Polygon objects; however, according to the standard, they should be represented as array of arrays of polygon coordinates.
For example, yours:
{   "type": "MultiPolygon",
    "coordinates":  [
        { "type": "Polygon",
          "coordinates":  [[[0.0,0.0,0.0],[1,0.0,0.0],[3,0.0,2],[2,0.0,1],[-4,0.0,-1],[0.0,0.0,0.0]]]
         }
    ]
}

Should be:
{   "type": "MultiPolygon",
    "coordinates":  [
         [[[0.0,0.0,0.0],[1,0.0,0.0],[3,0.0,2],[2,0.0,1],[-4,0.0,-1],[0.0,0.0,0.0]]]
    ]
}

http://geojson.org/geojson-spec.html#polygon
http://geojson.org/geojson-spec.html#id7
Am I correct? My mate has another generator of geojson, and it produces the second variant, that is why I try to understand, which variant should we use.
I do not mind fixing it on my own and pulling, if it is ok

Ugh I think you are correct, I might be doing the same thing with the multi
line and multi point objects, (I'm not at a computer atm).
I'd be happy if you pulled and fixed them up however I think you will find
most of the time will be spent fixing up doctests, it will be a lot if
iterations of running tests and cutting and pasting the output back into
the code... Sorry to out you through the bother.
Sorry I was going through a doctests phase at the time I wrote this, I've
since moved back to raw quickcheck code cos I find it too tempting to flood
my test suites with unit tests rather than property tests. Makes them a
pain to maintain.
I should probably go through remove them and replace then with props
Sorry again, I'll try to make the library easier to work with

Fixed in #7

Thanks, great! I am continuing testing it in my app :)


Continents lines

D'oh, forgot to create the PR, but made the branch.

@doutriaux1 @aashish24 Sorry, forgot to make the PR for the testdata update.

@chaosphere2112 then @doutriaux1 and I must have seen another image


OverflowError on World's bank data with caravel running on Windows 7 with Python 2.7

OverflowError
OverflowError: mktime argument out of range
Traceback (most recent call last)
File "C:\Python27\lib\site-packages\flask\app.py", line 2000, in call
return self.wsgi_app(environ, start_response)
File "C:\Python27\lib\site-packages\flask\app.py", line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "C:\Python27\lib\site-packages\flask\app.py", line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "C:\Python27\lib\site-packages\flask\app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "C:\Python27\lib\site-packages\flask\app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Python27\lib\site-packages\flask\app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Python27\lib\site-packages\flask\app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Python27\lib\site-packages\flask\app.py", line 1625, in dispatch_request
return self.view_functionsrule.endpoint
File "C:\Python27\lib\site-packages\flask_appbuilder\security\decorators.py", line 26, in wraps
return f(self, *args, **kwargs)
File "C:\Python27\lib\site-packages\caravel\models.py", line 1441, in wrapper
return f(*args, **kwargs)
File "C:\Python27\lib\site-packages\caravel\views.py", line 824, in explore
payload = obj.get_json()
File "C:\Python27\lib\site-packages\caravel\viz.py", line 314, in get_json
return self.json_dumps(payload)
File "C:\Python27\lib\site-packages\caravel\viz.py", line 318, in json_dumps
return json.dumps(obj, default=utils.json_int_dttm_ser, ignore_nan=True)
File "C:\Python27\lib\site-packages\simplejson_init_.py", line 397, in dumps
**kw).encode(obj)
File "C:\Python27\lib\site-packages\simplejson\encoder.py", line 275, in encode
chunks = self.iterencode(o, _one_shot=True)
File "C:\Python27\lib\site-packages\simplejson\encoder.py", line 357, in iterencode
return _iterencode(o, 0)
File "C:\Python27\lib\site-packages\caravel\utils.py", line 312, in json_int_dttm_ser
obj = int(time.mktime(obj.timetuple())) * 1000
OverflowError: mktime argument out of range
The debugger caught an exception in your WSGI application. You can now look at the traceback which led to the error.
To switch between the interactive traceback and the plaintext one, you can click on the "Traceback" headline. From the text traceback you can also create a paste of it. For code execution mouse-over the frame you want to debug and click on the console icon on the right side.
You can execute arbitrary Python code in the stack frames and there are some extra helpers available for introspection:
dump() shows all variables in the frame
dump(obj) dumps all that's known about the object
Brought to you by DON'T PANIC, your friendly Werkzeug powered traceback interpreter

This is a dup of https://github.com/airbnb/caravel/issues/721. Please close and subscribe to the other bug


Add SAHistoryNavigationViewController by @szk-atmosphere

Could you merge my pull request?

merged :smile:

thank you😃


Backport locales to 1.19

Backport locales
Generated by go run build/backport-locales.go release/v1.19

.

.

Change milestone to 1.19.1?

🤖 🎺


mRNA sample transcript quantification

I will set up a run of eXpress and possibly Sailfish on all mRNA samples.

Can pull from https://github.com/kdaily/pcbc_c4_analysis/tree/eXpress.

Running on AWS; almost done!

Finished (long time ago)


Incorrect caching gif images

I use delegate method for transform animated images.
- (UIImage *)imageManager:(SDWebImageManager *)imageManager
 transformDownloadedImage:(UIImage *)image
                  withURL:(NSURL *)imageURL

Method creates new animated image from source, and it works. But after restarting application cached gif image is no longer animated.
Problem consists in transforming the image to binary data.
- (void)storeImage:(UIImage *)image recalculateFromImage:(BOOL)recalculate imageData:(NSData *)imageData forKey:(NSString *)key toDisk:(BOOL)toDisk

imageIsPng is true! And because of this it is transformed as a png. //UIImagePNGRepresentation(image)

@patelmayank We have replaced our GIF component with a library from Flipboard (FLAnimatedImage). Could you try our 4.x branch (instructions) and see if it helps?

Closing. The FLAnimatedImage solution replacing our own GIF is integrated into the 4.x branch.


Swap THOR.RUNE to ETH.UST stuck in pending state on testnet

https://viewblock.io/thorchain/tx/DEDEFFCA6DB617056DDDC1966C2935548FCCECD790557B86449FBB50608E17F6?network=testnet

Could use a little bit more gwei on ropsten, to speed them up. Closing this one.


Merging reports fails if base report doesn't contain all files

I use istanbul on a node.js application where some services are started as subprocesses. To enable code coverage of those, I followed advice for issue #97.
So I start the subprocesses as "istanbul cover /index.js --dir  -- " and code coverage goes into uniquely named subdirectories of ./coverage.
Then I run "istanbul report" to generate a merged coverage report. However, parsing fails if coverage on a subprocess exists for source file names which are not mentioned in the base report. This happens because my tests do not "require" js files used by subprocesses, and the base report has no coverage for those files (the filenames are not included at all), as described in issue #142.
I can work around the problem by creating a no-op test that explicitly does "require" on all the source files from my services (as advised by someone in #142), the filenames then show up in the base report with near-zero code coverage, and parsing at "istanbul report" step works fine, giving me a merged report that includes subprocess coverage. But without the "require" hack, I get this error (doesn't matter which report format is used):
$ istanbul report
Using reporter [lcov]
/usr/local/lib/node_modules/istanbul/lib/object-utils.js:58
var line = statementMap[st].start.line,
^
TypeError: Cannot read property 'start' of undefined
at /usr/local/lib/node_modules/istanbul/lib/object-utils.js:58:44
at Array.forEach (native)
at Object.addDerivedInfoForFile (/usr/local/lib/node_modules/istanbul/lib/object-utils.js:57:37)
at Object.Collector.fileCoverageFor (/usr/local/lib/node_modules/istanbul/lib/collector.js:93:15)
at /usr/local/lib/node_modules/istanbul/lib/report/html.js:517:90
at Array.forEach (native)
at HtmlReport.Report.mix.writeReport (/usr/local/lib/node_modules/istanbul/lib/report/html.js:516:27)
at LcovReport.Report.mix.writeReport (/usr/local/lib/node_modules/istanbul/lib/report/lcov.js:45:19)
at /usr/local/lib/node_modules/istanbul/lib/command/report.js:100:22
at /usr/local/lib/node_modules/istanbul/lib/util/file-matcher.js:35:9

I'm not certain if I ran into the same issue or if this is a issue with nyc but I just got this error:
Ping @bcoe I ran this command nyc ./node_modules/.bin/_mocha ./test/*.js ./test/commands/*.js ./test/parser/*.js --timeout=8000
/home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/object-utils.js:59
                var line = statementMap[st].start.line,
                                           ^
TypeError: Cannot read property 'start' of undefined
    at /home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/object-utils.js:59:44
    at Array.forEach (native)
    at Object.addDerivedInfoForFile (/home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/object-utils.js:58:37)
    at Object.Collector.fileCoverageFor (/home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/collector.js:94:15)
    at /home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/report/text.js:210:27
    at Array.forEach (native)
    at TextReport.Report.mix.writeReport (/home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/report/text.js:208:27)
    at /home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/reporter.js:93:20
    at Array.forEach (native)
    at Object.Reporter.write (/home/ruben/repos/node_redis/node_modules/nyc/node_modules/istanbul/lib/reporter.js:87:30)

I can't reproduce it though.


BUG: Lambdas Environments not Unique

Issue
We specify python3 -m venv venv for all lambdas, this needs to be unique to the lambda to prevent importing unnecessary modules and avoiding conflicts.
Expected behavior
The virtual environment only has the modules necessary for that lambda.
Additional Information
This could lead to some lambdas failing if they were unwillingly importing modules without specifying them in their requirements.txt. Please ensure all lambdas deploy and act as expected after this change.
Example of where this happens:
https://github.com/department-of-veterans-affairs/notification-api/blob/master/.github/workflows/lambda-functions.yml#L159

Hey team! Please add your planning poker estimate with Zenhub @cris-oddball @EvanParish @ianperera @jessecanderson @k-macmillan @kalbfled @trevor2718 @jakehova

The work on this ticket is finished.


Update README.md

spyderlab.org add on of btc aml check alsi support KYT and provide api for exchange

Thank you.

Thank you for your response
On Mon, 10 Jun 2024 at 2:18 PM, Giacomo Giallombardo <
@.***> wrote:

Thank you.
—
Reply to this email directly, view it on GitHub
https://github.com/aaarghhh/awesome_osint_blockchain_analysis/pull/6#issuecomment-2157738737,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BEAPDZ5NV5UKWWWWRQGDV3LZGVR7DAVCNFSM6AAAAABJBU52U6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJXG4ZTQNZTG4
.
You are receiving this because you authored the thread.Message ID:
@.***>


feat: add io toolkit

This pull request introduces the new IO toolkit to control mouse and keyboard inputs, along with the necessary dependencies and tests. The most important changes include adding the pyautogui dependency, implementing the IO toolkit class, and creating tests for the new functionality.
New Toolkit Implementation:

Added pyautogui dependency to pyproject.toml to support mouse and keyboard control. (pyproject.toml)
Implemented the IO class in src/goose/toolkit/io.py to provide tools for moving the mouse, clicking the mouse, and typing text. (src/goose/toolkit/io.py)

Configuration Updates:

Added the io entry point for the new IO toolkit in pyproject.toml. (pyproject.toml)

Testing:

Created unit tests for the IO toolkit methods in tests/toolkit/test_io.py to ensure proper functionality. (tests/toolkit/test_io.py)

oh this is wild I will be trying this!

@Kvadratni this is actuallyu pretty awesome - if you can solve the co-ordinates and calculation stuff (and perhaps some of the prompts of when it is invoked) - do you have any testing tips?


Use H5Pisa_class to circumvent bug

Partially fixes #218. On the the bug in the issue memory usage decreases from 100MB++ to less than 50kB
@stormmathisen and @smutch Would be great if you tested this

Hi @mulimoen. Thanks for looking in to this.
I can confirm a noticeable improvement for my small test run (3,819 groups with ~11,500 datasets).
Before:

After:

There is a large reduction in persistent 96 byte allocations.

Probably makes sense to fix a few clippy complaints as well here?
Add this to changelog?
Why do you say it "partially" fixes #218? Wasn't this the root of the problem?

There are enough breaking clippy complaints that I've opened #220 for those.
Added an entry to the changelog and changed the description of this PR

Rebase on master?

👍


🍒 Cherry pick PR #6168 to staging 🍒

🍒 Cherry pick https://github.com/Expensify/App/pull/6168 to staging 🍒

🚀 Cherry-picked to staging by @OSBotify in version: 1.1.12-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by @yuwenmemon in version: 1.1.12-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[Feature Request] Possibility for configuring custom annotations for node-collector job and pod

As with the scan jobs using trivyOperator.scanJobAnnotations, it should be possible to configure the node collector's annotations
We need this feature to provide job/pod annotations to disable automatic injection of Linkerd proxies.
### Tasks

@Zeratoxx is this is what you are looking for ?

@Zeratoxx is this is what you are looking for ?

No, this is what I was referencing in my initial message :) Unfortunately, this value does not do the trick, it only alters the annotations for the scan jobs.
There could be a misunderstanding of my initial message here is a bit different version:
It should be possible to configure custom annotations for the node collector similar to how it is possible to set custom annotations for the scan jobs with trivyOperator.scanJobAnnotations.

@Zeratoxx I see thanks for clarification ,I will take a look at it

hey @Zeratoxx , looking at the code (https://github.com/aquasecurity/trivy-operator/blob/main/pkg/configauditreport/controller/node.go#L174) , it seems that scanJobAnnotations are also set to the node collector

ok, after tests, trivyOperator.scanJobAnnotations adds annotations at Job level but we would needs them at Job.Spec.Template level

ok, after tests, trivyOperator.scanJobAnnotations adds annotations at Job level but we would needs them at Job.Spec.Template level

Oh, thank you for your investigation! :)
Also nice findings in your tests 😄
So that means it's more like a bug I guess. But I would also say that it isn't very clear that scanJobAnnotations also alters the node collector.. A separate value would be more intuitive I think? 🤔

@sylvainOL @Zeratoxx the annotation is added in the job.spec.template already, can you please advice

@Zeratoxx please reviewed my lastest comment ?

@chen-keinan Sorry, I saw your comment but forgot about it 😅
As far as I see you linked me to the place where ScanJobs are getting built, am I right? If true, that's not what this Github Issue is about.
I created this issue because the contents that a user enters in the helm value scanJobAnnotations will not be applied to the NodeCollector job nor is it possible to set custom annotations solely for the NodeCollector. And that's what I need, if you ask me this directly :)

@chen-keinan Sorry, I saw your comment but forgot about it 😅
As far as I see you linked me to the place where ScanJobs are getting built, am I right? If true, that's not what this Github Issue is about.
I created this issue because the contents that a user enters in the helm value scanJobAnnotations will not be applied to the NodeCollector Job, nor is it possible to set custom annotations solely for the NodeCollector. And that's what I need, if you ask me this directly :)

thanks for clarification adding a fix now

@chen-keinan Soo, it was a bug that gets solved with a version bump?

its not just a version bump, the new version include the fix. see the PR

Ohhh okay, I just didn't understand the project structure, now it's clear to me 😅👍


Version negotiation?

Do we want to have some sort of version negotiation take place when the protocol is first connected? If we think we'll ever want this, specifying it early on will mean all implementations will support it at least well enough to say "no I don't support your protocol version". On the other hand, I expect most implementation distributions will be tightly-coupled enough that the embedder will always know which version of the compiler it's shipping (at least to within a semantic version range).
@strychnide What do you think?

I guess we could have a "hello" message of some kind, not enforcing anything but just outputting informations and letting the implementations do what they want (maybe suggesting to implement the version check/other checks).

If we do this, I think we should at least enforce some kind of behavior so one endpoint doesn't just send something completely incomprehensible to the other. Something like, "if you don't get the handshake you expect, close the connection and emit an error".

A handshake seems fine to me, some kind of "info" outbound message + an "everything ok" inbound message should be good enough, imo.

What would be in the "info" message? How should an endpoint react if the other's message isn't what it expects?

Info message -> everything the host would want to check (atm only the version comes to mind but there may be more).
if a message isn't what it expects maybe we could use the ProtocolError we already defined?


Remediate Michigan Transform Script

Problem
We have recently updated our transform python script framework to
make it easier to use and has far less boiler plate code. We need to migrate
the existing Michigan script to this new framework.
Tasks

[ ] Create a 1,000 record sample file and share in the dropbox folder. Ideally
use resevoir sampling step from Kettle a'la src/main/pdi/NewYork/SampleNewYork.ktr
[ ] Create the correct initializer code in
src/python/national_voter_file/us_states/mi/transfomer.py.
You can use state_transformer_template.py or the Florida transformer for guidance
[ ] We now have new handling for Unparsable USAddresses. Make sure the extract_registration_address
populates RAW_ADDR1, RAW_ADDR2, RAW_ZIP, and RAW_CITY. With uncleansed data straight
from the voter file. If you are able to correcly parse the address with  USAddress
parser then set VALIDATION_STATUS to 2. Set VALIDATION_STATUS to 1 if the parser fails
and leave the Registration data fields set to None
[ ] delete prepare.py
[ ] Test out your transformer using the sample data file and review the results.
[ ] Update the test method in test_transformers.py and run the full nosetests

Feel free to assign this to me! Should be able to make a PR by the end of the day


Session token is invalid

Hello,
I just Upgrade My osiset version 15 to 17.0.1

as Upgrading doc mention. i follow all the steps. and i got this error in get api call.

error: "Session token is invalid."
i also tried with window.sessionToken as per doc mention. but when i tried to use this. i can not get result.
when i console window.sessionToken its show 'undefined'.
if i remove verify.shopify middle-ware then i didn't get this error.
FYI. i am using React-SPA. and for the API call i am using axios.
@osiset

Hello,
any one found solution of this. ?
after upgrading to version 17.0.1 my app is crashed and not working any thing.
can any one please help me with this or any suggestion according to this ?

Hello,
Any updates on this issue.
I'm getting the same issue.  window.sessionToken is 'undefined' sometimes in API call.

I can't replicate this. I don't get undefined at any time. It constantly updates properly. Not sure what I can do to help without info.

I've found there is a race condition if you try and grab the session token too quickly. You get a session token is invalid because you are trying to get the window.sessionToken object too soon on page render.
The solve is fairly simple, just delay your request byt 50.100ms and let the blade file grab the token. Alternatively you can use something like an interceptor in axios to always get the token before any request. .
Here's my example:
axios.interceptors.request.use(function (config) {
    return utils.getSessionToken(window['app-bridge-app']) // requires a Shopify App Bridge instance
        .then((token) => {
            console.log('axios token');
            // Append your request headers with an authenticated token
            config.headers.Authorization = `Bearer ${token}`;
            return config;
        });
});


Issues loading tokenizer/Support loading tokenizer.model?

I'm having issues loading certain models on Huggingface that might largely be an issue with those repos rather than bumblebee.
What I'm seeing:
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "openlm-research/open_llama_3b_v2"})

** (MatchError) no match of right hand side value: {:error, "file not found"}

It looks like it's failing searching for a tokenizer.json. Unfortunately, the huggingface repo ships only with a tokenizer.model and related config files but not a tokenizer.json and it appears quite a few models on huggingface follow suit.
I'm not sure what the effort would be to support loading the model directly or if there are other ways around this.

Usually we look for a different repository that uses the same tokenizer and has tokenizer.json. In this case you can try yhyhy3/open_llama_7b_v2_med_instruct, which is fine-tuned version of the original repo and likely uses the same tokenizer.
According to this paragraph the "fast tokenizer" (dumped/loaded from tokenizer.json) used to give wrong results, but this seems to have been resolved in https://github.com/huggingface/transformers/issues/24233.
We can send a PR to the hf repo with the tokenizer file, which we did for a couple repos in the past, so I will keep this open :)

Thanks a lot! I was looking around pretty hard for a 1B or 3B model to test with on my laptop since I don't have the memory really needed to run with a 7B+ model but that makes sense.
For my own reference and usage, is generating a tokenizer.json as simple as it seems with the following:
# python
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("<model>")
tokenizer.save_pretrained("<dir_to_save_to>")

Do you have any sense for the work to handle the model file natively in elixir is?

@bianchidotdev this is precisely it! When you call AutoTokenizer.from_pretrained it will fetch the vocab/config/merges files and create a "slow tokenizer", then they attempt to convert it to a fast tokenizer if possible. If the conversion works, then tokenizer is "fast tokenizer" and save_pretrained dumps it into tokenizer.json, which is the file we rely on.
If you want to open a PR on the HF repos, here's an example, just make sure you have latest transformers installed locally before doing the conversion. No pressure though, I can also do it later :)

@bianchidotdev I opened a PR while testing a new conversion tool and I noticed you opened one already, thanks!
FTR you don't have to wait for the PR to be merged, you can just reference the PR commit directly:
{:ok, tokenizer} =
  Bumblebee.load_tokenizer(
    {:hf, "openlm-research/open_llama_3b_v2",
     revision: "52944fc4e35e6ca00e733b95df79498728016e1d"}
  )

Also, I improved the error messages in #256, so it will be clear why the tokenizer cannot be loaded. And we have a new section in the README with actions the user may take :)


New icon set

Current internal icon set doesn't fit macos look and program/statusbar icon. It should be replaced if possible.
Action buttons:
add.png, 48x48, used to add new shortcut;
apply.png, 48x48, used to apply shortcut changes;
cancel.png, 48x48, used to cancel shortcut changes;
del.png, 48x48, used to delete shortcut;
down.png, 48x48, used to move shortcut down;
edit.png, 48x48, used to edit shortcut;
test.png, 48x48, used to test shortcut;
up.png, 48x48, used to move shortcut up.
Shortcut list:
disabled.png, 32x32, used to show disabled shortcuts in the list;
failed.png, 32x32, used to show failed shortcuts in the list;
ok.png, 32x32, used to show active shortcuts in the list.
Resources are available here.

Some examples of the ongoing work:


fix: Bump @sveltejs/adapter-static peer dependency version

Bumps the version of @sveltejs/adapter-static in peerDependencies to resolve #33

Merging to Release Candidate branch for final tests and release.


About front Camera

Good Afternoon, i'm using ZXing.Mobile with the rear camera of my devices without issues and it's working great. Now i'm having a strange issue with the front camera, it's not being used on 3 devices i tested.
I'm using Zxing.Net.Mobile 2.2.9 and i've encountered this issue on a Huawei Y5 (Phone), Wico Freddy (Phone), Asus K015 (Tablet).
I'm trying with the simplest code i could think of:
MobileBarcodeScanner.Initialize(Application);
var scanner = new MobileBarcodeScanner();
MobileBarcodeScanningOptions opt = new MobileBarcodeScanningOptions()
{
    UseFrontCameraIfAvailable = true
 };
var result = await scanner.Scan(opt);
When the barcode scanner opens it uses the Back camera and in the log i see the following lines:
01-28 03:51:03.249 D/ZXing.Net.Mobile(10099): Checking Number of cameras...
01-28 03:51:03.260 D/ZXing.Net.Mobile(10099): Found 2 cameras...
01-28 03:51:03.273 D/Mono    (10099): DllImport searching in: '__Internal' ('(null)').
01-28 03:51:03.273 D/Mono    (10099): Searching for 'java_interop_jnienv_get_int_field'.
01-28 03:51:03.273 D/Mono    (10099): Probing 'java_interop_jnienv_get_int_field'.
01-28 03:51:03.273 D/Mono    (10099): Found as 'java_interop_jnienv_get_int_field'.
01-28 03:51:03.280 D/ZXing.Net.Mobile(10099): Found Back Camera, opening...
I'm missing something or i'm unlucky with these devices? I can try with some more phone models, but i don't think that it would solve the issue. Has someone tested a phone model that works for sure?
Thank you.

Not sure if related. We have some issues regarding the front camera. We are unable to start “CameraFacing.Front” in release mode. The front facing camera starts fine in debug mode, when deployed using release the back camera is always launched.
It might be a linking problem
Using v2.2.9. Xamarin native android. tested on multiple devices.

Thanks for reporting this issue!  Unforunately it took me way too long to respond 😭.  Sorry, I apologize!  Recently the source code for this project was completely refactored to modernize it.  Many PR's were included in this effort, and many bugs were hopefully fixed.  Please try out the latest 3.x series of NuGet packages (currently in prerelease).  To try and make the project more maintainable in my spare time going forward, I've decided to close all existing issues to start with a clean slate.  If you're still experiencing this issue on the newest version, please open a new issue with as much detail as possible.  Thank you for your patience and understanding!  Happy scanning!


add translation to Greek

Hil @polisaniftos are currently undertaking a translation, or is this just a (good!) suggestion?

Hi sandraM, yes it was a suggestion , but I can do much of the work by
myself. The only thing is, since I m not a programmer, I do not know how to
proceed. Should I take the white paper and translate it or do something
else ?
Στις Παρ, 15 Μαρ 2019 στις 9:42 μμ ο χρήστης SandraM <
<EMAIL_ADDRESS>έγραψε:

Hil @polisaniftos https://github.com/polisaniftos are currently
undertaking a translation, or is this just a (good!) suggestion?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/DemocracyEarth/paper/issues/387#issuecomment-473417746,
or mute the thread
https://github.com/notifications/unsubscribe-auth/At7yOGNdyQ6flxCckgWwU0frBaEsnW6Iks5vW_e9gaJpZM4bnl3o
.

--
Sent from Gmail Mobile


Update index.md

added related sources adding forms

As with the last PR, this does not related to Cloudflare Pages.
Please refrain from opening further PRs unless it is contributing to the material.


Not logging data from Unity.

Installed Unity plugin two days ago, python installed fine & I entered my API key.
However, no data is being logged to WakaTime from Unity, other IDEs are working fine.
Enabled the WakaTime debug in Unity, and the logs seem to be fine.

The wakatime-client version was old, I updated the client to the current version, it should be working now. Please check it and let me now if it's still failing. https://wakatime.com/api/v1/users/current Use this url to check the last heartbeat.
Thanks.

Works perfectly!
Thanks!


server bug fix: remove network details when network is removed

Description
This PR removes network details when network is removed.

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[x] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_multiplication_x: el7 :heavy_multiplication_x: el8 :heavy_multiplication_x: debian :heavy_check_mark: suse15. SL-JID 1724

@blueorangutan package

@weizhouapache a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_multiplication_x: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1735

@blueorangutan package

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_multiplication_x: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1770

@blueorangutan package

@sureshanaparti a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

Packaging result: :heavy_check_mark: el7 :heavy_check_mark: el8 :heavy_check_mark: debian :heavy_check_mark: suse15. SL-JID 1777

@blueorangutan test centos7 vmware-67u3

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

travis failures are not related to this pr
Tests ha enable/disable feature at cluster and zone level ... === TestName: test_ha_configure_enabledisable_across_clusterzones | Status : EXCEPTION ===
ERROR

and
Tests that an user is unable to remove, modify tags created by admin but should access ... === TestName: test_31_user_cant_remove_update_admin_tags | Status : EXCEPTION ===
ERROR

Trillian test result (tid-2580)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35311 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5675-t2580-vmware-67u3.zip
Smoke tests completed. 91 look OK, 0 have errors
Only failed tests results shown below:

Test
Result
Time (s)
Test File

Merged based on 3 approvals and trillian test result.


Add FIPs exclude list

Update JavaTestRunner to accept FIPS related exclude list.
Ref: /backlog/issues/758

Signed-off-by: Mesbah-Alam<EMAIL_ADDRESS>

Test with this PR: Grinder/23786
Test without the PR ,where the fips related exclude file is not considered: Grinder/23784


update license

Same as zstash, add IM release code and comply to LLNL software guidelines.

@forsyth2 @tomvothecoder Just FYI, this follows llnl software licensing guide (https://software.llnl.gov/about/licenses/) I will just go ahead and Merge.

Just making a note here that this pull request doesn't include a license (as in https://github.com/E3SM-Project/zstash/pull/153) because zppy already has the license file.


Remove woodstox dependency from tests

Description:
Ballerina unit tests were executed with "woodstox" dependencies in place and the correct expected behavior was observed with "woodstox". However, since this dependency was not included in the distribution, run-time behavior was different from behavior during test execution.
We need to remove this dependency from from test execution and correct unit tests accordingly.
(Additionally, it is required to make sure, if user defined the XML literal as  output also should be "", and if the literal was , then it should also print it as is)

@ayomawdb can we resolve this?

@uthaiyashankar No. This issue is still unaddressed and was low prioritized. @SupunS can confirm on this.


Windows compatibility and UTF16 support on string MD5

2 commits:
1- symbols exposing on Windows
2- Added md5_16 method to hash the UTF16 representation of a string

I've tried to test md5_16() method like this:
# sqlite3 :memory:
SQLite version 3.21.0 2017-10-24 18:55:49
Enter ".help" for usage hints.
sqlite> .load ./libsqlitemd5.so
sqlite> select hex(md5(x'feff0061'));
4B936244FB81C0500598B1938266F69A
sqlite> select hex(md5_16(x'feff0061'));
F67048ADB95031C3FF71C13E7EA98F5F
sqlite> .q

# echo -n 'a' | iconv -f utf8 -t utf16 | md5 | tr "[a-z]" "[A-Z]"
4B936244FB81C0500598B1938266F69A

The a character is UTF16-encoded in all of the cases, right?
Why does its hash generated by md5() method match one generated by md5 utility, and md5_16() method gives a different hash.
What am I missing here?

Hi,
don't forgot Endianness:
echo -n 'a' | iconv -f utf8 -t utf16le | od -t x1
0000000 61 00
echo -n 'a' | iconv -f utf8 -t utf16le | md5sum
4144e195f46de78a3623da7364d04f11
sqlite> select hex(md5_16(x'61')); // 'a'
4144E195F46DE78A3623DA7364D04F11
"sqlite3_value_text16" → | UTF-16 TEXT value in the native byteorder
We could fix the choice by using sqlite3_value_text16le or be variants.
Cheers

Thank you for clarification. I forgot to switch encoding in sqlite.
With pragma encoding = "UTF-16"; it works as expected.
Forgive my ignorance, but what benefits does TEXT have over BLOB from the point of view of hash computation? Why can't we use BLOB (the md5() method) for everything and don't care about encoding?

There is no real benefit. I needed, for my purpose, to convert utf8 ->utf16 strings before to md5 it inside a query. Maybe the sqliteshell is offering a way to convert utf8->utf16 directly ?
I was first tempted to write my own program using the lib but my lazyness drived me to google. Then I discovered the possibility to load plugins directly inside the shell by reading your code 👍 .

The simplest approach is to set  "UTF-16" database encoding.
# echo -n a | iconv -f utf8 -t utf16le | md5
4144e195f46de78a3623da7364d04f11

# sqlite3 :memory:
SQLite version 3.21.0 2017-10-24 18:55:49
Enter ".help" for usage hints.
sqlite> .load ./libsqlitemd5.so
sqlite> pragma encoding = "UTF-16";
sqlite> select hex(md5('a'));
4144E195F46DE78A3623DA7364D04F11

I have no objections to add your method as it is handy for some tasks. What if we change the method's name to md5_utf16 ? It seems clearer to me. Would you add a description to the method?

yes md5_utf16 looks good.
We could describe it like that:
"Returns the md5 hash of a converted utf8 to utf6 encoded string"
Feel free to change it since my english isn't that good.
all the best


Refactor chezmoi config template

This PR provides:

Use stdinIsATTY chezmoi helper with $codespaces var
Initialize variables on top
Reuse previous data, when running a chezmoi init on a non-fresh install

This doesn't reuse .email on interative init

I don't get it... why?

This doesn't reuse .email on interative init

I don't get it... why?

Suppose that you enter an empty string when using interative shell, what will be the final value of $email? "" or<EMAIL_ADDRESS>?

❯ chezmoi execute-template --init --promptString email="" <<'EOF'
{{- $codespaces := false -}}

{{- /* declare email variable with a default value */ -}}
{{-   $email :=<EMAIL_ADDRESS>-}}

{{- /* Checks if email is already set */ -}}
{{- if (hasKey . "email2") -}}
{{- /* if yes, use it */ -}}
{{-   $email = .email -}}

{{- /* Check if is interactive */ -}}
{{- else if (not $codespaces) -}}
{{- /* if yes, prompt email */}} -}}
{{-   $email = promptString "email" -}}

{{- end -}}

The email is {{ $email }} !
EOF
 -}}The email is  !

The final value will be "", but I guess this is expected, as it was what the user entered. There are 2 ways to solve it:

https://github.com/twpayne/chezmoi/pull/1531, then promptString "email" $email

Add validation to the email entered, with the regexMatch: https://masterminds.github.io/sprig/strings.html#:~:text=regexMatch%2C mustRegexMatch
But while the validation seems to be a good thing to use, it may be too complicated for this dotfiles... I'm not sure yet.

Chezmoi 2.7.1 was released with this feature, so now it can be $email = promptString "email" $email.


ObservableQuery lifecycle hangs in pending for non-networked fetch policies

We set the lifecycle to pending without consideration of whether we're waiting for anything:
https://github.com/zino-app/graphql-flutter/blob/2efce9f15e59208d86a2c4b1c9e0794cc28ea680/packages/graphql/lib/src/core/observable_query.dart#L192-L194
This was found by @blaugold because it breaks cacheFirst -> refetch() behavior: https://github.com/zino-app/graphql-flutter/issues/346#issuecomment-725305734

closed by #777


Ignore stale transitions

Currently, the dispatchHandler doesn't distinguish between stale (superseded) async transitions and the current (pending) ones. This works fine in most cases since a new transition will cause a cancellation of the current one. So when the old transition is finished, the router will call the abort handler which will simply ignore it.
However, I found two problems with this:

Besides executing the no-op abort handler, finishDispatch also sets pendingTransition to null.  This can be a problem if a stale transition ends while another async one is in progress. If a third transition occurs, the router tries to cancel the ongoing one as usual, but its reference was removed after the completion of the first transition. As a result, that transition will go through once it is complete.

If an async transition is aborted, but another transition is started before it completes, the router will still manipulate the history stack when completed (pop for abort, replace for redirect). This is because the cancellation by the router is ignored if the transition has already been aborted.
It can not happen if you always call the callback immediately after aborting the transition.

The PR contains tests for those two cases, as well as a possible fix.

Yeah I wrote that code and missed these cases. Awesome work.

This is strong work @taurose! Thank you so much.

That was quick :smile: . Glad to have helped.
Just noticed though that I changed the error handling behavior when a custom non-throwing handler is used (abort handler or run callback will still be called).
Thinking about this, maybe the default error handler should receive and abort the transition instead of throwing? This way, with the current code, location.pop() will be called by default (unless superseded, or transition was manually redirected/cancelled before). For server rendering you could just use onAbort to catch both transition.abort and callback errors. And it would allow for more flexible error handling (redirecting, retrying, or letting transition go through after dispatching some actions) while keeping location history intact. Error could still be thrown at the very end to make people aware.


How to name the env file?

How exactly am I supposed to name the env file? I have tried numerous naming conventions, but none of them allow the properties in my env file to be detected by the process.
If I use the name .env, I get this error message:
TypeError [ERR_INVALID_URL]: Invalid URL
Any ideas? TIA

I had to delete the npm modules and re-run npm install.


TeX icon not visible enough

By @alaymari from #42
Regarding the icons, I noticed that the TeX file has an icon which is so thin, it is nearly invisible. Can a different icon be set for that; maybe the txt file icon itself would be better than the existing one.

I have made a svg of tex-file-icon (sort of) which is similar to the prescription file icon of font-awesome.
Can that be used as TeX file icon?

I don't think it's possible to integrate easily icons from an another source than the nerd-font icons.
As this issue is open since a while and nobody have reacted on it it doesn't seems to be an issue so I close it. Don't hesitate to reopen it if needed.


FCOS head Bug：x1 y1 will be overstep the boundary and are set to 0。

Describe the bug
Before batched_distance2bbox function， all the inputs value are same between original model and trt model. But the outputs of the function are different, there are lots of x1, y1 are set to 0 in trt model, while the values of x1, y1 in original model are correct.
After I change batched_distance2bbox(points, bbox_pred, x.shape[2:]) to batched_distance2bbox(points, bbox_pred,(800, 1333)), the results turn to right, and the results is same to the original model。 But I don‘t know why？
In  batched_distance2bbox(points, bbox_pred, x.shape[2:]) function， I print(max_shape), and it's actually toch.Size([800, 1333])。Why x1, y1 are set to 0 ？

Hi
Are you using the latest code? Please update amirstan_plugin, torch2trt_dynamic and mmdetection-to-tensorrt and try again.
Also, note that the current master requires mmdet>=2.18 which is slightly different from the previous version.
If you still have the same problem, please provide your environment and script to reproduce the error.


Contacts Address on Admin

Works correctly on local & live system. Probably have been an error on test system.


Unable to download 360p video?

I am experimenting with Youtube-dl, and trying to fetch a science playlist.
keybounceMBP:science michael$ youtube-dl --version
2015.02.28
keybounceMBP:science michael$ youtube-dl -f height=360 "https://www.youtube.com/watch?v=wvtRXjQXaI0&index=1&list=PLbo
c_P9zcJhiqIetHQeNxNT8R5KnVSNE0"
[youtube:playlist] Downloading playlist PLboc_P9zcJhiqIetHQeNxNT8R5KnVSNE0 - add --no-playlist to just download video wvtRXjQXaI0
[youtube:playlist] PLboc_P9zcJhiqIetHQeNxNT8R5KnVSNE0: Downloading webpage
[download] Downloading playlist: CMB Playlist!
[youtube:playlist] playlist CMB Playlist!: Collected 5 video ids (downloading 5 of them)
[download] Downloading video 1 of 5
[youtube] wvtRXjQXaI0: Downloading webpage
[youtube] wvtRXjQXaI0: Extracting video information
[youtube] wvtRXjQXaI0: Downloading DASH manifest
ERROR: requested format not available

According to the help (call youtube-dl with -h) you need brackets around the height parameter, it also shows quotes around it, like this:
youtube-dl -f "[height=360]"
I believe the quotes are for if there are spaces in the argument, might not always need them but safer to put them anyway.

youtube-dl -f "[height=360]"

Aha! That (the brackets) did it. Thank you.
Somehow, I thought those were only for an expression.

Entertaining minecraft videos
http://YouTube.com/keybounce


Per-line excludes

Hi,
I've just started using lychee and it's fantastic!
I've got one suggestion, sometimes my code contains URL fragments such as:
    let update_link = format!(
        "https://raw.githubusercontent.com/Nitrokey/nethsm-sdk-py/main/tests/{}",
        file_name
    );

And the "not-quite-really-valid-URL" in there makes lychee sad.
I could add raw.githubusercontent.com to exclusion list but that'd unnecessarily exclude other links I want checked.
Most linters allow for granular exclusions, up to the level of the line. I've searched docs, issues and discussions but didn't find anything so my suggestion is to support a marker that'd make the URL in that particular line ignored.
An example:
    let update_link = format!(
        "https://raw.githubusercontent.com/Nitrokey/nethsm-sdk-py/main/tests/{}", // lychee:ignore
        file_name
    );

Thanks for your time!
(If this suggestion is not a good idea, feel free to close the ticket)

Are you testing Rust files with lychee? 😉
You could exclude links like so
lychee --exclude 'https://raw.githubusercontent.com/Nitrokey/nethsm-sdk-py/main/tests/'

it supports regex, so you might also try
lychee --exclude '\{\}' ...

but that won't work, because the strings get encoded:
https://raw.githubusercontent.com/Nitrokey/nethsm-sdk-py/main/tests/%7B%7D

but this would work
lychee --exclude '%7B'

Is that enough in your case?
Markers were discussed here, but it's a slippery-slope. The gist is that parsing is hard.

Are you testing Rust files with lychee? 😉

Yes, I've got quite extensive documentation in these doc-comments :sweat_smile:

Is that enough in your case?

Well, I ended-up marking these URLs in the config file, which, I guess, is good enough: https://gitlab.archlinux.org/archlinux/signstar/-/merge_requests/48/diffs#9a1a8f7c952ee697ad8a1b56a05a26b78db09c19_0_5

Markers were discussed https://github.com/lycheeverse/lychee/issues/1444, but it's a slippery-slope. The gist is that parsing is hard.

Agreed. I'll close this ticket even though it's not resolved as there's a link to prior discussion.
Thanks! :wave:

Nice. One final addition: you can also put exclusions into a dedicated .lycheeignore file. I see that you already have a lychee.toml, so you're all set, but just in case someone finds this and wants to quickly ignore a few links permanently, that's another way to do that. I realized the documentation around that functionality was lacking, so I updated it.

.lycheeignore is an excellent idea since the rest of the config won't change as often as this one. Thanks! 🙇♂


tests: Actually test data_loc output

I realized that the previous test would match against "Attaching 1
probe..." output. Which makes the test meaningless. This makes the test
actually test output.

Checklist

[ ] Language changes are updated in docs/reference_guide.md
[ ] User-visible and non-trivial changes updated in CHANGELOG.md
[ ] The new behaviour is covered by tests

Annoying b/c we can't control if there are any interrupts


Failed to run hand_tracking_gpu on Nvidia Jetson Nano with a CSI camera

Hi,
I manage to build a successful mediapipe in Nvidia Jetson nano with a CSI camera, and also success  to run hand track on CPU with below modification  modify the demo_run_graph_main.cc
const char* gst = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=1280, height=720, format=(string)NV12 ! "
                     "nvvidconv ! video/x-raw, format=BGRx, width=640, height=480, framerate=(fraction)30/1 ! "
                     "videoconvert ! video/x-raw, format=BGR ! "
                     "appsink";

  cv::VideoCapture capture;
  const bool load_video = !FLAGS_input_video_path.empty();
  if (load_video) {
    capture.open(FLAGS_input_video_path);
  } else {
    capture.open(gst, cv::CAP_GSTREAMER);
  }
  RET_CHECK(capture.isOpened());

  cv::VideoWriter writer;
  const bool save_video = !FLAGS_output_video_path.empty();
  if (!save_video) {
    cv::namedWindow(kWindowName, /*flags=WINDOW_AUTOSIZE*/ 1);
#if (CV_MAJOR_VERSION >= 3) && (CV_MINOR_VERSION >= 2)
    capture.set(cv::CAP_PROP_FRAME_WIDTH, 640);
    capture.set(cv::CAP_PROP_FRAME_HEIGHT, 480);
    capture.set(cv::CAP_PROP_FPS, 30);
#endif
  }

But right now I have an error when running hand track GPU ,

I copy the same pipeline as above and modify in file demo_run_graph_gpu.cc
export GST_DEBUG=2
rebuild the project and run it, somehow it returns the error as shown below

kokheng@jetson:~/mediapipe$ bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11   mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu
INFO: Analyzed target //mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
INFO: From Linking mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu:
bazel-out/aarch64-opt/bin/external/org_tensorflow/tensorflow/lite/nnapi/_objs/nnapi_implementation/nnapi_implementation.o:nnapi_implementation.cc:function (anonymous namespace)::ASharedMemory_create(char const*, unsigned long): warning: the use of `tmpnam' is dangerous, better use `mkstemp'
Target //mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu up-to-date:
  bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu
INFO: Elapsed time: 23.094s, Critical Path: 21.72s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
kokheng@jetson:~/mediapipe$ GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu   --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_mobile.pbtxt
I20201108 17:14:37.662433 17633 demo_run_graph_main_gpu.cc:51] Get calculator graph config contents: # MediaPipe graph that performs multi-hand tracking with TensorFlow Lite on GPU.
# Used in the examples in
# mediapipe/examples/android/src/java/com/mediapipe/apps/handtrackinggpu.

# GPU image. (GpuBuffer)
input_stream: "input_video"

# Max number of hands to detect/process. (int)
input_side_packet: "num_hands"

# GPU image. (GpuBuffer)
output_stream: "output_video"
# Collection of detected/predicted hands, each represented as a list of
# landmarks. (std::vector<NormalizedLandmarkList>)
output_stream: "hand_landmarks"

# Throttles the images flowing downstream for flow control. It passes through
# the very first incoming image unaltered, and waits for downstream nodes
# (calculators and subgraphs) in the graph to finish their tasks before it
# passes through another image. All images that come in while waiting are
# dropped, limiting the number of in-flight images in most part of the graph to
# 1. This prevents the downstream nodes from queuing up incoming images and data
# excessively, which leads to increased latency and memory usage, unwanted in
# real-time mobile applications. It also eliminates unnecessarily computation,
# e.g., the output produced by a node may get dropped downstream if the
# subsequent nodes are still busy processing previous inputs.
node {
  calculator: "FlowLimiterCalculator"
  input_stream: "input_video"
  input_stream: "FINISHED:output_video"
  input_stream_info: {
    tag_index: "FINISHED"
    back_edge: true
  }
  output_stream: "throttled_input_video"
}

# Detects/tracks hand landmarks.
node {
  calculator: "HandLandmarkTrackingGpu"
  input_stream: "IMAGE:throttled_input_video"
  input_side_packet: "NUM_HANDS:num_hands"
  output_stream: "LANDMARKS:hand_landmarks"
  output_stream: "HANDEDNESS:handedness"
  output_stream: "PALM_DETECTIONS:palm_detections"
  output_stream: "HAND_ROIS_FROM_LANDMARKS:hand_rects_from_landmarks"
  output_stream: "HAND_ROIS_FROM_PALM_DETECTIONS:hand_rects_from_palm_detections"
}

# Subgraph that renders annotations and overlays them on top of the input
# images (see hand_renderer_gpu.pbtxt).
node {
  calculator: "HandRendererSubgraph"
  input_stream: "IMAGE:throttled_input_video"
  input_stream: "DETECTIONS:palm_detections"
  input_stream: "LANDMARKS:hand_landmarks"
  input_stream: "HANDEDNESS:handedness"
  input_stream: "NORM_RECTS:0:hand_rects_from_palm_detections"
  input_stream: "NORM_RECTS:1:hand_rects_from_landmarks"
  output_stream: "IMAGE:output_video"
}
I20201108 17:14:37.664275 17633 demo_run_graph_main_gpu.cc:57] Initialize the calculator graph.
I20201108 17:14:37.674010 17633 demo_run_graph_main_gpu.cc:61] Initialize the GPU.
I20201108 17:14:37.942391 17633 gl_context_egl.cc:158] Successfully initialized EGL. Major : 1 Minor: 5
I20201108 17:14:38.036972 17639 gl_context.cc:324] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 32.4.4)
I20201108 17:14:38.037261 17633 demo_run_graph_main_gpu.cc:67] Initialize the camera or load the video.
0:00:00.437926497 17633   0x5598add720 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<nvvconv0> transform could not transform video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 in anything we support
0:00:00.438348953 17633   0x5598add720 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<nvvconv0> transform could not transform video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 in anything we support
0:00:00.438382443 17633   0x5598add720 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<capsfilter0:src> could not send sticky events
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 120.000005
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
0:00:01.268876380 17633   0x5598add720 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<nvvconv0> transform could not transform video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 in anything we support
0:00:01.269302116 17633   0x5598add720 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<nvvconv0> transform could not transform video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 in anything we support
0:00:01.269372378 17633   0x5598add720 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<nvarguscamerasrc0> error: Internal data stream error.
0:00:01.269400243 17633   0x5598add720 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<nvarguscamerasrc0> error: streaming stopped, reason not-negotiated (-4)
[ WARN:0] OpenCV | GStreamer warning: Embedded video playback halted; module nvarguscamerasrc0 reported: Internal data stream error.
0:00:01.269863637 17633   0x5598add720 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<nvvconv0> transform could not transform video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 in anything we support
0:00:01.270207237 17633   0x5598add720 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<nvvconv0> transform could not transform video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 in anything we support
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
[ WARN:0] OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] OpenCV | GStreamer warning: GStreamer: pipeline have not been created
E20201108 17:14:39.488559 17633 demo_run_graph_main_gpu.cc:192] Failed to run the graph: ; capture.isOpened()(mediapipe/examples/desktop/demo_run_graph_main_gpu.cc:81)
kokheng@jetson:~/mediapipe$

I have run successfully on hand track on GPU with below changes on demo_run_graph_gpu.cc

modify the pipeline as shown below
disable capture.set()

  const char* gst = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=640, height=480, format=(string)NV12, framerate=(fraction)30/1 ! "
                    "nvvidconv ! video/x-raw, format=(string)BGRx ! "
                     "videoconvert ! video/x-raw, format=(string)BGR ! "
                     "appsink";

  cv::VideoCapture capture;
  const bool load_video = !FLAGS_input_video_path.empty();
  if (load_video) {
    capture.open(FLAGS_input_video_path);
  } else {
    capture.open(gst, cv::CAP_GSTREAMER);
  }
  RET_CHECK(capture.isOpened());

  cv::VideoWriter writer;
  const bool save_video = !FLAGS_output_video_path.empty();
  if (!save_video) {
    cv::namedWindow(kWindowName, /*flags=WINDOW_AUTOSIZE*/ 1);
#if (CV_MAJOR_VERSION >= 3) && (CV_MINOR_VERSION >= 2)
    //capture.set(cv::CAP_PROP_FRAME_WIDTH, 640);
    //capture.set(cv::CAP_PROP_FRAME_HEIGHT, 480);
    //capture.set(cv::CAP_PROP_FPS, 30);
#endif

@kokheng1025 Hi, could you please share your steps about how you can run mediapipe on the Nano GPU? I can run all the CPU examples on Nano, but all failed when I tried to run the GPU examples.
PS: The GPU examples can be built successfully.
Thanks a lot and expect your kindly reply.

I have run successfully on hand track on GPU with below changes on demo_run_graph_gpu.cc

modify the pipeline as shown below
disable capture.set()

  const char* gst = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=640, height=480, format=(string)NV12, framerate=(fraction)30/1 ! "
                    "nvvidconv ! video/x-raw, format=(string)BGRx ! "
                     "videoconvert ! video/x-raw, format=(string)BGR ! "
                     "appsink";

  cv::VideoCapture capture;
  const bool load_video = !FLAGS_input_video_path.empty();
  if (load_video) {
    capture.open(FLAGS_input_video_path);
  } else {
    capture.open(gst, cv::CAP_GSTREAMER);
  }
  RET_CHECK(capture.isOpened());

  cv::VideoWriter writer;
  const bool save_video = !FLAGS_output_video_path.empty();
  if (!save_video) {
    cv::namedWindow(kWindowName, /*flags=WINDOW_AUTOSIZE*/ 1);
#if (CV_MAJOR_VERSION >= 3) && (CV_MINOR_VERSION >= 2)
    //capture.set(cv::CAP_PROP_FRAME_WIDTH, 640);
    //capture.set(cv::CAP_PROP_FRAME_HEIGHT, 480);
    //capture.set(cv::CAP_PROP_FPS, 30);
#endif

could you please share your steps about how you can run mediapipe on the  GPU?such as how to build android aar

I have run successfully on hand track on GPU with below changes on demo_run_graph_gpu.cc

modify the pipeline as shown below
disable capture.set()

  const char* gst = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=640, height=480, format=(string)NV12, framerate=(fraction)30/1 ! "
                    "nvvidconv ! video/x-raw, format=(string)BGRx ! "
                     "videoconvert ! video/x-raw, format=(string)BGR ! "
                     "appsink";

  cv::VideoCapture capture;
  const bool load_video = !FLAGS_input_video_path.empty();
  if (load_video) {
    capture.open(FLAGS_input_video_path);
  } else {
    capture.open(gst, cv::CAP_GSTREAMER);
  }
  RET_CHECK(capture.isOpened());

  cv::VideoWriter writer;
  const bool save_video = !FLAGS_output_video_path.empty();
  if (!save_video) {
    cv::namedWindow(kWindowName, /*flags=WINDOW_AUTOSIZE*/ 1);
#if (CV_MAJOR_VERSION >= 3) && (CV_MINOR_VERSION >= 2)
    //capture.set(cv::CAP_PROP_FRAME_WIDTH, 640);
    //capture.set(cv::CAP_PROP_FRAME_HEIGHT, 480);
    //capture.set(cv::CAP_PROP_FPS, 30);
#endif

could you please share your steps about how you can run mediapipe on the  GPU?such as how to build android aar


Add 8-task scores for rinna bilingual models

Description
Evaluated the following rinna bilingual models with our 8-task setting

https://huggingface.co/rinna/bilingual-gpt-neox-4b
https://huggingface.co/rinna/bilingual-gpt-neox-4b-instruction-sft
https://huggingface.co/rinna/bilingual-gpt-neox-4b-instruction-ppo

Hi, thank you for adding the evaluation for Rinna models. I have a question,
How to decide which prompt to be used for evaluating model?
In this foundation model case, why v0.2 is being used instead of v0.3?
because the accuracy result of both prompt are much different.
Thank you!

@effendijohanes it really depends on the data format during training.
Please see the below link for each prompt version.
https://github.com/Stability-AI/lm-evaluation-harness/blob/jp-stable/docs/prompt_templates.md


waterBoost persistence

The waterBoost entity can persist to other maps when loading directly into them from a map that uses it. This can cause its effect to persist into maps it when it shouldn't. This doesn't seem to happen when exiting normally.
"waterboost persists to other maps if you console load, presumably because Level.OnExit isn't invoked and it's stored in a weird meta object instead of the session" -@Kalobi

Fixed in 1.6.11, thanks for reporting!


Gives AGhosts feet for ghost socks

About the PR
Adds feet slots for AGhosts
Why / Balance
Neptune wants to add socks for AGhosts so they need this
Technical details
I went into the afterlife and broke the ghosts legs so they can fit into them
Media

Breaking changes
Theres no whitelist for what can go on AGhosts feet, i might get around to it at somepoint but since its an admin only thing its not that big
Changelog
Added: Admin Ghost Feet!

I like this, mind just adding all the clothes inventory components to admin ghosts while you are at it?

I like this, mind just adding gloves to ghosts while you're at it? also you should be able to just add a new tag called "GhostEquippable" or something and whitelist the slot to only accept that.

not sure how to get whitelists properly working, only reference i can see is for containers, and i cant decipher clothing slot whitelists

I like this, mind just adding gloves to ghosts while you're at it? also you should be able to just add a new tag called "GhostEquippable" or something and whitelist the slot to only accept that.

not sure how to get whitelists properly working, only reference i can see is for containers, and i cant decipher clothing slot whitelists

I'll look into it later, this is fine for now.


Time to process task, dispatchDeadline

I created the cloud task to call HTTP, in the documentation it says the default timeout is 10 minutes, but can be set to 30 minutes.
When my task runs, it times out at 60 seconds. In the task queue page I see:
Dispatch time
Nov 19, 2020, 9:07:52 PM
Time late
0.006881s
Time to process task
60.064613s
Status code
13 (INTERNAL)
Reason to retry
INTERNAL(13): HTTP status code 502

How to I change the time? Is it set during task creation or queue creation?

It turns out the 60 sec timeout was caused by firebase 60 second limit on cloud functions. It wasn't obvious because I use both google and firebase for cloud functions.
But, How do I set the timeout to be 30 minutes? I know the option is called dispatchDeadline, just don't know how or where to pass it? I'm using nodejs with google sdk.

@teebu are you using firebase functions? you should be able to specify a longer timeout in configuration:
https://firebase.google.com/docs/functions/manage-functions
It has a maximum timeout of 540 seconds however.

https://firebase.google.com/docs/hosting/functions
Note: Firebase Hosting is subject to a 60-second request timeout. Even if you configure your HTTP function with a longer request timeout, you'll still receive an HTTP status code 504 (request timeout) if your function requires more than 60 seconds to run. To support dynamic content that requires longer compute time, consider using an App Engine flexible environment.

Re-open if necessary


Pull leventov:utf8 into utf8 branch

@leventov
I want to pull this into a branch instead of master so I can view it in my IDE.

Coverage increased (+0.03%) to 97.114% when pulling 9f92f25708923c89915eb6cb0406c92dea6bf5e4 on leventov:utf8 into 240b580384f8be8d78b80871e2e17be298779827 on DataSketches:utf8.