Updating to 3.22.0 via ghost-cli may corrupt site settings

Issue Summary
When upgrading to 3.22.0 via ghost-cli the settings cache does not get reset following settings migrations on startup. This leads to the settings cache being in an inconsistent state with the database, and the settings API returning an empty settings array [] to the admin client.
If a user subsequently attempts to modify any settings via Ghost-Admin (such as enabling Members), the server settings get overridden with null/false values.
Who's affected?
This bug only affects sites which:

were installed prior to version 3.22.0
were upgraded to 3.22.0 via ghost-cli
were not restarted again prior to saving settings via Ghost-Admin, and
the site administrator attempted to update the site settings via Ghost-Admin

Restore Settings
If you have installed version 3.22.0 but not attempted to update any settings, you must restart Ghost by issuing ghost restart immediately, and upgrade to the latest version as soon as possible.
If you have installed version 3.22.0 and have experienced corruption of your site's settings, you may be able to restore your settings:

Rollback to 3.21.1
Import the JSON backup file created by ghost-cli as part of the initial ghost upgrade to 3.22.0 (stored in ./content/data)
Update to the latest version

To Reproduce

ghost install 3.21.1 --local
Set up the site and set up members
ghost update 3.22.0 --local
Visit the labs settings and see members is disabled; attempt to enable it and observe the isLang validation error.
ghost update 3.22.2 --local
Visit the labs settings and see the Stripe Subscription price settings are not rendering. Ghost logs errors when trying to configure stripe.
select * from settings;
See that the stripe_plans and other settings have an empty value, and that all bool values are now false.

Possible States

Prior to the 3.22.0 update, the database is in a consistent state.
Immediately following the update to 3.22.0, the database contains the correct settings values, but the settings API returns an empty settings array settings: []
After attempting to update the site settings via Ghost-Admin, the settings in the database are overridden with empty (null/false) values

Technical details:

Ghost Version: 3.22.0
Node Version: any
Browser/OS: any
Database: any

This will result in the stripe_plans setting being empty, the correct fix for this is to rollback to 3.21.1 and restore your settings from the backup created at migration (stored in ./content/data).
There is a migration for 3.23 in PR which forces the stripe_plan setting to the default value if they are not configured correctly

Closing as this has been fixed in 3.22.1 onwards, and directions for restoring broken settings are available in the issue description.


Assets of visited url not found

I have a basic laravel setup that automates the testing of  legacy php application. Somehow the images of the legacy app are not show in the dashboard screenshots.
Any ideas ?

I also cannot find the resources in any of my directory structures. I assume they are downloaded, but I am not sure where they are stored

Also the dashboard does not show the grey areas, probabably due to missing css components ?


Option to redirect output.

It is nice to put the results from benchmark runs in version control so that perf can be tracked over time.
Currently I hack it like this:
    using System.Collections.Generic;
    using System.IO;
    using BenchmarkDotNet.Reports;
    using BenchmarkDotNet.Running;

    public class Program
    {
        private static readonly string DesinationDirectory = System.IO.Path.Combine(new DirectoryInfo(Directory.GetCurrentDirectory()).Parent.Parent.FullName, "Results");

        public static void Main()
        {
            foreach (var summary in RunAll())
            {
                CopyResult(summary.Title);
            }
        }

        private static IEnumerable<Summary> RunAll()
        {
            ClearAllResults();
            var switcher = new BenchmarkSwitcher(typeof(Program).Assembly);
            var summaries = switcher.Run(new[] { "*" });
            return summaries;
        }

        private static IEnumerable<Summary> RunSingle<T>()
        {
            var summaries = new[] { BenchmarkRunner.Run<T>() };
            return summaries;
        }

        private static void CopyResult(string name)
        {
#if DEBUG
#else
            var sourceFileName = System.IO.Path.Combine(Directory.GetCurrentDirectory(), "BenchmarkDotNet.Artifacts", "results", name + "-report-github.md");
            System.IO.Directory.CreateDirectory(DesinationDirectory);
            var destinationFileName = System.IO.Path.Combine(DesinationDirectory, name + ".md");
            File.Copy(sourceFileName, destinationFileName, true);
#endif
        }

        private static void ClearAllResults()
        {
            if (Directory.Exists(DesinationDirectory))
            {
                foreach (var resultFile in Directory.EnumerateFiles(DesinationDirectory, "*.md"))
                {
                    File.Delete(resultFile);
                }
            }
        }
    }

We could add an additional string property in the config (like ArtifactFolderName, can be absolute or relative) with BenchmarkDotNet.Artifacts as a default value.
Probably, we also have to add some option to disable creating of the artifact folder. In most cases, I don't need the artifacts, I just look at the summary table.

It would nice if there were a general purpose sink for this.

I'm not sure about it. If you want to send result in a custom format to a custom sink, you always could take the summary and implement export manually. Do we really need an additional layer of abstraction for this?
@adamsitnik, what do you think?

We could add an additional string property in the config (like ArtifactFolderName, can be absolute or relative) with BenchmarkDotNet.Artifacts as a default value.

I like this idea and I believe it should not break anything an be easy to implement.
As for the sinks: it's a great feature, but IMO we (BenchmarkDotNet) should only provide an api for that, which could be implemented by the community. Example: serilog. Currently it could be achieved my implementing custom IExporter?

@adamsitnik, do you want to implement it? =)

@AndreyAkinshin sure. When are you releasing 0.10.0?

Probably it would be early October. I did a lot of changes in the measurement engine (some additional changes are coming) and I need some time to carefully check all my benchmarks. =)


Implement settings file for Ollama agent

PR Summary
This PR implements basic settings file for Ollama agent:

The settings file is stored at $HOME\.aish\agent-config\ollama\ollama.config.json
Currently only model and endpoint parameters are stored in the config file.
Default config file is created when none exists:

{
    // To use Ollama API service:
    // 1. Install Ollama:
    //      winget install Ollama.Ollama
    // 2. Start Ollama API server:
    //      ollama serve
    // 3. Install Ollama model:
    //      ollama pull phi3

    // Declare Ollama model
    "Model": "phi3",
    // Declare Ollama endpoint
    "Endpoint": "http://localhost:11434"
}

PR Context

This PR allows user to specify custom Ollama model  and  Ollama enpoint location, instead of hardcoded values.
This partially implements #155 (no history and streaming)
@StevenBucher98 : This is quick and dirty, but I needed the settings file ASAP to test the different models without constant agent recompilation. I'm open for suggestions.

@StevenBucher98 I have just discovered OllamaSharp nuget package that would simplify the agent code and provide easy streaming service. Would you consider switching to it?

OllamaSharp uses the MIT license, so it's totally fine to use it for this agent.
I will close this PR to favor the other one.


Try to get it work for paycoin

Hey there,
so i spend hours now to try to get this nice thing working for a new coin called paycoin but i am not able to get it to work. So it seems never to connect right.
Plese contact me okay? I am also willing to pay for it to set it up.
Thanks!
My Mail<EMAIL_ADDRESS>

Edit network.js in the bitcore module (and in /lib/bitcore.js
Update genesis and merkle
For the genesis block, split actual hash into a string of hex pairs and flip each pair and also reverse the order of the pairs like this:
00 01 02 03 becomes
30 20 10 00

So i had many people now Try to get this coin work and no one was able to. The problem is that paycoin xpy has another timestamp Format. Also we was able To get the Genesis Block information, but was not able To find the Magic number of that Coin. I hope someone could set this up for me :( so if You are able To do so genecyber please add me On Skype. The Name is ticer89. Thank you!


add original swagger name as metadata for operations

ApiView needs to know the original swagger name. We should add a private _metadata var for each operation that includes an entry for it's original swagger name. For example, generated operations should look like this:
def generated_operation(self, ...):
    ...
generated_operation._metadata = {"operationId": "OperationGroup_GeneratedOperation"}

Both legacy and version tolerant generations should have this private _metadata var.

implemented this with crossLanguageDefinitionId


[Web Portal] Refine home page

available gpu chart (chartjs -> c3)
virtual cluster list

Coverage increased (+0.01%) to 51.234% when pulling 077608408b2093ee0e1b5ad193373653a1af97b7 on qinsu/portal/c3 into 27c44a6c625eafca0e292d6606bdff153bf34a70 on master.


added default initializer for pointers in generator.hpp

trying to make shure all primitives (in this case pointers) are initialized with some defined values

Thanks a lot


Incorrect atomic initializer for arm 32/64 in libponyrt memory pool

Hi,
I was tying to compile ponyc on 32bit armv7 cpu (armada xp MV78460, Xenial with gcc 5.4.0) and got a warning/an error while compiling src/libponyrt/mem/pool.c:
# the same warning for all 16 elements of pool_global array

src/libponyrt/mem/pool.c:104:3: note: (near initialization for ‘pool_global[0].central’)
src/libponyrt/mem/pool.c:104:3: error: braces around scalar initializer [-Werror]
src/libponyrt/mem/pool.c:104:3: note: (near initialization for ‘pool_global[0].central’)
src/libponyrt/mem/pool.c:104:55: error: excess elements in scalar initializer [-Werror]
   {POOL_MIN << 0, POOL_MAX / (POOL_MIN << 0), {{NULL, 0}}},

I did some investigation and it looks like this is happening because central field of pool_global_t
struct is defined as a simple pointer on non x86:
typedef struct pool_global_t
{
  size_t size;
  size_t count;
  pool_central_t* _Atomic central;
} pool_global_t;

So correct way to initialize pool_global on arm (both 32 and 64 if I understand correctly) will be:
{POOL_MIN << 0, POOL_MAX / (POOL_MIN << 0), NULL}

I created simple macro as a temporary workaround, but not sure if it's good enough as a permanent solution
#ifdef PLATFORM_IS_X86
#  define POOL_CENTRAL_INIT {{NULL, 0}}
#else
#  define POOL_CENTRAL_INIT NULL
#endif

static pool_global_t pool_global[POOL_COUNT] =
{
  {POOL_MIN << 0, POOL_MAX / (POOL_MIN << 0), POOL_CENTRAL_INIT},
  ...
  {POOL_MIN << 15, POOL_MAX / (POOL_MIN << 15), POOL_CENTRAL_INIT},
};

That fix sounds good to me.

Is this resolved by #2665, or was that a different issue?

@jemc No, it's a different issue.

Fixed by #2668.


Build test fix

I noticed that the build test is successful on your latest PR (#49), when it actually should fail because of two things (see the log I linked to above):

gettext is not installed in the testing environment
The zipping process fails, because there is no locale directory. It wasn't created because of the above error.

I already committed a fix for (1) on this branch.
How can we make sure to pass on any error code that might get thrown by a command in our scripts?
Yes, we could query the exit status of each and every command we execute, but there has to be a more elegant solution to this.
I am too tired to work on this right now, but might have a look during next week.
Ideas are very welcome!

Thank you for spotting this! I think a set -e at the beginning of each of the scripts should do the trick. Maybe you could add this to this PR!
PS: Do you live in GMT+1? Maybe you should be sleeping at this time? :rofl: ... :sleeping: ... This is dedication!

Yep, GMT+1 😂🤙🏼
I have a night shift today and I usually stay up late the night before, so that my sleep rhythm doesn't change too abruptly - I just woke up 😊

Done! :)

Thank you once more!


Add Log > Trips section

This page will:

[x] Appear under the Log section
[x] Show all of the Trips a User has created, or is part of
[x] Add OneToMany relationship for Trips to Users (ie allow more than one User to be attached to a trip)
[ ] Trip visibility - can be public, or visible only to those attached

This will likely need a new table user_trips - this can be used to record the users attendance of a trip

Removing trip visibility from scope


Make node id configurable

Could you make the node id configurable? for me data is provided on http://192.168.xxx/data.json?node_id=5 which makes it impossible to use the integration.

here you go... update to 1.0.12 and adjust the node number during the initial setup - if this works, please close this issue TIA (and you might like to use my tibber-invite-link, if you haven't used one yet)

Thanks a lot works perfect now. Have used an invite already though


[BUG] photo_upload and photo_upload_to_story fails when logged in via sessionid

Try Instagrapi SaaS with a free trial https://hikerapi.com/p/5GBWznd3
Describe the bug
When logging in via a sessionid and calling the photo_upload or the photo_upload_to_story function the post results in a
POST /post HTTP/1.1" 500
To Reproduce
Login using a sessionid and call the photo_upload or  upload_to_story method
client = Client()
sessionid = "xxxx"
 try:
        result = client.login_by_sessionid(sessionid)
    except (PrivateError, ValidationError) as e:
        print(f"An error occurred: {e}")

try:
        if upload_to_feed:
            media = cl.photo_upload(image_path, caption=content)
            if media:
                print("Photo uploaded to feed successfully!")
            else:
                print("Failed to upload photo to feed.")

        if upload_to_story:
            media = cl.photo_upload_to_story(image_path, caption=content)
            if media:
                print("Story shared successfully with location!")
            else:
                print("Failed to upload story to Instagram.")

        if upload_to_feed or upload_to_story:
            os.remove(image_path)
            print(f"Deleted image: {image_path}")

        return True
    except Exception as e:
        print(f"Upload failed: {e}")
        if "Please wait a few minutes before you try again" in str(e):
            print("Encountered rate limiting. Waiting for a minute before retrying...")
            time.sleep(60)  # Wait for a minute before retrying
        return False

will result in
Upload failed: login_required
<IP_ADDRESS> - - [24/Jul/2024 20:37:45] "POST /post HTTP/1.1" 500 -

Traceback
Show your full traceback so that it is clear where exactly the error occurred.
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. Ubuntu 21.04]
Python version [e.g. 3.8.3]
instagrapi version [e.g. 1.9.3, not "latest"]
moveipy version if used
imagemagick version if used

Additional context
Loggin in via username and passwords works great!

Consider relogin to get a new cookie.
Make sure to keep uuiud same.
This is how I manage authentication:
https://github.com/sujay1599/InstagramTheftyScraperPosterHuman/blob/main/auth.py
Hope that helps.

Hmm, may situation be a bit unique since I am using the Instagram oauth flow to login. Not manually hardcoding the values

Hmm, my situation be a bit unique since I am using the Instagram oauth flow to login. Not manually hardcoding the values…

So not session?
Can u share ur method?

Hmm, my situation be a bit unique since I am using the Instagram oauth flow to login. Not manually hardcoding the values…

Mine creates a session.json too.
And if same account logs it uses that session.

Hey there, take the following. I think the issue also may be that I am not getting the sessionId properly. You will also need to create an app here and use your creds for the placeholders in the code - https://developers.facebook.com/apps/?show_reminder=true. Attached are the steps to get the instagram creds for the oauth flow to work -

Py code -
from flask import Flask, redirect, request, session, url_for, render_template_string
import requests
import os

app = Flask(__name__)
app.secret_key = os.urandom(24)

# Instagram OAuth settings
CLIENT_ID = ''
CLIENT_SECRET = ''
REDIRECT_URI = 'demo.com/callback'

# Step 1: Redirect to Instagram for authorization
@app.route('/login')
def login():
    instagram_auth_url = (
        'https://api.instagram.com/oauth/authorize'
        f'?client_id={CLIENT_ID}'
        f'&redirect_uri={REDIRECT_URI}'
        '&scope=user_profile,user_media'
        '&response_type=code'
    )
    return redirect(instagram_auth_url)

# Step 2: Handle callback from Instagram
@app.route('/callback')
def callback():
    code = request.args.get('code')
    if not code:
        return 'Authorization failed.'

    token_url = 'https://api.instagram.com/oauth/access_token'
    payload = {
        'client_id': CLIENT_ID,
        'client_secret': CLIENT_SECRET,
        'grant_type': 'authorization_code',
        'redirect_uri': REDIRECT_URI,
        'code': code,
    }

    response = requests.post(token_url, data=payload)
    if response.status_code != 200:
        return 'Failed to get access token.'

    access_token = response.json().get('access_token')
    print("Short-Lived Access Token:", access_token)  # Debugging

    # Exchange short-lived token for long-lived token
    long_lived_token_url = 'https://graph.instagram.com/access_token'
    long_lived_payload = {
        'grant_type': 'ig_exchange_token',
        'client_secret': CLIENT_SECRET,
        'access_token': access_token,
    }

    long_lived_response = requests.get(long_lived_token_url, params=long_lived_payload)
    if long_lived_response.status_code != 200:
        return f"Failed to exchange for long-lived token: {long_lived_response.json()}"

    long_lived_access_token = long_lived_response.json().get('access_token')
    print("Long-Lived Access Token:", long_lived_access_token)  # Debugging
    session['access_token'] = long_lived_access_token

    # JavaScript to close the pop-up and redirect the main window
    close_popup_script = '''

    '''
    return close_popup_script

@app.route('/upload', methods=['GET'])
def upload():
    access_token = session.get('access_token')
    print("Access Token for Upload:", access_token)  # Debugging
    if not access_token:
        return redirect(url_for('login'))

# Serve the main page with JavaScript to open the login in a pop-up window
@app.route('/')
def index():
    index_html = '''
    <!DOCTYPE html>

        <title>Instagram OAuth</title>

Welcome to Instagram OAuth

        <button onclick="openPopup()">Login with Instagram</button>

    '''
    return render_template_string(index_html)

if __name__ == '__main__':
    port = int(os.environ.get('PORT', 5000))
    app.run(debug=True, host='<IP_ADDRESS>', port=port)


Command line stake delegation tool works on Testnet.

As part of ensuring full-stack operation and also giving wallets a tool to interactively experiment / test stake delegation mechanics, we will publish a command line tool that can delegate/undelegate stake and claim rewards.
Because this requires portions of wallet functionality, one plausible possibility is that it's a fork of a pre-existing production-quality command line wallet tool.

This tool should also work on Mainnet after activation, but we should caution end-users away from it. It is a testing, validation, development tool.


FR: Add ability to enable flat props of a single block

Idea
Add the ability to enable flat props on a single block, for optional use.

single block?
any additional info? screen? md file example?

done!
https://discord.com/channels/725182569297215569/896368413243494430/1078030318340157532


Python file extraction in the repo

Hi San,
Thank you for your blog regarding file creation using.
I have a similar requirement.
For starters, I am having sample .py code, which creates a file.
But unable to do so... It runs successfully, but the file is not generating. Kindly help.
In your py code, I can see that you have directly used the "data/" (relative path). How did you do it? I don't see any code in your py code that it creates the file in the current path.
Even I have placed the python code in the same repo path and the file which I want to create also is present in the same path.
Please have a look at my below code.
Python code:
import os
print("Start of File")
cwd = os.getcwd()  # Get the current working directory (cwd)
files = os.listdir(cwd)  # Get all the files in that directory
print("Files in %r: %s" % (cwd, files))
f = open("/home/runner/work/country_profile/country_profile/QWC.txt", "w")
f.write("Now the file has more content!")
f.close()
print("End of File")
.yml code: ( I am not using any secrets).
name: Python Test run file
on:
Triggers the workflow on push or pull request events but only for the "main" branch
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
  - uses: actions/checkout@v3

  - name: Run python script test file
    run: python python_test.py

  - name: commit files
    run: |
      git config --local user.email<EMAIL_ADDRESS>      git config --local user.name "GitHub Action"
      git add -A
      git commit -m "update data" -a


comment on manual validation of webhook subscription creation event when using ngrok

[docs augmentation - solution provided]
When you create a storage event subscription as a webhook to connect storage events to your ngrok endpoint, the current version expects validation (see here).
I created the event subscription via the cli, and so the information as to how to validate manually would be helpful to have in the docs:
ngrokendpoint=https://<guid_from_ngrok_output>.ngrok.io
az eventgrid event-subscription create  --name testdocsub \
                                         --source-resource-id $sourceid \
                                         --included-event-types Microsoft.Storage.BlobCreated \
                                         --subject-begins-with /blobServices/default/containers/mycontainer/path/ \
                                         --subject-ends-with .json \
                                         --endpoint-type webhook \
                                         --endpoint  $ngrokendpoint \
                                         --labels mylabel

This command will hang with a Running... line in the terminal, as it is waiting for validation, and then time out with a provisioning status of Failed.
To fix this, within 5 minutes, navigate to localhost:4040 in the web browser on the local machine to be able to inspect the messages received by ngrok.
The message received by POST request on execution of the above command will be shown, and conforms to the validation format described in the docs here.
Navigating to the validationUrl in the browser then validates the subscription creation, the browser will return '"Webhook succesfully validated as a subscription endpoint."' and the command above will complete the provisioning.
It would be useful to have this in the docs, because to new users of ngrok it may not be obvious where to find the validation message content.

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ae03e2fc-39bc-4401-82b1-34ae745e936b
Version Independent ID: 617f4f97-dd3a-4af7-ced1-6e58920b36f9
Content: Azure Functions Event Grid local debugging
Content Source: articles/azure-functions/functions-debug-event-grid-trigger-local.md
Service: azure-functions
GitHub Login: @craigshoemaker
Microsoft Alias: cshoe

@lindacmsheard Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.


DRAFT: Update themes to use new Automatic History Management

Description
This follows up on my history (#1940) branch to adopt the two new functions in all themes which had already included history management.
Closes #1595.
Motivation and Context
This PR adopts the new functions from #1940 and removes all use of the old function _save-and-reload-history(), but does not remove the function yet (in case anyone is using it somewhere out of tree...for the time being).
A future PR should remove all history-related activities from all the themes. I didn't do that here as I want to be as minimally-invasive as possible in this PR. At that time, it will need to be determined how to account for users who have not enabled plugin/history but who use a theme which triggers history -a...their history would after that future PR suddenly stop saving the way they're used to.
How Has This Been Tested?
All tests pass.
Types of changes

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My code follows the code style of this project.
[ ] If my change requires a change to the documentation, I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] If I have added a new file, I also added it to clean_files.txt and formatted it using lint_clean_files.sh.
[ ] I have added tests to cover my changes, and all the new and existing tests pass.

@davidpfarrell, this is "stacked" on my automatic history management PR, which has been approved, but I'd love to start getting feedback on this one pending that one getting merged.
Do you think I should just remove all history management from themes rather than just adopting the new functions? I think that might require an update to the documentation too.

@cornfeedhobo, would you suggest just yanking all history-related tomfoolery from themes or to update them to interact with the new automatic history management? Long term, I vote for themes-don't-touch-history, but I'm conflicted about short-term.

I'm all for a hook mechanism, of course, where themes don't touch any other function like history.
Could you just merge the conflicts? I was not able to figure out how to unbreak it correctly.


I tried pxctest on Xcode 9 beta and I get  "Abort trap: 6" on every command

I tried pxctest on Xcode 9 beta and I get  "Abort trap: 6" on every command.
cbennett-70ap:iOSxm8 i58288$ sudo xcode-select -s ../../.././../../../../Applications/Xcode-beta.app/Contents/Developer/
Password:
cbennett-70ap:iOSxm8 i58288$ pxctest     run-tests     --destination 'name=iPad Air 2,os=iOS 10.3'     --testrun build/Products/iOSxm8_iphonesimulator11.0-x86_64.xctestrun
Abort trap: 6
cbennett-70ap:iOSxm8 i58288$ pxctest     run-tests     --destination 'name=iPad Air 2,os=iOS 11.0'     --testrun build/Products/iOSxm8_iphonesimulator11.0-x86_64.xctestrun
Abort trap: 6
cbennett-70ap:iOSxm8 i58288$ pxctest run-tests --help
Abort trap: 6
cbennett-70ap:iOSxm8 i58288$

Any support for Xcode beta?

It looks like FBSimulatorControl has added support for Xcode 9 just recently. I tried to merge these updates into the FBSimulatorControl version that pxctest is currently using and there are many conflicts. Is there any plan to update FBSimulatorControl so that pxctest can support Xcode 9 also?
Thanks!

I won't have time to update pxctest to support Xcode 9. However pull requests are welcome!


Fix generated class.spec testing import

Class specs generated by ng via the ng generate class command include
an incorrect import - it should import from @angular/core/testing, but
it currently imports from @angular/testing. This commit corrects the
import location.

This was fixed by https://github.com/angular/angular-cli/commit/23c7181770dc81e46332b0d3a35ef9a8f0b4e085. Now in beta.1.


Map tag_list to StoryBlokStory

There is no way to retreive the tag_list form the Client. Can we map the tag_list that is returned from the API to StoryBlokStory?

Yes, I can do that.

https://github.com/adliance/Storyblok/pull/11


fix: converted jest config to typescript

:tada: This PR is included in version 0.3.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add cache for CI

Codecov Report

Merging #168 (3abdac3) into main (7f0f484) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##              main      #168   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            2         2
  Lines          184       184
=========================================
  Hits           184       184

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f0f484...3abdac3. Read the comment docs.


Connection Error

Sorry if it's no a v4l2rtspserver error
I have installed your software on a rpi whithout internet
systemctl status v4l2rtspserver.service
● v4l2rtspserver.service - V4L2 RTSP server
   Loaded: loaded (/lib/systemd/system/v4l2rtspserver.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-06-10 16:48:07 CEST; 20h ago
 Main PID: 338 (v4l2rtspserver)
    Tasks: 2 (limit: 725)
   CGroup: /system.slice/v4l2rtspserver.service
           └─338 /usr/local/bin/v4l2rtspserver -H 1920 -W 1080 -F 30 -P 8888 /dev/video0

juin 10 16:48:08 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:08,513 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:08 raspberrypi v4l2rtspserver[338]:         Device /dev/video0 nb buffer:10
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,115 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         Create Source .../dev/video0
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,123 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         begin thread
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,130 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         format:video/H264
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,131 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         Play this stream using the URL "rtsp://192.168.1.XXX/unicast

When I try the last command in VlC like :
cvlc rtsp://<IP_ADDRESS>

I got :
VLC media player <IP_ADDRESS> Vetinari (revision <IP_ADDRESS>-0-g52483f3ca2)
[0000558d4c855f50] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[0000558d4c870730] dummy interface: using the dummy interface module...
Created new TCP socket 7 for connection
Created new TCP socket 6 for connection
[00007f4190001120] live555 demux error: Failed to connect with rtsp://<IP_ADDRESS>:554
[00007f4190004950] main stream error: connection failed: Connection refused
[00007f4190004950] satip stream error: Failed to connect to RTSP server <IP_ADDRESS>:554
[00007f4190004950] main stream error: connection failed: Connection refused
[00007f4190004950] access_realrtsp stream error: cannot connect to <IP_ADDRESS>:554
[00007f4190004950] main stream error: Connection failed
[00007f4190004950] main stream error: VLC could not connect to "<IP_ADDRESS>:554".
[00007f4198000c80] main input error: Your input can't be opened
[00007f4198000c80] main input error: VLC is unable to open the MRL 'rtsp://<IP_ADDRESS>'. Check the log for details.

Where do I have to look  to resolve this bug ?
Many thanks for this greatfull app.

Looks like your server runs on port 8888 - you need to specify port for
cvlc
(VLC uses 554 as default)
fre. 11. jun. 2021 kl. 13:28 skrev Jean-Emmanuel @.***>:

Sorry if it's no a v4l2rtspserver error
I have installed your software on a rpi whithout internet
systemctl status v4l2rtspserver.service
● v4l2rtspserver.service - V4L2 RTSP server
Loaded: loaded (/lib/systemd/system/v4l2rtspserver.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2021-06-10 16:48:07 CEST; 20h ago
Main PID: 338 (v4l2rtspserver)
Tasks: 2 (limit: 725)

CGroup: /system.slice/v4l2rtspserver.service
       └─338 /usr/local/bin/v4l2rtspserver -H 1920 -W 1080 -F 30 -P 8888 /dev/video0

juin 10 16:48:08 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:08,513 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:08 raspberrypi v4l2rtspserver[338]:         Device /dev/video0 nb buffer:10
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,115 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         Create Source .../dev/video0
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,123 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         begin thread
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,130 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         format:video/H264
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]: 2021-06-10 16:48:09,131 [NOTICE] - /home/pi/v4l2rtspse
juin 10 16:48:09 raspberrypi v4l2rtspserver[338]:         Play this stream using the URL "rtsp://192.168.1.XXX/unicast```
When I try the last command in VlC like :
cvlc rtsp://<IP_ADDRESS>
I got :
VLC media player <IP_ADDRESS> Vetinari (revision <IP_ADDRESS>-0-g52483f3ca2)
[0000558d4c855f50] vlcpulse audio output error: PulseAudio server
connection failure: Connection refused
[0000558d4c870730] dummy interface: using the dummy interface module...
Created new TCP socket 7 for connection
Created new TCP socket 6 for connection
[00007f4190001120] live555 demux error: Failed to connect with rtsp://
<IP_ADDRESS>:554
[00007f4190004950] main stream error: connection failed: Connection refused
[00007f4190004950] satip stream error: Failed to connect to RTSP server
<IP_ADDRESS>:554
[00007f4190004950] main stream error: connection failed: Connection refused
[00007f4190004950] access_realrtsp stream error: cannot connect to
<IP_ADDRESS>:554
[00007f4190004950] main stream error: Connection failed
[00007f4190004950] main stream error: VLC could not connect to "
<IP_ADDRESS>:554".
[00007f4198000c80] main input error: Your input can't be opened
[00007f4198000c80] main input error: VLC is unable to open the MRL 'rtsp://
<IP_ADDRESS>'. Check the log for details.
Where do I have to look  to resolve this bug ?
Many thanks for this greatfull app.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mpromonet/v4l2rtspserver/issues/251, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC6EOPWJU7HSVYNHWT4A5CLTSHXPNANCNFSM46QRDT3Q
.

--
@.*** * +47 4160 1702

Your right but when I have corrected this mistake I got :
 cvlc rtsp://<IP_ADDRESS>:8888/unicast
VLC media player <IP_ADDRESS> Vetinari (revision <IP_ADDRESS>-0-g52483f3ca2)
[000055e0b2bb5f50] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[000055e0b2bea8f0] dummy interface: using the dummy interface module...
Created new TCP socket 6 for connection
Created new TCP socket 6 for connection
[00007f1450000c00] live555 demux error: no data received in 10s, aborting

I will at least use the flux in motion eyes wich say : "unable to open video device"...

AFICR, there is no audio for this setup (Streaming a Raspberry Pi camera)
It would be better if the server created a “dummy” mp3 audio track, since
one could then forward to for instance YouTube, which REQUIRES an audio
track in the stream.
fre. 11. jun. 2021 kl. 14:05 skrev Jean-Emmanuel @.***>:

Your right but when I have corrected this mistake I got :
cvlc rtsp://<IP_ADDRESS>:8888/unicast
VLC media player <IP_ADDRESS> Vetinari (revision <IP_ADDRESS>-0-g52483f3ca2)
[000055e0b2bb5f50] vlcpulse audio output error: PulseAudio server connection failure: Connection refused
[000055e0b2bea8f0] dummy interface: using the dummy interface module...
Created new TCP socket 6 for connection
Created new TCP socket 6 for connection
[00007f1450000c00] live555 demux error: no data received in 10s, aborting
I will at least use the flux in motion eyes wich say : "unable to open
video device"...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mpromonet/v4l2rtspserver/issues/251#issuecomment-859533443,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC6EOPQETCPSFS4KGC6TVZ3TSH3YFANCNFSM46QRDT3Q
.

--
@.*** * +47 4160 1702

The raspberry pi do not have access to Internet. Is it possible to disable sound support ?
The system works well with a rpi with internet access I copy the card on another change the ip adress and the rpi refuse all connexion rstp.

Hi,
Based on the command you used to start (using -P8888), you should be able to reach the stream with cvlc rtsp://<IP_ADDRESS>:8888/unicast.
Best Regards,
Michel.

Hi,
I have try to edit the /usr/lib/systemd/system/v4l2rtspserver.service like this :
[Unit]
Description=V4L2 RTSP server
After=network.target

[Service]
Type=simple
Restart=always
RestartSec=5
ExecStart=/usr/local/bin/v4l2rtspserver -H 1200 -W 1600 -F 60 -P 8888 /dev/video0,
WorkingDirectory=/usr/local/share/v4l2rtspserver

[Install]
WantedBy=multi-user.target

Without coma all works at a first time like this :
systemctl status v4l2rtspserver.service
Warning: The unit file, source configuration file or drop-ins of v4l2rtspserver.service changed on disk. Run 'systemctl daemon-reload' to reload units.
● v4l2rtspserver.service - V4L2 RTSP server
   Loaded: loaded (/lib/systemd/system/v4l2rtspserver.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2021-05-22 10:34:29 CEST; 3h 48min ago
 Main PID: 410 (v4l2rtspserver)
    Tasks: 2 (limit: 1938)
   CGroup: /system.slice/v4l2rtspserver.service
           └─410 /usr/local/bin/v4l2rtspserver -H 1200 -W 1600 -F 60 -P 8888 /dev/video0

mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]: 2021-05-22 10:34:30,038 [NOTICE] - /home/pi/v4l2rtspserver/v4l2wrapper/src/V4l2MmapDevice.cpp:73
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]:         Device /dev/video0 nb buffer:10
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]: 2021-05-22 10:34:30,650 [NOTICE] - /home/pi/v4l2rtspserver/src/V4l2RTSPServer.cpp:61
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]:         Create Source .../dev/video0
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]: 2021-05-22 10:34:30,652 [NOTICE] - /home/pi/v4l2rtspserver/inc/ServerMediaSubsession.h:47
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]:         format:video/H264
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]: 2021-05-22 10:34:30,652 [NOTICE] - /home/pi/v4l2rtspserver/src/DeviceSource.cpp:93
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]:         begin thread
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]: 2021-05-22 10:34:30,652 [NOTICE] - /home/pi/v4l2rtspserver/inc/V4l2RTSPServer.h:81
mai 22 10:34:30 pbsup136-76 v4l2rtspserver[410]:         Play this stream using the URL "rtsp://<IP_ADDRESS>:8888/unicast"

After reload and restart I get this message :
systemctl status v4l2rtspserver.service
● v4l2rtspserver.service - V4L2 RTSP server
   Loaded: loaded (/lib/systemd/system/v4l2rtspserver.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) since Sat 2021-05-22 14:24:59 CEST; 3s ago
  Process: 749 ExecStart=/usr/local/bin/v4l2rtspserver -H 1200 -W 1600 -F 60 -P 8888 /dev/video0, (code=exited, status=0/SUCCESS)
 Main PID: 749 (code=exited, status=0/SUCCESS)

And no more camera. Have I made something wrong ?


Render empty value if render returns false

React supports returning null or false from a render function to indicate nothing to render.
Currently returning false during a server render will render the string ‘false’.
As we use inline conditional rendering quite heavily e.g. return condition &&
yo
this is causing our server renders with preact-compat to surface a fair amount of these unwanted strings.

I guess this could be seen as a breaking change for people using preact directly? Not sure how likely that would be though.

Tech ically breaking yeah, no worries though. I think react also renders nothing for the value true - are you able to verify and add that if so?

I had a try but React will throw in dev mode if you return true.
render(): A valid React element (or null) must be returned. You may have returned undefined, an array or some other invalid object

Perfect, this looks good to me then.


refactor feature review with status decorator

The decorator does no longer delegate to queries. Instead it's the FeatureReviewQuery which instantiates the decorator with all necessary objects.

@dideler please review the general approach of this refactor.

I'm still in the process of reviewing/cleaning -- I'll try to finish up tomorrow.

LGTM


set tab title Typeface from BOLD to NORMAL

set tab title Typeface from BOLD to NORMAL, default.

Hi, @linsea
Sorry, This change can not be accepted.
Please use the setCustomTabView or stl_customTabTextLayoutId attribute
However, I consider to be able to specify the Typeface in the near future.


Apply renovate postUpgradeTasks to bump balena.yml

Signed-off-by: Kyle Harding<EMAIL_ADDRESS>

Ok I think this makes sense to me - the re= line takes care of padding the 0 to make it a proper semver?

Yeah, that's the idea anyway. Unfortunately I missed the small print in the renovate docs.

Post-upgrade tasks can only be used on self-hosted Renovate instances.

So back to the drawing board I guess. I'm going to iterate on this until we get something that's actually helpful, so don't mind the noise while I try to figure things out.


Much faster to zed.grab() on demand rather than continually in a loop thread.

Preliminary Checks

[x] This issue is not a duplicate. Before opening a new issue, please search existing issues.
[X] This issue is not a question, feature request, or anything other than a bug report directly related to this project.

Description
Running the zed.grab() method in a separate thread to a callback to fetch images is considerably slower and less consistent than running zed.grab() immediately at the start of every callback.
Steps to Reproduce
Using the Python API, the two options are described below:

Have a separate thread running self.zed.grab() inside a class. Callbacks which 'get' an image can then call self.zed.retrieve_image(...) as soon as a request is received (e.g. it could be a web request).
Have self.zed.grab() at the start of every callback, before then calling self.zed.retrieve_image(...) to fetch the requested image.

Expected Result
Option 1 uses more CPU as the program is always doing something, even when there are no requests for images. But requests are faster because when a callback is triggered, the most recent image can be retrieved instead of having to grab a brand new one.
Actual Result
Option 1 does use more CPU, however, unexpectedly, each image request takes considerably longer, and is much less consistent.
Example from first method:
Request 1:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.066    0.066    0.066    0.066 {method 'retrieve_image' of 'pyzed.sl.Camera' objects}
        1    0.002    0.002    0.002    0.002 {method 'get_data' of 'pyzed.sl.Mat' objects}

Request 2:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.067    0.067    0.067    0.067 {method 'get_data' of 'pyzed.sl.Mat' objects}
        1    0.064    0.064    0.065    0.065 {method 'retrieve_image' of 'pyzed.sl.Camera' objects}

Request 3:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.198    0.198    0.198    0.198 {method 'retrieve_image' of 'pyzed.sl.Camera' objects}
        1    0.065    0.065    0.066    0.066 {method 'get_data' of 'pyzed.sl.Mat' objects}

Example from second method:
Request 1:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.035    0.035    0.035    0.035 {method 'retrieve_image' of 'pyzed.sl.Camera' objects}
        1    0.015    0.015    0.015    0.015 {method 'grab' of 'pyzed.sl.Camera' objects}
        1    0.002    0.002    0.002    0.002 {method 'get_data' of 'pyzed.sl.Mat' objects}

Request 2:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.037    0.037    0.037    0.037 {method 'retrieve_image' of 'pyzed.sl.Camera' objects}
        1    0.009    0.009    0.009    0.009 {method 'grab' of 'pyzed.sl.Camera' objects}
        1    0.001    0.001    0.001    0.001 {method 'get_data' of 'pyzed.sl.Mat' objects}

Request 3:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.038    0.038    0.038    0.038 {method 'retrieve_image' of 'pyzed.sl.Camera' objects}
        1    0.006    0.006    0.006    0.006 {method 'grab' of 'pyzed.sl.Camera' objects}
        1    0.002    0.002    0.002    0.002 {method 'get_data' of 'pyzed.sl.Mat' objects}

ZED Camera model
ZED2
Environment
- Jetson Xavier NX
- Ubuntu 18.04.5 (Running in Docker from `nvcr.io/nvidia/l4t-base:r32.6.1`)
- CUDA confirmed working with `deviceQuery` sample inside container
- ZED SDK v3.6.0

Anything else?
No response

Also for reference, option 3 (which seems like a terrible idea) is to have zed.grab() in both a separate thread and at the start of each callback. This option is slower than either of the other options.

Hi,
The grab function does not only retrieve images, it also performs computation.
It is not recommended to do grab and retrieve Image in different threads as the ZED SDK is meant to do them serially.
You can do this in a threads and then share the retrieved data with your own app thread.


Affects
All target frequencies except LoP only and Event.
#Issue
It is possible to multiply targets by hitting Save changes really quickly before the confirmation message appears.
To reproduce

Create and enter target values for an indicator, using any of the affected target frequencies.
Hit Save changes multiple times at high speed, before the confirmation message appears.

Repercussions

Removing extras: You can't delete any of these extra periods. Your only option is to start all over.
Results form: You see all of your duplicated targets in the result form's Measure against target menu.
Results on indicators list page: For Midline/Endline and Event targets, you can enter results for all of your duplicated periods and see them on the Indicators list page. In the case of Annual/Semi-annual/etc. targets, it looks like the system just picks one of the duplicated periods for you and assigns all results to that period.
IPTT: Results are aggregated in the IPTT. So for example, if you have two copies of Midline/Endline and add results to all four targets, the system will only display each period once, summing the results across duplicates.

Slack observation from Sanjuro: Maybe it’s a race condition.    Maybe disabling the save button until the message comes back would fix it?

The first time I click "Save changes", the button disables, so that I am unable to click it multiple times no matter how quickly I click.


How to load .pth into the model

If I use bash tools/dist_train.sh configs/classification/tiny_imagenet/automix/basic/r18_l2_a2_near_mb_mlr1e_3_bb_mlr5e_2.py 1 to get the trained weight epoch_400.pth, how to load epoch_400.pth into the resnet18 model for the other task?
# https://github.com/Westlake-AI/openmixup/blob/main/configs/classification/tiny_imagenet/automix/basic/r18_l2_a2_near_mb_mlr1e_3_bb_mlr5e_2.py
model_r18_automix = dict(
    type='AutoMixup',
    pretrained=None,
    alpha=2.0,
    momentum=0.999,  # 0.999 to 0.999999
    mask_layer=2,
    mask_loss=0.1,  # using mask loss
    mask_adjust=0,  # prob of adjusting bb mask in terms of lam by mixup, 0.25 for CIFAR
    lam_margin=0.08,  # degenerate to mixup when lam or 1-lam <= 0.08
    mask_up_override=None,  # If not none, override upsampling when train MixBlock
    debug=True,  # show attention and content map
    backbone=dict(
        type='ResNet_CIFAR',  # CIFAR version
        depth=18,
        num_stages=4,
        out_indices=(2,3),  # stage-3 for MixBlock, x-1: stage-x
        style='pytorch'),
    mix_block = dict(  # AutoMix
        type='PixelMixBlock',
        in_channels=256, reduction=2, use_scale=True,
        unsampling_mode=['nearest',],  # str or list
        lam_concat=False, lam_concat_v=False,  # AutoMix: no lam cat for small-scale datasets
        lam_mul=False, lam_residual=False, lam_mul_k=-1,  # SAMix lam: none
        value_neck_cfg=None,  # SAMix: non-linear value
        x_qk_concat=False, x_v_concat=False,  # SAMix x concat: none
        # att_norm_cfg=dict(type='BN'),  # norm after q,k (design for fp16, also conduct better performace in fp32)
        mask_loss_mode="L1", mask_loss_margin=0.1,  # L1 loss, 0.1
        frozen=False),
    head_one=dict(
        type='ClsHead',  # default CE
        loss=dict(type='CrossEntropyLoss', use_soft=False, use_sigmoid=False, loss_weight=1.0),
        with_avg_pool=True, multi_label=False, in_channels=512, num_classes=200),
    head_mix=dict(  # backbone & mixblock
        type='ClsMixupHead',  # mixup, default CE
        loss=dict(type='CrossEntropyLoss', use_soft=False, use_sigmoid=False, loss_weight=1.0),
        with_avg_pool=True, multi_label=False, in_channels=512, num_classes=200),
    head_weights=dict(
        head_mix_q=1, head_one_q=1, head_mix_k=1, head_one_k=1),
)

if __name__ == '__main__':
    # build the model and load checkpoint
    from openmixup.models import build_model
    from mmcv.runner import load_checkpoint
    cfg_model = model_r18_automix
    checkpoint = './openmixup/work_dirs/classification/tiny_imagenet/automix/basic/r18_l2_a2_near_mb_mlr1e_3_bb_mlr5e_2/epoch_400.pth'
    model = build_model(cfg_model)
    load_checkpoint(model, checkpoint, map_location='cpu')

    print(model)

Is this code correct? Do you have a more concise way to load the model?

There is a concise way to load the model
  from mmcv import Config
  from mmcv.runner import load_checkpoint
  from openmixup.models import build_model

  cfg = Config.fromfile('configs/classification/tiny_imagenet/automix/basic/r18_l2_a2_near_mb_mlr1e_3_bb_mlr5e_2.py')
  model = build_model(cfg.model)
  load_checkpoint(model, 'work_dirs/classification/tiny_imagenet/automix/basic/r18_l2_a2_near_mb_mlr1e_3_bb_mlr5e_2/epoch_400.pth', map_location='cpu')

There is a concise way to load the model
  from mmcv import Config
  from mmcv.runner import load_checkpoint
  from openmixup.models import build_model

  cfg = Config.fromfile('configs/classification/tiny_imagenet/automix/basic/r18_l2_a2_near_mb_mlr1e_3_bb_mlr5e_2.py')
  model = build_model(cfg.model)
  load_checkpoint(model, 'work_dirs/classification/tiny_imagenet/automix/basic/r18_l2_a2_near_mb_mlr1e_3_bb_mlr5e_2/epoch_400.pth', map_location='cpu')

Hi @zeyuanyin, thanks for your question and the concise answer. The way you mentioned is also the official way we loading a full checkpoint as in tools/train.py. Feel free to ask me if you have more questions.

Thanks a lot for your confirmation.
I have other questions about the output of the pretrained model.
  print(model)

  model.cuda()
  import torch
  input = torch.randn(4, 3, 64, 64).cuda()
  input.requires_grad = True
  dummy_label = torch.tensor([1, 2, 3, 4]).cuda()

  print('----train')
  model.train()
  output = model(input, gt_label=dummy_label, return_loss=True)
  print(output)

  print('----eval')
  model.eval()
  output = model(input, gt_label=dummy_label, return_loss=True)
  print(output)

Output is
...
  (head_mix_q): ClsMixupHead(
    (criterion): CrossEntropyLoss()
    (fc): Linear(in_features=512, out_features=200, bias=True)
    (post_process): Softmax(dim=1)
  )
  (head_mix_k): ClsMixupHead(
    (criterion): CrossEntropyLoss()
    (fc): Linear(in_features=512, out_features=200, bias=True)
    (post_process): Softmax(dim=1)
  )
  (head_one_q): ClsHead(
    (criterion): CrossEntropyLoss()
    (fc): Linear(in_features=512, out_features=200, bias=True)
    (post_process): Softmax(dim=1)
  )
  (head_one_k): ClsHead(
    (criterion): CrossEntropyLoss()
    (fc): Linear(in_features=512, out_features=200, bias=True)
    (post_process): Softmax(dim=1)
  )
)
----train
{'loss': tensor(16.9432, device='cuda:0', grad_fn=<AddBackward0>), 'acc_mix_q': tensor([0.], device='cuda:0'), 'acc_one_q': tensor([0.], device='cuda:0'), 'acc_mix_k': tensor([0.], device='cuda:0')}
----eval
{'loss': tensor(27.7222, device='cuda:0', grad_fn=<AddBackward0>), 'acc_mix_q': tensor([0.], device='cuda:0'), 'acc_one_q': tensor([0.], device='cuda:0'), 'acc_mix_k': tensor([0.], device='cuda:0')}

I want to know which strategy is used to calculate the loss value because the loss output is different in train and eval modes.
Is there any way I can modify the output of the model to let it output the logits directly?
What is the meaning of the other return values, acc_mix_q, acc_one_q,acc_mix_k?

Thanks for your detailed questions and I answer them as follows:

As for the classification loss, you should use model.train() during training while using model.eval() for testing. The main difference between train() and eval() is the behaviours of BN and Dropout (e.g., refer to https://zhuanlan.zhihu.com/p/357075502 for detailed explanations).

You can choose the mode of the classification model with mode=xxx (refer to base_model.py), e.g., get the logits directly with mode="inference".
output = model(input, gt_label=dummy_label, mode="inference")  # the logit of the first head

Meanwhile, the basic method of AutoMix variants (using AutoMixup class) is different from other Mixup methods (using MixupClassification class). forward_inference returns the logits of the first classification head while AutoMixup should output the logits of four classification heads. We report the better results among them (usually acc_mix_k) for AutoMix experiments using find_automix_val_median. You might rewrite the forward_inference for AutoMixup to get the logits of four heads.

As for a normal classifier using MixupClassification class, the testing results (mode="test") will be "head0_top1": xxx, where head0 denotes the first classification head and topX denotes the top-X accuracy. As for the AutoMix classifier using AutoMixup class, the output will be acc_mix_q_topX, acc_mix_k_topX, acc_one_q_topX, and acc_one_k_topX, which means the top-X accuracy of the certain head (mix denotes the mixup head and one denotes the one-hot head). Please refer to the paper of AutoMix for details.

I hope these will be helpful to you. 😄

Thanks for your very detailed reply.
It seems not clear about my first question statement,  we use CrossEntropy to calculate the loss, but I want to know whether the input will apply mixup in training mode and not in the inference mode and whether the used target is one-hot or mixed in different modes.
Test code is
print('----args inference model')
logits = model(input, gt_label=dummy_label, mode="inference")
assert logits.shape == (4, 200)
criterion = nn.CrossEntropyLoss()
loss = criterion(logits, dummy_label)
print(loss)

print('----train')
model.train()
output = model(input, gt_label=dummy_label)
print(output)

print('----eval')
model.eval()
output = model(input, gt_label=dummy_label)
print(output)

The output is:
----args inference model
tensor(5.2924, device='cuda:0', grad_fn=<NllLossBackward0>)
----train
{'loss': tensor(16.6683, device='cuda:0', grad_fn=<AddBackward0>), 'acc_mix_q': tensor([0.], device='cuda:0'), 'acc_one_q': tensor([0.], device='cuda:0'), 'acc_mix_k': tensor([0.], device='cuda:0')}
----eval
{'loss': tensor(26.5577, device='cuda:0', grad_fn=<AddBackward0>), 'acc_mix_q': tensor([0.], device='cuda:0'), 'acc_one_q': tensor([0.], device='cuda:0'), 'acc_mix_k': tensor([0.], device='cuda:0')}

In my test result, the first loss calculated by logits and CrossEntropyLoss does not match the loss in model.eval model. There exists the difference among the loss calculation strategies.

As for the further question, mixup loss is applied to mode=train in the classification head (see ClsMixupHead class), and testing and inference modes will perform sample mixup. Please refer to the implementation of forward_train in AutoMixup for more details.

Hi, @zeyuanyin. I guess the above answers can be helpful to your questions. If you have further questions about mixup augmentations and AutoMix implementations, maybe we can discuss them through WeChat (finding me by Lupin_1998). I close this issue and you can open a new one for other questions.


Add py37

Using the build number hack from numpy. I don't really like that but I see that the defaults version is always picked up instead of the conda-forge version even with the channel preference feature.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Restarting OS X now that https://travis-ci.org/conda-forge/python-feedstock/builds/403767727 is OK.


I see nothing but the Timer

I cloned maseter, npm install, then npm start. The page at http://localhost:3000/ isnt showing any Notes interface. I see just timer and the little dev tools menu. Am I doing something wrong? Going to try updating Node.

Was my mistake...something silly.


Neosvr beta headless client / server image

To solve issues with this pull request parkervcp/eggs#1173.
Kind of expected that they wouldn't let someone put a docker image they don't control in an egg for security reasons.
Moved from https://github.com/pterodactyl/images/pull/79

@Synkstar Why doe my mono image not work for this?

It crashes with this error when I try to use the core:mono image I also made this image check for updates with steamcmd when it starts
Unhandled Exception:

System.AggregateException: One or more errors occurred. (The type initializer for 'CodeX.TextureDecoder' threw an exception.) ---> System.TypeInitializationException: The type initializer for 'CodeX.TextureDecoder' threw an exception. ---> System.TypeInitializationException: The type initializer for 'FreeImageAPI.Plugins.PluginRepository' threw an exception. ---> System.DllNotFoundException: FreeImage
  at (wrapper managed-to-native) FreeImageAPI.FreeImage.GetFIFCount()
  at FreeImageAPI.Plugins.PluginRepository..cctor () [0x00000] in <4c4d3d8df61a4031ace2678adeb3a87f>:0
   --- End of inner exception stack trace ---
  at CodeX.TextureDecoder..cctor () [0x0001b] in <0d54ce4da2014fb4b232904d9516124a>:0
   --- End of inner exception stack trace ---
  at FrooxEngine.LibraryInitializer.Initialize (FrooxEngine.Engine engine) [0x0006b] in <c25ec4b82b38439cb9a552350333499e>:0
  at FrooxEngine.EngineInitializer.InitializeFrooxEngine (FrooxEngine.Engine engine) [0x00024] in <c25ec4b82b38439cb9a552350333499e>:0
  at FrooxEngine.Engine.Initialize (System.String appPath, System.String dataPath, System.String cachePath, FrooxEngine.ISystemInfo systemInfo, FrooxEngine.IInternalResource resources, System.Boolean verboseInit) [0x0083f] in <c25ec4b82b38439cb9a552350333499e>:0
   --- End of inner exception stack trace ---
  at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in <de0ee687c1ad4ea99fca2db5d4839ca1>:0
  at System.Threading.Tasks.Task.Wait (System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00043] in <de0ee687c1ad4ea99fca2db5d4839ca1>:0
  at System.Threading.Tasks.Task.Wait () [0x00000] in <de0ee687c1ad4ea99fca2db5d4839ca1>:0
  at NeosHeadless.Program.Main (System.String[] args) [0x002fe] in <e029fe23788b440b904a0aa81db42ebd>:0
---> (Inner Exception #0) System.TypeInitializationException: The type initializer for 'CodeX.TextureDecoder' threw an exception. ---> System.TypeInitializationException: The type initializer for 'FreeImageAPI.Plugins.PluginRepository' threw an exception. ---> System.DllNotFoundException: FreeImage
  at (wrapper managed-to-native) FreeImageAPI.FreeImage.GetFIFCount()
  at FreeImageAPI.Plugins.PluginRepository..cctor () [0x00000] in <4c4d3d8df61a4031ace2678adeb3a87f>:0
   --- End of inner exception stack trace ---
  at CodeX.TextureDecoder..cctor () [0x0001b] in <0d54ce4da2014fb4b232904d9516124a>:0
   --- End of inner exception stack trace ---
  at FrooxEngine.LibraryInitializer.Initialize (FrooxEngine.Engine engine) [0x0006b] in <c25ec4b82b38439cb9a552350333499e>:0
  at FrooxEngine.EngineInitializer.InitializeFrooxEngine (FrooxEngine.Engine engine) [0x00024] in <c25ec4b82b38439cb9a552350333499e>:0
  at FrooxEngine.Engine.Initialize (System.String appPath, System.String dataPath, System.String cachePath, FrooxEngine.ISystemInfo systemInfo, FrooxEngine.IInternalResource resources, System.Boolean verboseInit) [0x0083f] in <c25ec4b82b38439cb9a552350333499e>:0 <---

[ERROR] FATAL UNHANDLED EXCEPTION: System.AggregateException: One or more errors occurred. (The type initializer for 'CodeX.TextureDecoder' threw an exception.) ---> System.TypeInitializationException: The type initializer for 'CodeX.TextureDecoder' threw an exception. ---> System.TypeInitializationException: The type initializer for 'FreeImageAPI.Plugins.PluginRepository' threw an exception. ---> System.DllNotFoundException: FreeImage
  at (wrapper managed-to-native) FreeImageAPI.FreeImage.GetFIFCount()
  at FreeImageAPI.Plugins.PluginRepository..cctor () [0x00000] in <4c4d3d8df61a4031ace2678adeb3a87f>:0
   --- End of inner exception stack trace ---
  at CodeX.TextureDecoder..cctor () [0x0001b] in <0d54ce4da2014fb4b232904d9516124a>:0
   --- End of inner exception stack trace ---
  at FrooxEngine.LibraryInitializer.Initialize (FrooxEngine.Engine engine) [0x0006b] in <c25ec4b82b38439cb9a552350333499e>:0
  at FrooxEngine.EngineInitializer.InitializeFrooxEngine (FrooxEngine.Engine engine) [0x00024] in <c25ec4b82b38439cb9a552350333499e>:0
  at FrooxEngine.Engine.Initialize (System.String appPath, System.String dataPath, System.String cachePath, FrooxEngine.ISystemInfo systemInfo, FrooxEngine.IInternalResource resources, System.Boolean verboseInit) [0x0083f] in <c25ec4b82b38439cb9a552350333499e>:0
   --- End of inner exception stack trace ---
  at System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) [0x00011] in <de0ee687c1ad4ea99fca2db5d4839ca1>:0
  at System.Threading.Tasks.Task.Wait (System.Int32 millisecondsTimeout, System.Threading.CancellationToken cancellationToken) [0x00043] in <de0ee687c1ad4ea99fca2db5d4839ca1>:0
  at System.Threading.Tasks.Task.Wait () [0x00000] in <de0ee687c1ad4ea99fca2db5d4839ca1>:0
  at NeosHeadless.Program.Main (System.String[] args) [0x002fe] in <e029fe23788b440b904a0aa81db42ebd>:0
---> (Inner Exception #0) System.TypeInitializationException: The type initializer for 'CodeX.TextureDecoder' threw an exception. ---> System.

@Synkstar have you tried to add fixes to the mono image?

I just tried modifying the mono-6-complete image and for some reason it can't find steamcmd  but it does start the server

@Synkstar have you tried to add fixes to the mono image?

I just got it to work by modifying your mono6 image. I'm guessing you wanted me to do that because it simplifies it.


AppSettings::AllowExternalSubcommands [sometimes] triggers suggestions

Rust Version
rustc 1.23.0 (766bd11c8 2018-01-01)
Affected Version of clap
2.29.4
Expected Behavior Summary
The program executions to print:
# 1
subcommand: z
# 2
subcommand: te

Actual Behavior Summary
# 1
subcommand: z
# 2
error: The subcommand 'te' wasn't recognized
	Did you mean 'test'?

If you believe you received this message in error, try re-running with 'example -- te'

USAGE:
    example [SUBCOMMAND]

For more information try --help

Steps to Reproduce the issue
src/main.rs:
extern crate clap;

use clap::{App, AppSettings, SubCommand};

fn main() {
    let matches = App::new("My Super Program")
        .setting(AppSettings::AllowExternalSubcommands)
        .subcommand(SubCommand::with_name("test").about("controls testing features"))
        .get_matches();

    if let Some(_) = matches.subcommand_matches("test") {
        println!("pre-defined subcommand: test");
    } else {
        let (command, _) = matches.subcommand();
        println!("subcommand: {}", command);
    }

}

Run with
# 1
cargo run -- z
# 2
cargo run -- te

Debug output
$ cargo run -- te
   Compiling clap v2.29.4
   Compiling example v0.1.0 (file:///Users/yrashk/tmp/example)
    Finished dev [unoptimized + debuginfo] target(s) in 8.11 secs
     Running `target/debug/example te`
DEBUG:clap:Parser::add_subcommand: term_w=None, name=test
DEBUG:clap:Parser::propagate_settings: self=My Super Program, g_settings=AppFlags(
    (empty)
)
DEBUG:clap:Parser::propagate_settings: sc=test, settings=AppFlags(
    NEEDS_LONG_HELP | NEEDS_LONG_VERSION | NEEDS_SC_HELP | UTF8_NONE | COLOR_AUTO
), g_settings=AppFlags(
    (empty)
)
DEBUG:clap:Parser::propagate_settings: self=test, g_settings=AppFlags(
    (empty)
)
DEBUG:clap:Parser::get_matches_with;
DEBUG:clap:Parser::create_help_and_version;
DEBUG:clap:Parser::create_help_and_version: Building --help
DEBUG:clap:Parser::create_help_and_version: Building --version
DEBUG:clap:Parser::create_help_and_version: Building help
DEBUG:clap:Parser::get_matches_with: Begin parsing '"te"' ([116, 101])
DEBUG:clap:Parser::is_new_arg:"te":NotFound
DEBUG:clap:Parser::is_new_arg: arg_allows_tac=false
DEBUG:clap:Parser::is_new_arg: probably value
DEBUG:clap:Parser::is_new_arg: starts_new_arg=false
DEBUG:clap:Parser::possible_subcommand: arg="te"
DEBUG:clap:Parser::get_matches_with: possible_sc=false, sc=None
DEBUG:clap:usage::create_usage_with_title;
DEBUG:clap:usage::create_usage_no_title;
DEBUG:clap:usage::get_required_usage_from: reqs=[], extra=None
DEBUG:clap:usage::get_required_usage_from: after init desc_reqs=[]
DEBUG:clap:usage::get_required_usage_from: no more children
DEBUG:clap:usage::get_required_usage_from: final desc_reqs=[]
DEBUG:clap:usage::get_required_usage_from: args_in_groups=[]
DEBUG:clap:usage::needs_flags_tag;
DEBUG:clap:usage::needs_flags_tag:iter: f=hclap_help;
DEBUG:clap:usage::needs_flags_tag:iter: f=vclap_version;
DEBUG:clap:usage::needs_flags_tag: [FLAGS] not required
DEBUG:clap:usage::create_help_usage: usage=example [SUBCOMMAND]
DEBUG:clap:Parser::color;
DEBUG:clap:Parser::color: Color setting...Auto
DEBUG:clap:is_a_tty: stderr=true
DEBUG:clap:Colorizer::error;
DEBUG:clap:Colorizer::warning;
DEBUG:clap:Colorizer::good;
DEBUG:clap:Colorizer::good;
DEBUG:clap:Colorizer::good;
error: The subcommand 'te' wasn't recognized
	Did you mean 'test'?

If you believe you received this message in error, try re-running with 'example -- te'

USAGE:
    example [SUBCOMMAND]

For more information try --help

Currently the fix is to not compile the suggestions feature.
[dependencies.clap]
version = "2"
default-features = false
features = ["color", "wrap_help"] # assuming one would still want the other defaults

This is a hard one to solve generally for the masses because clap has to match incoming arguments to something and if they happen to be close enough to trigger a "possible subcommand" there's no way for clap to know if this is what this particular use case wanted or not.
A future fix would be to add a AppSettings::DontUseSuggestions which would only apply to the current (sub)command. Possibly also narrowed down at a later date with the ability to only not suggest on flags/options or subcommands. I'll mark this as a todo, or good first issue for someone.
I can look at adding it in v3 if no one has knocked it out by then.

Kevin,
Thank you so much for the detailed explanation.
I suspect this might be a good solution for my use case where I'm trying
(in the absence of a matching subcommand) to see if I can execute
myprog-given-subcommand. What I was trying to accomplish was along the
lines of "try to match these subcommands without suggestions first, if it
fails, try finding the external binary; if there is none, enable
suggestions and run matching again". But obviously this doesn't work well
when suggestions are enabled or disabled compile time.
I'll try to see if I can prepare a patch for this.
On Feb 10, 2018 10:54 AM, "Kevin K."<EMAIL_ADDRESS>wrote:
Currently the fix is to not compile the suggestions feature.
[dependencies.clap]version = "2"default-features = falsefeatures =
["color", "wrap_help"] # assuming one would still want the other
defaults
This is a hard one to solve generally for the masses because clap has to
match incoming arguments to something and if they happen to be close
enough to trigger a "possible subcommand" there's no way for clap to know
if this is what this particular use case wanted or not.
A future fix would be to add a AppSettings::DontUseSuggestions which would
only apply to the current (sub)command. Possibly also narrowed down at a
later date with the ability to only not suggest on flags/options or
subcommands. I'll mark this as a todo, or good first issue for someone.
I can look at adding it in v3 if no one has knocked it out by then.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kbknapp/clap-rs/issues/1169#issuecomment-364623723,
or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAABxMeWT1w2HZukDQ_xd1TU2mefKEl0ks5tTRLqgaJpZM4R9zl2
.

Awesome! If you'd like to try this issue, I'd suggest adding a new setting (by following the examples of all other settings) in src/app/settings.rs, then to actually use this setting, you'd probably check if this setting was used in src/app/parser.rs at the start of possible_subcommand and just return (false, None) if that setting was used.
I'd also add some tests to tests/app_settings.rs to ensure it's all working correctly.


Rviz2 for the browser

There are some cool projects that ported parts of rviz to the browser, unfortunately they are heavily tied to the ros binaries and not easy to port for rviz2. Then there is following package: https://github.com/emersonknapp/omni-rviz2 but it's stale. So I would like to make a feature request for rviz2 in the browser. I was wondering if this was discussed before and if somebody has an estimate of how long this would take and some high level milestones.

You could also have a look at the Zethus project which reimplements RViz in JS + ThreeJS (github.com/robostack/zethus)

I'm not aware of any prior discussions about supporting RViz in the browser. It sounds like a large endeavor. If someone would like to make an effort towards browser support, I suggest they start with a design document (e.g. a new article here), describing the planned approach and what would have to change architecturally. Having a design doc reviewed and approved would make subsequent pull requests more likely to be accepted.

This is something that would be useful for me - the device I'm developing on is headless and so I'm running RViz2 on another computer on the network.
This works with ROS Docker containers with host networking, but sometimes bridge networking is preferred/required, and as such a different computer on the network cannot see ROS topics inside the bridge network.
A possible solution for my use case is running a desktop environment with RViz2 inside a Docker container, and using a VNC server to connect remotely. I've tried a few examples I've found online as well as building a container myself, but haven't been able to get anything working.

For what it is worth, there are projects like https://webviz.io/ and https://foxglove.dev/ that do exactly this.
Whether it makes sense to port this particular RViz2 code to the browser, or just use one of those projects, isn't totally clear to me.

I had seen some of these before and they look like they would work perfectly. I've been having lots of trouble getting https://github.com/RobotWebTools/rosbridge_suite/ to work, which both depend on for live viewing. Apparently an official release is happening Soon™ (https://github.com/RobotWebTools/rosbridge_suite/issues/536), so maybe when that's out I will get it working! Thanks.

Foxglove developer here. If you use the desktop version, you can connect to a native ROS 1 robot without the rosbridge WebSocket bridge. Support for native connections to ROS 2 robots from the desktop app should land near the end of this month. The web version is unable to open raw TCP/UDP sockets so it relies on rosbridge for connectivity, but this will be true of any web app.
We have patched rosbridge_suite to work with ROS 2 and are in the process of releasing it for Foxy and Galactic right now. You can follow https://github.com/foxglove/studio/issues/716 for updates on this.

Hi John! After a few hours of fiddling I did manage to get rosbridge_suite working with ROS2. Foxglove seems to be working for the most part, I need to do some more testing before I can confirm it'll work for my use case.
Slightly off topic, but for some reason I was getting formatting errors in the launch file, and in the Python script the ParameterDescriptor didn't like the dynamic_typing argument. The below fixed that issue for me, not sure if this is known.
RUN sed -i "s/'/''/g" src/rosbridge_suite/rosbridge_server/launch/rosbridge_websocket_launch.xml && \
    sed -i "s/ParameterDescriptor(dynamic_typing=True)/ParameterDescriptor()/g" src/rosbridge_suite/rosbridge_server/scripts/rosbridge_websocket.py

What do you mean by native connections? The topic you linked maybe indicates Foxglove Studio will be able to ssh into e.g. a container and attach to topics as needed? That would be very nice.

The ParameterDescriptor issue is known and seems to be a difference between Foxy and Galactic (assuming you’re using Foxy?) — I’ll try to get this fixed before making a Foxy release. Might ask for your help in testing it :)

Ah sure, yep using Foxy. Sure, I can do my best to help :)
Gotcha. I think again that would run into the limitation that Studio would need to be on the same network - which isn't the case if using Docker bridge for containers (but can connect to services on specific ports, e.g. start an ssh server).

Given that there are other solutions for the original request here, I'm going to close this out.


character_controller_height, character_controller_radius should be given together

The doc says:

If an entity has both this and a character_controller_radius, it will be given a physical character collider.

If the guest developer provides only one of them, it should not compile.
Perhaps it should also panic, if [physics_controlled] is absent.
there are some other contexts in which we may want to reject some components. For example, sometimes it doesn't make any send to use a dynamic but the code will compile while nothing will happen.

Perhaps we can use ensure_has_component?

So the "concepts" were kind of aiming at solving this, but I think there's still room for improvement.
I ran into the same problem when doing the new animation system; I've added structs to the guest api which are just wrapping EntityId's, just to make them "strongly typed": https://github.com/AmbientRun/Ambient/blob/main/guest/rust/api_core/src/animation.rs
@philpax let's have a chat about this once you're back.


⚠ SceneTime has degraded performance

In f9fcd65, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1099 ms

Resolved: SceneTime performance has improved in 669bb8c after 6 minutes.


missing boost_unit_test_framework-vc142

Hi, I'm updating from boost 1.69 to 1.70 but I noticed there is no boost_unit_test_framework-vc142 on nuget, while it was there in previous releases? Was this a conscious choice or a missed package?

I'm having the same problem.  Any word on this would be appreciated.

Boost 1.71 has it.


WIP: BOS - Reorganising Docs

Currently our docs are a mixture of different efforts, which do not finish to click into one coherent documentation website.
With the addition of the bot-gateways and bot-components there is a lot of documentation to be created about decentralised frontends. This begs for a massive re-organization of the documentation website, in which we should aim to improve and simplify the navigation of the docs while making them revolve around the theme "NEAR is the BOS".
While I personally do not agree with the "BOS slogan", I think we should make our best effort for the developers that come to our ecosystem and help them find their way through.

👍 LGTM
@gagdiez just merge this PR when you're ready 💪


pybot is not recognized as an internal or external command

I have installed on my machine
Python 2.7.17
pip 19.2.3
I set the environment variables
C:\Python27
C:\Python27\Scripts
When I run pybot --version I get: "'pybot' is not recognized as an internal or external command,
operable program or batch file."
Any help is really appreciated.

pybot has been removed already in 3.1. See #2818 and #2216.


[All Hosts] (dialog) document error 12011

Fixes 1959

Docs Build status updates of commit d278f86:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/develop/dialog-handle-errors-events.md
:white_check_mark:Succeeded
View

docs/reference/javascript-api-for-office-error-codes.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


fireEvent.drop is not including event.dataTransfer

react-testing-library version: 6.0.1
react version: 16.8.1
node version: 8.9.4
npm version: 5.6.0
jest version: 24.0.0

Relevant code or config:
Below is a simplified, generic version of my code
export const FileDropZone = props => {

    const handleDrop = useCallback(
        event => {
            event.stopPropagation();
            event.preventDefault();
            handleFiles(event.dataTransfer.files);
        },
        [handleFiles]
    );

    // Rest of code

    return (
        <div
            onDragEnter={handleDragEnter}
            onDragLeave={handleDragLeave}
            onDrop={handleDrop}
        >
            Drag File Here
        </div>
    );
}

describe('FileDropZone', () => {
    test('should handle a file being dropped in the drop zone', () => {
        const file = new File(['(⌐□_□)'], 'chucknorris.png', {
            type: 'image/png',
        })
        const {getByText, queryByText} = render(<FileDropZone {...props} />);
        expect(queryByText('chucknorris.png')).toBeNull();
        const dropZone = getByText('Drag File Here');
        act(() => {
            fireEvent.drop(dropZone, {dataTransfer: {files: [file]}});
        });
        expect(queryByText('chucknorris.png')).toBeInTheDocument();
    })
})

What you did:
I am trying to test a drop event that is used for uploading files via drag-and-drop
What happened:
When I run my unit test, the event is firing, but I get the following error
TypeError: Cannot read property 'files' of undefined
When I do a console.log of the event it shows that event.dataTransfer is undefined.
Reproduction:
Unable to provide a link to code repository due to the repo being private
Problem description:
Unable to test file upload via drag-and-drop using react-testing-library

Hi @christopher-schroer,
I'm not sure what this could be. Without a simple reproduction in a codesandbox there's only so much we can do.

@kentcdodds I figured as much. Once I have some free time, I will try to create a repo for sharing. Unfortunately, I'm going to be on vacation until Apr 11.

I managed to put together a basic example
https://github.com/christopher-schroer/react-testing-library-drop-issue
Clone it, run npm install, and then run npm test

Ah, this appears to be an issue with jsdom not supporting dataTransfer. Based on these tests.
You could file an issue against jsdom to see what it would take to contribute support for dataTransfer.

Closing this in favor of: https://github.com/jsdom/jsdom/issues/1568
Thanks.

For anyone that finds their way to this ticket via a search whilst they are stuck on the same problem, I managed to work around the limitation in jsdom using Object.defineProperty like so:
const file = new File([""], "guybrushthreepwood.png", {
  type: "image/png",
}

const mockDropEvent = new window.Event("drop")
Object.defineProperty(event, "dataTransfer", {
  value: {
    files: [file]
  }
})

const target = getByText("Drag File Here")

fireEvent(target, mockDropEvent)

I'm not sure how evil this approach is, but it might come in handy :)

Seems legit. Would be cool to get this added to the examples: https://codesandbox.io/s/github/kentcdodds/react-testing-library-examples

@andypearson I wasn't able to get that exact code to work, but I got a very similar version working:
const fileDropzone = getByText('Drag and Drop Excel Files Here');
const fileDropEvent = createEvent.drop(fileDropzone);

Object.defineProperty(fileDropEvent, 'dataTransfer', {
  value: {
    files: {
      item: itemIndex => [file][itemIndex],
    },
  },
});

fireEvent(fileDropzone, fileDropEvent);

Key differences here:
I'm using the RTL createEvent to create the event. window.Event wasn't working for me.
Rather than return files: [file] I created the item property which mocks the FileItem object which has the item property that looks roughly like the function I defined for it. Using an actual FileItem object is a major headache, but if someone else wanted to look into it more here is a pretty in depth post that goes into it: https://github.com/jsdom/jsdom/issues/1272#issuecomment-486088445

@kentcdodds I'll see what I can do!

For anyone testing drag and drop and wants to test setData and the above two approach are not working
Using
const mockEv = new window.Event('dragstart') //Object.defineProperty ... fireEvent(draggableNode, mockEv)
doesn't fire event, resulting in handleDragStart() not being called
Second approach with using createEvent fires the desired event, but, Object.defineProperty(event, 'dataTransfer', { setData: jest.fn() }) doesn't actually defines the desired 'dataTransfer' property.
A quick workaround for me was to use Object.assign(event, {dataTransfer: {setData: jest.fn()}})
Full working example as of<EMAIL_ADDRESS>`
test('on drag start sets dataTransfer property correctly', () => {
const setData = jest.fn()
const ev = {
dataTransfer: {
setData
}
}
const {
getByTestId
} = render();
const draggableOpNode = getByTestId('ops-bucket-item');
const mockDropEvent = createEvent.dragStart(draggableOpNode);

Object.assign(mockDropEvent, ev);
fireEvent(draggableOpNode, mockDropEvent)

expect(setData).toHaveBeenCalledTimes(1)
expect(setData).toHaveBeenCalledWith({ /*your correct data*/ })

`


Added Core FirebaseApp swift tests

Adding FirebaseAppTests.swift that aims to test the FirebaseApp api

Adding an objc class, ExceptionCatcher, that you can call in Swift to wrap code that may throw objc originating exceptions. You could then handle the exception as a normal swift error.

Changed the FirebaseCore.podspec file to provide the ExceptionCatcherclass and sample GoogleService-Info.plist for testing

Closing in favor of #5678


Dev

A few things that maybe we should create new issues for:

It looks like the link here still goes to the HOTH 9 gallery page

Nit: For the FAQ section, maybe we should add a "We look forward to hearing from you!" at the end because our answer seems a little abrupt.


Add IsMasterKeyActive RPC

This RPC is between the API server and stellard.

this PR also adds note encryption, is this intended?

I didn't put reviewers yet because of that. Hoping to merge its predecessors and rebase first.


[OPS] Add mongo/redis to helm chart dependencies

Description
Add redis and mongo as individual pods to our helm charts.
Fixes 15929
PFA
Redis:

Mongo:
Demo of the deployment of how it looks.
https://user-images.githubusercontent.com/15846947/187432318-59f8251e-d8d8-4874-a497-130b32732a70.mov
The ending error is because of the JS library which returns false even though the replicaset is configured
Mongo deployment pics:

JS Library issue

@pratapaprasanna, thanks for opening this.
This new chart will need manual upgrading, right? For example, if someone is running the current Helm Chart (v1.5.3), their MongoDB data would be in the Appsmith pod's PV. But with this chart, that data should be in the mongodb pods' PV. Which means moving from that Helm Chart to this one, will need some migration effort, right? Can we document the steps needed for someone already using current Helm Chart to migrate to this one please?
Also, because of this reason, let's make this Helm Chart version v2.0.0 in Chart.yaml. What say?
cc @mohanarpit

Will this mean that we will deprecate v1.x for our Helm charts and urge users to follow the migration steps v2.x? If yes, will any critical fixes be made to both v1.x & v2.x Helm charts? Also, when users run the command helm upgrade today, will Helm pick up the v1.x chart or v2.x chart?

@mohanarpit, I'd rather we take a call on deprecating v1.x, after we know what it takes to upgrade to this Helm Charts. If it's a simple one or two steps, then lets deprecate. We are still hosting charts all the way from 1.3 I think, which is just becoming a baggage.
Yeah, critical fixes will need to be made in both, but, we haven't seen any such critical issues in our Helm Charts so far. 🤞
I don't think helm upgrade upgrades the version of the Helm charts. That command is to apply any changes in our values.yml configuration to our existing chart. @pratapaprasanna, please correct if this is not right.

@pratapaprasanna, I see that you've requested a review, but I don't see a comment with your thoughts on backwards compatibility and a migration path. Can you take a look at https://github.com/appsmithorg/appsmith/pull/16096#pullrequestreview-1094413731 and share your thoughts please?

Hi,
im currently facing an issue with in an kubernetes environment which doesn't have access to the internet.
The appsmith container is always in the INIT:0/2 state because the helm chart is using an alpine image and is installing the redis-cli through the command durring the initialization:
sh       -c       apk add redis; until redis-cli -h appsmith-redis-master.manufacturing-appsmith-dev.svc.cluster.local ping; do echo waiting for redis; sleep 2; done
that is why the init process is logging the following messages all the time:
sh: redis-cli: not found
waiting for redis
looks like the current helm chart is not usable in such an offline environment and should be improved.


Why define shared_ptr and unique_ptr

In the 'Pointer.h', there are 2 functions: shared and unique, which basically create a shared point and a unique pointer. Why not directly invoke constructor of std::shared_ptr and std::unique_ptr? Besides clarity, using these 2 functions can have safety issues. Since it requires a raw pointer, the real object can be accessed elsewhere, and potentially be deleted. For example:
int* p1 = new int( 42 );
auto p2 = shared( p1 );
delete p1;              // p2 is now dangling.

So I am creating this issue to see why it is designed liked this. Thanks.

AFAIR, I did the indirection to isolate and collect all smart pointers in Pointer.h . At the time of writing the code, smart pointer in the std library where just evolving I think.
As of today, shared_ptr and unique_ptr are perfectly established and base parts of C++ and I would not do such indirection. Next time I touch the code base, I will remove Pointer.h and use std::shared_ptr and std::unique_ptr directly.
The consequences you draw are not correct. The functions shared() and unique() are certainly inlined by every compiler and vaporize in the binary output - even without an inline hint. The functions are safe, you misunderstood the mechanics off smart pointers. Every smart pointer is a somewhat intelligent (therefore smart) resource managing utility and stores a raw pointer to the resource among maybe other things (like a reference count in case of share_ptr). If you delete the resource with a plain delete call, then the smart pointer points to invalid memory. Such deletion would violate the usage pattern of smart pointers.


How to open link in editor?

What does onLaunchUrl do? How can I launch URL form the editor?

It's what happens when url link is clicked

By default we use url_launcher package to open webview

By default we use url_launcher package to open webview

There is a problem with this, what if the user types google.com instead of https://www.google.com ? Most users would do that.

The url is not launching.
I have passed the _onLaunchUrl_ function but it is not doing anything.

Why do you even need to pass anything?

Why do you need custom behavior?
By default it already works

It only launch in readonly mode

Why do you need custom behavior?
By default it already works

I removed the function... It is not working even then.

It only launch in readonly mode

Oh now it is working. Thanks.

I have tried using OnLauncher, but I don't know how to implement it correctly, since when clicking on the link, the Url should open and it doesn't

By default we use url_launcher package to open webview

How can we open the link in a browser instead of webview?

I'm stuck at this. Is there way already present to do it?


Taxonomy visualizer image not showing

Docs Update Description
The taxonomy visualization image is not working and hasn't been for a few days.  It's very helpful to see.  https://ethyca.github.io/fideslang/
Describe what you'd like to see documented, and link the page if applicable
Additional context
Add any other context about the problem here.

Looks like this is fixed now

fixed in #123


How to pronounce "Mesop"?

Hi @richard-to, @wwwillchen, etc. – what's the official pronunciation of "Mesop"?
I've heard many different variations on YouTube :)

Yeah, there's a ton of variations and I think it's like gif, there's no one right way to say it :)
That said, I usually pronounce it as "mee" - "sawp"


HDFS-15975. Use LongAdder instead of AtomicLong for branch-3.3

JIRA: HDFS-15975

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
18m 25s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ branch-3.3 Compile Tests _

+0 :ok:
mvndep
13m 32s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 38s

branch-3.3 passed

+1 :green_heart:
compile
17m  9s

branch-3.3 passed

+1 :green_heart:
checkstyle
2m 45s

branch-3.3 passed

+1 :green_heart:
mvnsite
4m 10s

branch-3.3 passed

+1 :green_heart:
javadoc
4m  8s

branch-3.3 passed

+1 :green_heart:
spotbugs
8m  2s

branch-3.3 passed

+1 :green_heart:
shadedclient
18m 41s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
19m  5s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 26s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 49s

the patch passed

+1 :green_heart:
compile
16m 35s

the patch passed

+1 :green_heart:
javac
16m 35s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m 36s
/results-checkstyle-root.txt
root: The patch generated 5 new + 243 unchanged - 5 fixed = 248 total (was 248)

+1 :green_heart:
mvnsite
4m  8s

the patch passed

+1 :green_heart:
javadoc
4m  8s

the patch passed

+1 :green_heart:
spotbugs
8m 40s

the patch passed

+1 :green_heart:
shadedclient
18m 34s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 35s

hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 29s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
185m 34s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m  7s

The patch does not generate ASF License warnings.

370m 55s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR

hadoop.hdfs.server.datanode.TestBPOfferService

hadoop.hdfs.TestReconstructStripedFile

hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy

hadoop.hdfs.TestReconstructStripedFileWithValidator

hadoop.hdfs.TestStripedFileAppend

hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2940/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2940

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 5e9773e8c1f5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.3 / c9772e37b331850419680bc45fa693b3d1c965ce

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2940/1/testReport/

Max. process+thread count
3557 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2940/1/console

versions
git=2.17.1 maven=3.6.0 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Could rebase on branch-3.3 and force-push it to clean commit log?

Could rebase on branch-3.3 and force-push it to clean commit log?

Hi @tasanuma . I'm sorry, I just saw the message today. I have rebase and commit it.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
21m 12s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s

The patch appears to include 1 new or modified test files.

_ branch-3.3 Compile Tests _

+0 :ok:
mvndep
15m  0s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
20m 43s

branch-3.3 passed

+1 :green_heart:
compile
17m 17s

branch-3.3 passed

+1 :green_heart:
checkstyle
2m 40s

branch-3.3 passed

+1 :green_heart:
mvnsite
4m 11s

branch-3.3 passed

+1 :green_heart:
javadoc
4m  9s

branch-3.3 passed

+1 :green_heart:
spotbugs
8m  5s

branch-3.3 passed

+1 :green_heart:
shadedclient
18m 41s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 51s

the patch passed

+1 :green_heart:
compile
16m 37s

the patch passed

+1 :green_heart:
javac
16m 37s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
2m 39s
/results-checkstyle-root.txt
root: The patch generated 5 new + 243 unchanged - 5 fixed = 248 total (was 248)

+1 :green_heart:
mvnsite
4m 10s

the patch passed

+1 :green_heart:
javadoc
4m 11s

the patch passed

+1 :green_heart:
spotbugs
8m 43s

the patch passed

+1 :green_heart:
shadedclient
18m 44s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
17m  8s
/patch-unit-hadoop-common-project_hadoop-common.txt
hadoop-common in the patch passed.

+1 :green_heart:
unit
2m 33s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
187m 22s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
1m  8s

The patch does not generate ASF License warnings.

377m 14s

Reason
Tests

Failed junit tests
hadoop.fs.ftp.TestFTPFileSystem

hadoop.hdfs.TestReconstructStripedFile

hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy

hadoop.hdfs.TestReconstructStripedFileWithValidator

hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks

hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints

hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2940/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/2940

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 0806822b7565 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
branch-3.3 / c4c0469cbef81be6268bbac9a840009619e2d6b0

Default Java
Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2940/2/testReport/

Max. process+thread count
3759 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2940/2/console

versions
git=2.17.1 maven=3.6.0 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

The failed tests are not related. (TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight fails with/without this PR. I will cherry-pick HDFS-15378.)

Merged. Thanks for your work, @tomscut!

The failed tests are not related. (TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight fails with/without this PR. I will cherry-pick HDFS-15378.)

I also founded this problem, but I haven't had time to analyze why. Thanks @tasanuma for you help.


[Engine] Cache & Re-use Compiled Shaders

Materials always compile whatever shader they need, even if we have already compiled it 20 times and using all 20 :(
Need to Cache and Re-use them

Caching was implemented ages ago, i think i forgot when creating this issue.