[WIP] Add textarea popup for pasting code

This is a first attempt at implementing a popup (textarea), in which code can be pasted which in turn is parsed into a diagram. The function fromTeX is not yet polished (in particular w.r.t. code style). Nevertheless, I already upload this to discuss the popup functionality.
Currently, the functionality is very basic: clicking on the new button in the toolbar (which still lacks a unique icon) opens a popup. The style of the popup is taken mainly from the input for arrow labels. The popup can be closed again by clicking in the area not covered by the popup. If code is pasted into the popup, it gets parsed and the popup is closed.

Sounds good! 👍 But I don't think we should close the popup automatically, so the user can check their pasted code and/or make further adjustments.

I have just pushed an update:

The fromTeX function should now adhere to the code style and parse any piece of code generated by tikzcd-editor, with the following exceptions: if a node or arrow label contains special characters like & or ], unexpected things might happen. However, since these can be escaped by simply defining some LaTeX commands as aliases, this is not a big problem I think.
The popup is not closed automatically anymore. However, typing in it triggers hotkeys. Is there a way do disable hotkeys globally and enable them again later?
I have no experience with creating svgs. How did you create the icons?

Regarding 1: I might be looking into that.
Regarding 3: I'm getting my icons from iconmonstr. Maybe you can find a fitting icon?
Regarding 2: Hotkeys for changing arrow properties are handled in Properties.js and will not be triggered when the properties toolbox is hidden. We can just not show the properties toolbox when the code popup is shown. You can change the logic here in App.js.

Now that I think about it, do we really still need the button for 'Copy code to clipboard'? One can open the code popup and copy the code from there. We can make it so all text will be selected when the code popup opens. What do you think?

Note: to be able to properly write in the textarea, I also had to change the value of user-select to text in CSS.

Does user-select: all not work?

For my workflow, there is no reason to keep the 'Copy code to clipboard' button, so I agree that we can remove it.
My problem with user-select seems to be browser specific. In Firefox, I get the following behaviour with -moz-user-select: all on a textarea:
I cannot move the cursor using the mouse. Using the arrow keys, I can only move the cursor to the beginning of a line. I cannot select text using the mouse and can only select a whole line when using the keyboard.
On the other hand, Opera and Chromium seem to ignore user-select on textareas completely, i.e. user-select: none still lets me select text.
I'm not sure which behaviour is correct since the specification is quite confusing for me and still a draft. Is there a reason not to use user-select: text (which seems to work in all three browsers)?

Is there a reason not to use user-select: text (which seems to work in all three browsers)?

No, I was just wondering; if text helps with all browsers, please keep the change. Thanks for the thorough research.

For my workflow, there is no reason to keep the 'Copy code to clipboard' button, so I agree that we can remove it.

Yes, let's remove the copy button; it would only cause confusion when there's a clipboard icon which shows code and a code icon which copies into the clipboard. Then we can use the code icon for the code popover as well.

Perfect, I have replaced the 'Copy code to clipboard' button.
Now this should be ready for review :-)

I try to find some time on the weekend to test everything out, but it looks pretty good 👍


Unable to Download Stories

Snapchat Downloader version: 1.4.3
Python version: 2.7.16
Operating System: Mac OS 10.15.7 (Catalina)

Description
I was trying to download stories as usual from my list but the command I normally use no longer seems to work. It was working as of 14 hours ago. Normally, when I type the command, it just starts saving stories from my list. I've tried this a few times and no luck. (Not really good at coding/tech stuff, so I don't really understand the error message... please also let me know if there are missing elements to this report and how I can find these things if needed). Thank you!
What I Did
Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

snapchat-dl -u -i Usernames0.bat  -P SnapDownloads
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 63, in _web_fetch_story
response_json = json.loads(response_json_raw[0])
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/bin/snapchat-dl", line 8, in
sys.exit(main())
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 58, in main
download_users(usernames)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 53, in download_users
downlaoder.download(username)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 93, in download
stories, snap_user = self._web_fetch_story(username)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 82, in _web_fetch_story
raise APIResponseError
snapchat_dl.utils.APIResponseError

I'm also getting this same error, if anyone has figured out a fix or wants to fork this project to fix this issue reach out to me please.

I used chatgpt to fix it, just replace the snapchat_dl.py with this:
`"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re
import requests
from loguru import logger
from snapchat_dl.downloader import download_url
from snapchat_dl.utils import APIResponseError
from snapchat_dl.utils import dump_response
from snapchat_dl.utils import MEDIA_TYPE
from snapchat_dl.utils import NoStoriesFound
from snapchat_dl.utils import strf_time
from snapchat_dl.utils import UserNotFoundError
class SnapchatDL:
"""Interact with Snapchat API to download story."""
def __init__(
    self,
    directory_prefix=".",
    max_workers=2,
    limit_story=-1,
    sleep_interval=1,
    quiet=False,
    dump_json=False,
):
    self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
    self.max_workers = max_workers
    self.limit_story = limit_story
    self.sleep_interval = sleep_interval
    self.quiet = quiet
    self.dump_json = dump_json
    self.endpoint_web = "https://www.snapchat.com/add/{}/"
    self.regexp_web_json = (
        r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
    )
    self.response_ok = requests.codes.get("ok")

def _api_response(self, username):
    web_url = self.endpoint_web.format(username)
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36"
    }
    response = requests.get(web_url, headers=headers)
    return response.text

def _web_fetch_story(self, username):
    response = self._api_response(username)
    response_json_raw = re.findall(self.regexp_web_json, response)

    if not response_json_raw:
        logger.error(f"No JSON response found for {username}. Response: {response}")
        raise APIResponseError

    try:
        response_json = json.loads(response_json_raw[0])

        def util_web_user_info(content: dict):
            if "userProfile" in content["props"]["pageProps"]:
                user_profile = content["props"]["pageProps"]["userProfile"]
                field_id = user_profile["$case"]
                return user_profile[field_id]
            else:
                raise UserNotFoundError

        def util_web_story(content: dict):
            if "story" in content["props"]["pageProps"]:
                return content["props"]["pageProps"]["story"]["snapList"]
            return list()

        user_info = util_web_user_info(response_json)
        stories = util_web_story(response_json)
        return stories, user_info

    except (IndexError, KeyError, ValueError) as e:
        logger.error(f"Failed to fetch story for {username}. Error: {e}, Response: {response}")
        raise APIResponseError

def download(self, username):
    """Download Snapchat Story for `username`.

    Args:
        username (str): Snapchat `username`

    Returns:
        [bool]: story downloader
    """
    stories, snap_user, *_ = self._web_fetch_story(username)

    if len(stories) == 0:
        if self.quiet is False:
            logger.info("\033[91m{}\033[0m has no stories".format(username))

        raise NoStoriesFound

    if self.limit_story > -1:
        stories = stories[0 : self.limit_story]

    logger.info("[+] {} has {} stories".format(username, len(stories)))

    executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
    try:
        for media in stories:
            snap_id = media["snapId"]["value"]
            media_url = media["snapUrls"]["mediaUrl"]
            media_type = media["snapMediaType"]
            timestamp = int(media["timestampInSec"]["value"])
            date_str = strf_time(timestamp, "%Y-%m-%d")

            dir_name = os.path.join(self.directory_prefix, username, date_str)

            filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
                snap_id, username, MEDIA_TYPE[media_type]
            )

            if self.dump_json:
                filename_json = os.path.join(dir_name, filename + ".json")
                media_json = dict(media)
                media_json["snapUser"] = snap_user
                dump_response(media_json, filename_json)

            media_output = os.path.join(dir_name, filename)
            executor.submit(
                download_url, media_url, media_output, self.sleep_interval
            )

    except KeyboardInterrupt:
        executor.shutdown(wait=False)

    logger.info("[✔] {} stories downloaded".format(username))

`

Thanks @promers for the suggestion. I copy pasted your code and completely replaced what was in the original snapchat_dl.py and saved it but it still did not work.
Maybe I'm doing something wrong?

Ok, thanks to @promers, I also tried Chatgpt to fix things and this is what mine came up with (and it works). Like @promers suggested, replace snapchat_dl.py with the following:
``
"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re
import requests
from loguru import logger
from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
APIResponseError,
dump_response,
MEDIA_TYPE,
NoStoriesFound,
strf_time,
UserNotFoundError,
)
class SnapchatDL:
"""Interact with Snapchat API to download story."""
def __init__(
    self,
    directory_prefix=".",
    max_workers=2,
    limit_story=-1,
    sleep_interval=1,
    quiet=False,
    dump_json=False,
):
    self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
    self.max_workers = max_workers
    self.limit_story = limit_story
    self.sleep_interval = sleep_interval
    self.quiet = quiet
    self.dump_json = dump_json
    self.endpoint_web = "https://story.snapchat.com/@{}"
    self.regexp_web_json = (
        r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
    )

def _api_response(self, username):
    web_url = self.endpoint_web.format(username)
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3"
    }
    return requests.get(web_url, headers=headers)

def _web_fetch_story(self, username):
    """Download user stories from Web.

    Args:
        username (str): Snapchat `username`

    Raises:
        APIResponseError: API Error

    Returns:
        (list, dict): stories, user_info
    """
    response = self._api_response(username)

    if response.status_code != requests.codes.ok:
        logger.error(f"Failed to fetch data for {username}. Status code: {response.status_code}. "
                     f"Response: {response.text}")
        raise APIResponseError

    response_json_raw = re.findall(self.regexp_web_json, response.text)

    try:
        response_json = json.loads(response_json_raw[0])

        def util_web_user_info(content: dict):
            if "userProfile" in content["props"]["pageProps"]:
                user_profile = content["props"]["pageProps"]["userProfile"]
                field_id = user_profile["$case"]
                return user_profile[field_id]
            else:
                raise UserNotFoundError

        def util_web_story(content: dict):
            if "story" in content["props"]["pageProps"]:
                return content["props"]["pageProps"]["story"]["snapList"]
            return list()

        user_info = util_web_user_info(response_json)
        stories = util_web_story(response_json)
        return stories, user_info
    except (IndexError, KeyError, ValueError) as e:
        logger.error(f"Error parsing response for {username}: {e}")
        raise APIResponseError

def _download_media(self, media, username, snap_user):
    snap_id = media["snapId"]["value"]
    media_url = media["snapUrls"]["mediaUrl"]
    media_type = media["snapMediaType"]
    timestamp = int(media["timestampInSec"]["value"])
    date_str = strf_time(timestamp, "%Y-%m-%d")

    dir_name = os.path.join(self.directory_prefix, username, date_str)
    os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

    filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
        snap_id, username, MEDIA_TYPE[media_type]
    )

    if self.dump_json:
        filename_json = os.path.join(dir_name, f"{filename}.json")
        media_json = dict(media)
        media_json["snapUser"] = snap_user
        dump_response(media_json, filename_json)

    media_output = os.path.join(dir_name, filename)
    return media_url, media_output

def download(self, username):
    """Download Snapchat Story for `username`.

    Args:
        username (str): Snapchat `username`

    Raises:
        NoStoriesFound: If no stories are found
    """
    stories, snap_user = self._web_fetch_story(username)

    if not stories:
        if not self.quiet:
            logger.info(f"\033[91m{username}\033[0m has no stories.")
        raise NoStoriesFound

    if self.limit_story > -1:
        stories = stories[:self.limit_story]

    logger.info(f"[+] {username} has {len(stories)} stories.")

    executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
    try:
        for media in stories:
            media_url, media_output = self._download_media(media, username, snap_user)
            executor.submit(download_url, media_url, media_output, self.sleep_interval)

    except KeyboardInterrupt:
        executor.shutdown(wait=False)

    logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

``

Alot of verbose output with 404'd users with those fixes.
Here's a fix for that too.
"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re

import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
    APIResponseError,
    dump_response,
    MEDIA_TYPE,
    NoStoriesFound,
    strf_time,
    UserNotFoundError,
)

class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=2,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://story.snapchat.com/@{}"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/58.0.3029.110 Safari/537.3"
            )
        }
        return requests.get(web_url, headers=headers)

    def _web_fetch_story(self, username):
        """Download user stories from Web.

        Args:
            username (str): Snapchat `username`

        Raises:
            APIResponseError: API Error

        Returns:
            (list, dict): stories, user_info
        """
        response = self._api_response(username)

        if response.status_code != requests.codes.ok:
            if response.status_code == 404:
                logger.error(f"User '{username}' not found (404).")
            else:
                logger.error(f"Failed to fetch data for '{username}'. Status code: {response.status_code}.")
            raise APIResponseError

        response_json_raw = re.findall(self.regexp_web_json, response.text)

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info
        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Error parsing response for '{username}': {e}")
            raise APIResponseError

    def _download_media(self, media, username, snap_user):
        snap_id = media["snapId"]["value"]
        media_url = media["snapUrls"]["mediaUrl"]
        media_type = media["snapMediaType"]
        timestamp = int(media["timestampInSec"]["value"])
        date_str = strf_time(timestamp, "%Y-%m-%d")

        dir_name = os.path.join(self.directory_prefix, username, date_str)
        os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

        filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
            snap_id, username, MEDIA_TYPE[media_type]
        )

        if self.dump_json:
            filename_json = os.path.join(dir_name, f"{filename}.json")
            media_json = dict(media)
            media_json["snapUser"] = snap_user
            dump_response(media_json, filename_json)

        media_output = os.path.join(dir_name, filename)
        return media_url, media_output

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`
        """
        try:
            stories, snap_user = self._web_fetch_story(username)
        except APIResponseError:
            logger.error(f"Could not fetch data for '{username}'. The user may not exist or has no public stories.")
            return

        if not stories:
            if not self.quiet:
                logger.info(f"{username} has no stories.")
            return

        logger.info(f"[+] {username} has {len(stories)} stories.")

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                media_url, media_output = self._download_media(media, username, snap_user)
                executor.submit(download_url, media_url, media_output, self.sleep_interval)
        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

Thanks for the correction. It works better now!
Interestingly, some SnapStar profiles (have golden stars beside their usernames) have stories when viewed on Snap but not when their profile is viewed on the web (i.e. as https://www.snapchat.com/add/username). As such, snapchat-dl is unable to download their stories. Is there any workaround for those?

Thanks @promers for the suggestion. I copy pasted your code and completely replaced what was in the original snapchat_dl.py using a text editor, and then saved it, but it still did not work.
Maybe I'm doing something wrong?
This is what I get when I try:
snapchat-dl foodwithmichel -P SnapDownloads

2024-10-02 20:21:03.367 | ERROR    | snapchat_dl.snapchat_dl:_web_fetch_story:73 - Failed to fetch story for foodwithmichel. Response: <!doctype html><meta charset="utf-8"><meta name=viewport content="width=device-width, initial-scale=1"><title>403</title>403 Forbidden
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 46, in _web_fetch_story
    response_json = json.loads(response_json_raw[0])
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/snapchat-dl", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 58, in main
    download_users(usernames)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 53, in download_users
    downlaoder.download(username)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 85, in download
    stories, highlights, snap_user, *_ = self._web_fetch_story(username)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 74, in _web_fetch_story
    raise APIResponseError
snapchat_dl.utils.APIResponseError

Oops my bad. Even though it worked on my end, it's seems that chatgpt did not fully update the code when i tried to add highlights. Glad you fixed it.

Alot of verbose output with 404'd users with those fixes.
Here's a fix for that too.
"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re

import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
    APIResponseError,
    dump_response,
    MEDIA_TYPE,
    NoStoriesFound,
    strf_time,
    UserNotFoundError,
)

class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=2,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://story.snapchat.com/@{}"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/58.0.3029.110 Safari/537.3"
            )
        }
        return requests.get(web_url, headers=headers)

    def _web_fetch_story(self, username):
        """Download user stories from Web.

        Args:
            username (str): Snapchat `username`

        Raises:
            APIResponseError: API Error

        Returns:
            (list, dict): stories, user_info
        """
        response = self._api_response(username)

        if response.status_code != requests.codes.ok:
            if response.status_code == 404:
                logger.error(f"User '{username}' not found (404).")
            else:
                logger.error(f"Failed to fetch data for '{username}'. Status code: {response.status_code}.")
            raise APIResponseError

        response_json_raw = re.findall(self.regexp_web_json, response.text)

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info
        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Error parsing response for '{username}': {e}")
            raise APIResponseError

    def _download_media(self, media, username, snap_user):
        snap_id = media["snapId"]["value"]
        media_url = media["snapUrls"]["mediaUrl"]
        media_type = media["snapMediaType"]
        timestamp = int(media["timestampInSec"]["value"])
        date_str = strf_time(timestamp, "%Y-%m-%d")

        dir_name = os.path.join(self.directory_prefix, username, date_str)
        os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

        filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
            snap_id, username, MEDIA_TYPE[media_type]
        )

        if self.dump_json:
            filename_json = os.path.join(dir_name, f"{filename}.json")
            media_json = dict(media)
            media_json["snapUser"] = snap_user
            dump_response(media_json, filename_json)

        media_output = os.path.join(dir_name, filename)
        return media_url, media_output

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`
        """
        try:
            stories, snap_user = self._web_fetch_story(username)
        except APIResponseError:
            logger.error(f"Could not fetch data for '{username}'. The user may not exist or has no public stories.")
            return

        if not stories:
            if not self.quiet:
                logger.info(f"{username} has no stories.")
            return

        logger.info(f"[+] {username} has {len(stories)} stories.")

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                media_url, media_output = self._download_media(media, username, snap_user)
                executor.submit(download_url, media_url, media_output, self.sleep_interval)
        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

Thanks for this. I was also having the same issue and this resolved it.

Alot of verbose output with 404'd users with those fixes.
Here's a fix for that too.
"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re

import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
    APIResponseError,
    dump_response,
    MEDIA_TYPE,
    NoStoriesFound,
    strf_time,
    UserNotFoundError,
)

class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=2,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://story.snapchat.com/@{}"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/58.0.3029.110 Safari/537.3"
            )
        }
        return requests.get(web_url, headers=headers)

    def _web_fetch_story(self, username):
        """Download user stories from Web.

        Args:
            username (str): Snapchat `username`

        Raises:
            APIResponseError: API Error

        Returns:
            (list, dict): stories, user_info
        """
        response = self._api_response(username)

        if response.status_code != requests.codes.ok:
            if response.status_code == 404:
                logger.error(f"User '{username}' not found (404).")
            else:
                logger.error(f"Failed to fetch data for '{username}'. Status code: {response.status_code}.")
            raise APIResponseError

        response_json_raw = re.findall(self.regexp_web_json, response.text)

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info
        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Error parsing response for '{username}': {e}")
            raise APIResponseError

    def _download_media(self, media, username, snap_user):
        snap_id = media["snapId"]["value"]
        media_url = media["snapUrls"]["mediaUrl"]
        media_type = media["snapMediaType"]
        timestamp = int(media["timestampInSec"]["value"])
        date_str = strf_time(timestamp, "%Y-%m-%d")

        dir_name = os.path.join(self.directory_prefix, username, date_str)
        os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

        filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
            snap_id, username, MEDIA_TYPE[media_type]
        )

        if self.dump_json:
            filename_json = os.path.join(dir_name, f"{filename}.json")
            media_json = dict(media)
            media_json["snapUser"] = snap_user
            dump_response(media_json, filename_json)

        media_output = os.path.join(dir_name, filename)
        return media_url, media_output

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`
        """
        try:
            stories, snap_user = self._web_fetch_story(username)
        except APIResponseError:
            logger.error(f"Could not fetch data for '{username}'. The user may not exist or has no public stories.")
            return

        if not stories:
            if not self.quiet:
                logger.info(f"{username} has no stories.")
            return

        logger.info(f"[+] {username} has {len(stories)} stories.")

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                media_url, media_output = self._download_media(media, username, snap_user)
                executor.submit(download_url, media_url, media_output, self.sleep_interval)
        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

ok so i downloaded the source code and then i edited the snapchat_dl.py file and copy pasted this code. how do i actually run the program now? (which directory do i have to cmd into and what command do i use?)

You dont need to download the source code manually.
Install it via pip like normal, then locate your copy of snapchat_dl.py that pip gave you.
Something like this:
"C:\Users\USERNAME\AppData\Local\Programs\Python\Python311\Lib\site-packages\snapchat_dl\snapchat_dl.py"
Basically the editable file will be in your site-packages/snapchat_dl directory.

Seems like Snapchat changed again, can't view posts anymore with their original username. Im not sure but it seems they changed the username to a random string like this (https://www.snapchat.com/p/7f66910b-1cee-4824-8562-8b00ac90217a) I could be wrong.

Thanks for this guys, can someone let me know if you can download stories like these? https://www.snapchat.com/p/6f098158-691e-46b4-a82d-6a98b3f15dc8/3298905170300928
When I input the username it says it has no stories so I am not exactly sure what is going on.


Question regarding alignment generation

Hi there!
Great paper and great code! I just have a small question regarding the binary alignment matrix part of the code and paper.
In this part in the main function:
align_info = open(align_path, 'r').readlines()[1:-1] # get rid off the head and tail info
        align_info = [re.split('\ +', x) for x in align_info]
        align_info = [(x[-1].strip('\n').split('(')[0].lower(),int(x[1]),int(x[2])) for x in align_info]
        # For the silence probably we can make some use
        align_info = [x for x in align_info if x[0] not in ['<s>','<sil>','</s>']]
        align_input = []
        for _, begin_time, end_time in align_info:
            begin_idx = int(begin_time * 0.01 / 0.01)
            end_idx = int(end_time * 0.01 / 0.01) + 1
            align_slice = torch.zeros(audio_input.size(0))
            align_slice[begin_idx:end_idx] = 1.0
            align_input.append(align_slice[None,:])
        align_input = torch.cat(align_input, dim=0)

If align info is a generic alignment file with timestamps based on seconds, then I am not sure how begin_idx = int(begin_time * 0.01 / 0.01) would work. According to my intuition this should probably be:
begin_idx = int(begin_time * sampling_rate / frame_step)

Same question for end_idx.
Please let me know if there is something wrong in my understanding!
Thank You

Hi,
Thanks for your appreciation of our work. And I am sorry that I didn't leave too much comments for some details for our code.
For the alignment, we use the Montreal-Force-Aligner(https://github.com/MontrealCorpusTools/Montreal-Forced-Aligner). The output timestamps for each token(word) is saved as frame index, e.g.,
SFrm  EFrm    SegAScr Word
0     6     300581
7    71    3580522
72    94    -819956 LIKE
95   137     -99401 WHAT(2)
138<PHONE_NUMBER>
260   276    -623939 LIKE
277   282    -161498 A
283   288     -60769
289   311    -668106 BIRTH
312   375    -988951 CERTIFICATE(6)
376   391     274192
392   408     557909
The default frame step for Montreal-Force-Aligner is set as 0.01, and to make it compatible with our model's frame step, we choose to use the formula: frame_index_by_Force_Aligner * Frame_Step_of_Force_Aligner / Frame_Step_of_Ours. And this makes our code looks like int(begin_time * 0.01 / 0.01).
Hope this can be helpful for you.
Best

hi @hzlihang99 ,
Thank you for your answer. I had one more question. I wanted to know the significance of passing the alignment matrix through nn.AvgPool2d(). Since the ultimate goal is to find out relevant frames for each word and do mean + max pool I was just thinking how an average pool over the alignment matric would affect the alignments.
Thank You!


The Need for a Chat Feature in the Application for Follow-up Questions.

I feel like this application needs a chat feature to ask follow-up questions. Currently, when using any feature, we cannot ask follow-up questions, especially when debugging. If there were a chat feature to ask additional questions, it would be greatly appreciated .
Thank you .

Hello @abirmehmed ,
Thank you for the suggestion.
A lot of change is coming soon. I agree with you about the Chat Feature.
Some additional features for logged used will be available (so we could put more easily quota)
Best Regards,
Jon


Incorporating the writings of 3idiots.md in the actual website

3idiots.md is there is the repo. What has to be done is that whatever that is written in 3idiots.md  will have to be incorporated in the website just the way the other reviews are there. Follow the pattern and incorporate it.

can i be assigned to this issue please?

Ok, assigning it to you

Thank you for assigning me and allowing me to contribute.


Feature.agent startup msg fix 418

What issues does this address?
Fixes #418
WIP #563
...
What's this change do?
This is another way to handle the dependencies, but errors out gracefully if/when the user runs the agent.py directly.
Where should the reviewer start?
In the commit
Any background context?
The need to handle this in exceptions is not really there.  One could create a crash.py and handle it there just as simply; however, as all other errors are handled in the exceptions.py, I thought it best to keep it simple with it being there.  Users can then decide whether or not they wish to except on that class upon its creation.  Note that, by default, the arg test will show up in the exception message.

Technically, if the /var/log/neutron/f5-oslbaasv2-agentdoes not exist, then a IOError is thrown at line 411 (currently) of the f5_openstack_agent.lbaasv2.drivers.bigip.exceptions module.
This can be handled to gracefully exit and attempt to simply store the log under syslog.syslog(syslog.LOG_ERR, <msg>).  This would eliminate the issue of an unhandled exception croaking the agent.py.  The down side is that it would appear to fail silently if the user does not have central logging for syslog set up for either /var/log/syslog, /dev/log, or /var/log/messages (apple would be /var/run/syslog, I believe).
Let me know your thoughts.  It could be a 3-5 line change.  Possibly more to handle it more sustainably.

@richbrowne @jlongstaf I think this can be merged.


[Major change]Switch to DAG push down mode.

Original done via SelectRequest, the variety and flexibility of operations that can be pushed down is limited. Switching to DAGRequest to support more operations and benefit from it's high flexibility.

This change is

@Novemser Please resolve the conflicts. In addition, is this PR ready to review?

@zhexuany Conflicts resolved. Streaming interface is still under construction, so the PR is not yet fully ready for review, but other parts is finished and ready for review. If hurrying, you may start from codes that do not concern about streaming interface.

@zhexuany After some discussion, streaming interface will be implemented in another branch, so this PR is ready for review.

@Novemser Please cleanup code changes. I saw a lot of commented codes.

@zhexuany Thanks, code clean up complete.

After some integration test, this PR still has some bug and need to be fixed, currently I'm working on it and we may review this PR later.

Fixed an issue and made TPCH test passed.

PTAL @zhexuany @ilovesoup

Review status: 0 of 61 files reviewed at latest revision, 31 unresolved discussions, some commit checks failed.

src/main/java/com/pingcap/tikv/expression/scalar/Divide.java, line 39 at r19 (raw file):

  @Override
  public DataType getType() {

Just want to confirm, this function has same implementation in its super class?

src/main/java/com/pingcap/tikv/expression/scalar/If.java, line 40 at r19 (raw file):

  @Override
  protected void validateArguments(TiExpr... args) throws RuntimeException {
  }

Why not implement this? At least an Exception need to be thrown here.

src/main/java/com/pingcap/tikv/expression/scalar/IfNull.java, line 40 at r19 (raw file):

  @Override
  protected void validateArguments(TiExpr... args) throws RuntimeException {
  }

Same as above comments.

src/main/java/com/pingcap/tikv/expression/scalar/In.java, line 26 at r19 (raw file):

import static java.util.Objects.requireNonNull;

public class In extends ScalarFunction {

For now, Every classes have a prefix with Ti. It is better we are consistent with this. We can always change it later.

Comments from Reviewable

@Novemser please resolve the conflicts locally and push newly committed commit to your branch. Thanks. 👍

@zhexuany Thanks for your review! New commits have been pushed, still looking forward to your review. : )


Wood

Wood is now a good produced by forests each tick.

Wood is used to build new houses, and if wood is not available, people become displaced

A variable is in the wood object that determines if wood will be bought if a deficit is encountered

Wood is not simulated for other countries because they do not have maps. It is assumes that there is infinite wood in the market


To avoid error of "DDL Length Exceeded"

Increased default harcoded limit on Varchar from 256 to MAX

archiving repository


TypeError: nil can't be coerced into Fixnum

I have following code:
bank = EuCentralBank.new
bank.add_rate(:EUR, :USD, 10)
Money.default_bank = bank
Money.new(100, :EUR).exchange_to(:USD)

And i get
TypeError: nil can't be coerced into Fixnum
from /Users/dmitrij/.rvm/gems/ruby-2.2.2@my_app/gems/eu_central_bank-0.5.0/lib/eu_central_bank.rb:57:in `/'

Apparently rate is set to nil, but:
bank.get_rate(:EUR, :USD)
=> 10
Money.default_bank.get_rate(:EUR, :USD)
=> 10

What could be the problem?
Thanks in advance.

This gem is unique in that it needs to know how both the from and to of a rate conversion relate to :EUR, so if you do bank.add_rate(:EUR, :EUR, 1) everything would work. But more importantly, why are you defining your own rates and not just calling bank.update_rates?

Ok, thank you very much. It's working perfect! It is necessary to run tests, because Real HTTP connections are disabled in test env
Counter question. Can I bypass euro relation and add a custom rate for example from USD to CAD?

If you're using the eu_central_bank gem, no. Can you use the VCR gem in your test env to record the update_rates call once and then use it in future tests?

I'll try. Thanks for the advice!


Set version on Azure/action-release-workflows.

thanks for updating this! i added the latest hash in #98 since the latest version doesn't have a tag


Automatic testing, formation and coverage refactoring

Discussion on suggested changes to testing/formation/coverage by @Borda

@williamFalcon I have extended the CI so it runs overall package with py.test and checks basic PEP8 with flake8. Moreover, there are real badges, not fake ones like codecov and I have added CodeFactor.io which may help improve the code quality by suggestions in general and also in the time of PR... You may have look at updated sample project - https://github.com/Borda/py_sample-project

Adding here again for clarity:
The code coverage badge was generated by doing this:
https://github.com/williamFalcon/pytorch-lightning/tree/master/tests#running-coverage

CircleCI does not offer any GPU plans for free. If you're interested in upgrading to a paid performance plan you can review the attached document... Performance Plan - Pricing Infosheet Q1_19.pdf
This is the official reply from CircleCI, @williamFalcon have a look

thanks for doing research on this. will stick to free for now ;)


Notebook JSON is invalid ('header' was unexpected)

After upgrade to JupyterLab 2.0.1, we get this error intermittently. Annoying, but it doesn't actually seem to cause any harm?
[I 13:13:33.708 LabApp] Saving file at /ihaskell_examples/ihaskell/Untitled.ipynb
[E 13:13:33.739 LabApp] Notebook JSON is invalid: Additional properties are not allowed ('header' was unexpected)

    Failed validating 'additionalProperties' in error:

    On instance['cells'][5]['outputs'][0]:
    {'ename': '',
     'evalue': '',
     'header': 'MessageHeader {mhIdentifiers = '
               '["4e16c3b3-fe92-4de9-9c64-39a0a3c...',
     'output_type': 'error',
     'traceback': ['<interactive>:5:14: error: Variable not in scope: void :: m0 '
                   '(To...']}

this problem went away


Node v5.1.1, v4.2.3, v0.12.9, v0.10.41

4 new versions of Node were released today to resolve various security issues in OpenSSL, V8 and Node itself.
Could you add these to Docker Hub please?
Thanks!

Hey @billinghamj,
Sorry for the delay, (the process of getting them onto hub is still a manual one), they are building locally now! Will push soon :smile:

@billinghamj I stand corrected. It looks like we are building the binaries in the upstream repo right now, once they become available there I'll trigger the build here.

Any chance this might get resolved today? Very keen to update to deal with the vulnerabilities.

+1

+1

Hey everyone. The registry is updated now and the builds have been triggered. The Docker registry doesn't have an API endpoint for updating automatic build tags yet, so we are still doing it by hand which adds to the time. There were ~140 changes that need to be made when shipping these. Looking into better solutions that will allow us to ship faster.

@wblankenship Thanks for your work. Our production app depends on the nodesource docker image. It has been in very high quality.

Thank you for the kind words @d6u :heart:
Would love to chat sometime about your use of the images<EMAIL_ADDRESS>

@wblankenship Thanks for your work. I wanted to rebuild our image but I realized, that there is no update. I'm using nodesource:trusty/4 image tag. It seems tag 4 and 4.2 was forgotten to update (and possibly others). They are also missing from build page: https://hub.docker.com/r/nodesource/trusty/builds/
Also from command line can be seen that the digest for logically same image differs:
$ docker pull nodesource/trusty:4.2.3
Digest: sha256:c6257044c78b2cb7a50dc621716aff4b0951bc69cdced25bfb22b49a49c706fd

$ docker pull nodesource/trusty:4.2
Status: Downloaded newer image for nodesource/trusty:4.2

$ docker pull nodesource/trusty:4
Digest: sha256:46afe50c84b4b98ac51d9ed51d7fdb7877902a9e1edd71892090d139c0db232a

Do you still support tags prefixes or they were forgotten by mistake?

@xmikus01 I'll break this up into two parts, both of which are super weird.
Part 1: Not having an update for 4
The 4[.2[.3]] values were updated in the automated build settings. I just visually verified:

I also have a tool that verifies this for me (since I can't get an API to automate the updates). When dealing with 400+ Docker images each with several aliases, human error causes quite a bit of drift so I don't trust myself to maintain it by hand. https://github.com/nodesource/docker-node/blob/master/tools/registry_diff.js
I pulled down nodesource/trusty:4 and can confirm that it is 4.2.3

That being said, this wouldn't be the first time the Docker Registry didn't properly update a tag. We are considering dropping the Automated Build tag and uploading our own build logs and pushing our own images (we generate thorough logs every-time we run our integration tests).
Part 2: Digest not matching
I have rarely been able to get the digests to match on Automated Builds, even when the Dockerfiles are aliases. I think the culprit is the apt-get commands, and the downloads not being deterministic enough (perhaps timestamps?) to generate an identical hash.
Again, this would be solved by dropping the Automated Build status.
I hope to propose a better solution for all of this in the coming weeks.

@wblankenship Thank you for your very detailed response. You are right, I must apologize for my lack of consistency in the search for the real cause. I was led down to the wrong path believing the Docker Hub tags page and my ignorance of different hashes for the same image aliases.
I understand that it must be a very big task to build a such big number of images without appropriate API and support for full automation and I very appreciate your effort to make things better :-)


[GSSOC'23] SportsBot for Telegram

I want to make a sports bot for telegram. Introducing "SportsBot" – your ultimate companion for all things sports! This dynamic Telegram bot keeps audience engaged and up-to-date with the latest sports news, match scores, player statistics, and exciting sports trivia. Whether the user is  a passionate fan or simply curious about sports, SportsBot provides a comprehensive platform to satisfy their cravings for sports-related information.
SportsBot ensures that the user will never miss a moment of the action, keeping you at the edge of his seat. It also offers interactive sports trivia quizzes.

Please assign me this issue under GSSOC'23 !! I would love to work on this.

having prior knowledge of how to make telegram bots i am confident about doing this task fully. Kindly give me a chance to solve this issue.

Hi!
I want to work on this issue as a part of GSSoC'23. Could you please assign me and I am giving you my best and 100%.

Please assign me this issue under GSSOC'23


[WIP][UI] Source/target storage

Motivation and context

How has this been tested?

Need to update e2e tests:

[x] actions_projects_models/case_103_project_export.js
[x] actions_projects_models/case_104_project_export_3d.js
[x] actions_projects_models/case_114_backup_restore_project.js
[x] actions_tasks/case_52_dump_upload_annotation.js
[x] actions_tasks/issue_2473_import_annotations_frames_dots_in_name.js
[x] actions_tasks2/case_97_export_import_task.js
[x] actions_tasks3/case_47_export_dataset.js
[x] issues_prs2/issue_1568_cuboid_dump_annotation.js
[x] canvas3d_functionality/case_91_canvas3d_functionality_dump_upload_annotation_point_cloud_format.js
[x] canvas3d_functionality/case_92_canvas3d_functionality_dump_upload_annotation_velodyne_points_format.js
[x] canvas3d_functionality/case_93_canvas3d_functionality_export_dataset.js

Checklist

[ ] I submit my changes into the develop branch
[ ] I have added a description of my changes into CHANGELOG file
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[ ] I have linked related issues (read github docs)
[ ] I have increased versions of npm packages if it is necessary (cvat-canvas,
cvat-core, cvat-data and cvat-ui)

License

[ ] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

@Marishka17 , could you please add a couple of screenshots into description of the PR?

Let's add a space between these two fields (<Col offset={1} ...)

I am not sure this hint is correct (I want to export annotations from a task in a project with attached source and target storages)

If I try to export a task dataset from a task (within a project with attached source and target storages) without specifying a custom name:

If I try to export with these settings:

It downloads file locally, but works without specifying a custom name:

A regular user who is assigned to a job can not export annotations for a job:

Related issue is that exporting a dataset from a job page actually exports a dataset for a task (task instance is used instead of job instance).
Maybe these are not exactly issues of this patch, but would be great if you tried to fix it here.

If I try to upload annotations to a task from a cloud storages:

For exporting backups/datasets we have export-actions.ts and export-reducer.ts.
For importing backups/datasets we have import-actions.ts, import-backup-actions.ts and import-reducer.ts, import-backup-reducer.ts.
Why so?


Partial Resolve issue #12 (part 2)

Moving height/width into render resolves issue where overlay uses initial orientation dimensions instead of current orientation dimensions.
If the overlay is already open however, the issue is not resolved.

@dorthwein
This PR looks good, thank you for your PR.


New PCL project, can't build

Error is:
1>C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(1410,2): error : Exception while loading assemblies: System.IO.FileNotFoundException: Could not load assembly 'BottomBar.XamarinForms, Version=1.0.6133.37691, Culture=neutral, PublicKeyToken='. Perhaps it doesn't exist in the Mono for Android profile?
1>C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(1410,2): error : File name: 'BottomBar.XamarinForms.dll'
1>C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(1410,2): error :    at Java.Interop.Tools.Cecil.DirectoryAssemblyResolver.Resolve(AssemblyNameReference reference, ReaderParameters parameters)
1>C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(1410,2): error :    at Xamarin.Android.Tasks.ResolveAssemblies.AddAssemblyReferences(ICollection`1 assemblies, AssemblyDefinition assembly, Boolean topLevel)
1>C:\Program Files (x86)\MSBuild\Xamarin\Android\Xamarin.Android.Common.targets(1410,2): error :    at Xamarin.Android.Tasks.ResolveAssemblies.Execute()

This is a brand new PCL project - BottomBar 1.1.2 is installed. If I remove BottomNavigationBarXF it can build/run again, as soon as I add BarXF back in I get the above error when building.  Have tried it numerous times with with fresh PCL projects. Have tried BottomNavigationBarXF 1.0.0, same error.
<packages>
  <package id="BottomNavigationBar" version="1.1.2" targetFramework="monoandroid60" />
  <package id="Newtonsoft.Json" version="8.0.3" targetFramework="monoandroid60" />
  <package id="ThriveGmbH.BottomNavigationBar.XF" version="1.0.1" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.Animated.Vector.Drawable" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.Design" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.v4" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.v7.AppCompat" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.v7.CardView" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.v7.MediaRouter" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.v7.RecyclerView" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Android.Support.Vector.Drawable" version="23.3.0" targetFramework="monoandroid60" />
  <package id="Xamarin.Forms" version="<IP_ADDRESS>" targetFramework="monoandroid60" />
</packages>

I have the same exact problem, could you find a workaround?

No I haven't. I'm guessing that the underlying issue may be the fact that you can't install the package at the PCL root (it will install to Project.xyz.Droid but not Project.xyz) due to this error:
Could not install package 'ThriveGmbH.BottomNavigationBar.XF 1.0.1'. You are trying to install this package into a project that targets '.NETPortable,Version=v4.5,Profile=Profile259', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.

I thought to try installing only BottomNavigationBar 1.1.2 in the root PCL and that throws a similar error:
Could not install package 'Xamarin.Android.Support.v4 23.3.0'. You are trying to install this package into a project that targets '.NETPortable,Version=v4.5,Profile=Profile259', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.
So my guess is that neither package is up to date with the latest Xamarin PCL packages. I don't know how to fix that - there is a fork of this project that is more recent and I'm planning to try to get that installed at some point but probably not today https://github.com/mikescandy/BottomNavigationBarXF/commits/master his recent commits look like they are related to this bug.

Awesome! Do you have any idea how I can install his branch? I have no idea D:

No I haven't gotten that far yet. If you look at the other repository it contains a solution and nuget.exe - from what I've gathered it might be possible to create a nupkg from that code. I'm guessing it's simpler than that but I haven't figured it out either.

Ok, well, Ifirst just downloaded the whole project from that guy, it includes an example, it works perfectly.
Then, I figured what he changed and all he did was to change the PCL target profile to use 259 (which is the default from some recent xamarin versions) from 78 (the one used by the original plugin) and I found this:
http://danrigby.com/2014/04/16/xamarin-pcl-profile-notes/
And I managed to get my PCL project to be profile 78, first you need to uninstall every nuget package, close the project, reopen, install xamarin forms <IP_ADDRESS>, then install the BottomNavigationBarXF plugin from nuget and everything seems to go perfect, everything installs without errors.
The thing now is that when you create a tabbed page, put some pages there and run the project, you don't get the page in the bottom, it's like nothing changed.
I haven't been able to install the updated version from Michele (mikescandy repo) because I don't know how to install the nuget from local folder.
So, I wanted to dig a bit more, I downloaded this repo and opened the example project, works as expected. I then tried to see the TabbedPage definition from there and compare to the one I have in my project and it's not the same, this one implements some interfaces that are not part of my xamarin forms.
So, that's it, I'm gone for now, but I'll get back to this issue in a couple of days, I wanted to share this with you to see if you come to an idea or something that can help.

I worked out how to do it with adding the latest fork's nuget - but, I think there is an easier and cleaner way. Basically, you're not getting dependency management if you make your own nuget package from a fork (since any attempt to update/downgrade that package won't work). So adding the dlls from the fork makes more sense. Building the solution in release mode gives you the right dlls under the project bin/ dir.
You want to add BottomBar.XamarinForms.dll to the PCL and iOS projects (I haven't tested it under iOS btw, and I probably won't test it for a couple more days since I'm focusing on Android right now). For Android you want three dlls, BottomBar.Droid, BottomBar.XamarinForms and BottomNavigationBar. I haven't tested this approach 100% yet - there's an XML file, BottomNavigationBar.xml that looks like it would be required, I'm trying to just copy it to the same path as BottomNavigationBar.dll and hopefully that'll work, if it doesn't I'll write another comment with the procedure for building a nupkg (which I have tested and it is working, it's just a lot of steps and somewhat pointless for the reasons I mentioned above about nuget - another problem is that Visual/Xamarin Studio aren't able to download the nuget for other users since it only exists on your local machine, which is again, another violation of dependency management)

This is what worked for me.
I updated to the most recent version of Xamarin Forms. (<IP_ADDRESS>)
If you have BottomNavigationBar and ThriveGmbH.BottomNavigationBar.XF installed remove them first and clean your directory.
In this order, installed BottomNavigationbar v<IP_ADDRESS> then install ThriveGmbH.BottomNavigationBar.XF v1.0.1 then in your PCL project add a reference to BottomBar.XamarinForms.dll which should be located in your packages folder C:<path-to-your-xamarin-project>\packages\ThriveGmbH.BottomNavigationBar.XF.1.0.1\lib\portable-net45+win+wp80+MonoAndroid10+xamarinios10+MonoTouch10\BottomBar.XamarinForms.dll".
To Test use the following code(not sure if this is going to be formatted correctly or not):
public class TestPage : BottomBarPage     {         public TestPage()         {             Children.Add(new ContentPage()             {                 Content =                 new Label                 {                     Text = "Page 1",                     HorizontalOptions = LayoutOptions.CenterAndExpand,                     VerticalOptions = LayoutOptions.CenterAndExpand                 },                 Title = "Page 1"             });             Children.Add(new ContentPage()             {                 Content =                 new Label                 {                     Text = "Page 2",                     HorizontalOptions = LayoutOptions.CenterAndExpand,                     VerticalOptions = LayoutOptions.CenterAndExpand                 },                 Title = "Page 2"             });             Children.Add(new ContentPage()             {                 Content =                 new Label                 {                     Text = "Page 3",                     HorizontalOptions = LayoutOptions.CenterAndExpand,                     VerticalOptions = LayoutOptions.CenterAndExpand                 },                 Title = "Page 3"             });             NavigationPage.SetHasNavigationBar(this, true);             ToolbarItems.Add(new ToolbarItem()             {                 Text = "Something"             });         }     }

@tuneis Tested on iOS, only need to add the reference dll to the iOS project.
Thanks for your solution, it works like a charm

@tuneis variant works, but much better install BottomNavigationbar v<IP_ADDRESS> instead of v<IP_ADDRESS> to prevent a lot of warnings caused by different android.support version. Read release note for <IP_ADDRESS> version.


Optimising joins parquet

Merge request template: please remove the appropriate parts of this template.
Pre-merge request checklist (to be completed by the one making the request):

[X] I have performed a full review of this code myself.

For Python code in PySpark specific sections, all code should have been run in Jupyter notebooks.
For code in sections of the book containing both Python and R code, the page of the book should be constructed as described in the contributing guide and converted to a markdown file.

[X] I have formatted the outputs of code blocks correctly (to match other outputs in the book and in line with the style guide [coming soon])
[X] I have built the book as outlined in the contributing guide and confirmed that any additional/modified content is displaying as expected.

Details of this request (such as):
Modified the optimising joins chapter to read in a parquet file rather than a CSV file.
Also updated the Spark UI screenshots.
Requirements for review (such as):

Build book, check code and images display properly
Compare images against previous chapter to ensure they're displaying the same content (with the exception of CSVs being replaced with parquets)

@alexsnowdon I made your suggested changes, do you mind giving them a review and merging them in when you get a chance :)

@NathanKelly-ONS I can see you have made the suggested changes, unfortunately when I build the book figure 18 and figure 21 are now broken. For figure 18 it seems that in the .md file and in the .ipynb the figure is referenced differently. In the .md a 'soft_merge_join_ui_new.png' is referenced, whereas in the images folder and in the .ipynb this is just 'sort_merge_join_ui.png'! For figure 21, I cannot see what could be causing this as in all files it is referenced as when_ui.png.

@alexsnowdon finally managed to get it to work (I think!). It's working on my laptop and I've built and checked that all images are displaying, and they seem to be. Hopefully it's all good on your end too :)


(chore): introduce lint-staged

Main fix:
Currently for local development, developers are capable of committing source code that has potentially linting errors. To overcome this, I have installed lint-staged. Together with  husky it will enforce contributors` changes in the staging area to pass all linting checks before a commit. If the changes in the staging area fail linting checks, then the commit will fail.
This is configured to only lint .js, .ts, or .tsx files.
@rohinivsenthil & @Mukundan314  Going forward this will helpfully enforce better & stricter code style! 😄
Small Fixes:

Fix lint script to lint all  *.ts & *.tsx files,
Fix lint:fix script to fix all linting issues for all *.ts & *.tsx files
Fix all the current linting issues in source code.

Could you please run the lint on push rather than on commit?

I could do. However it will abandon the use of lint-staged & I can just enforce a pre-push hook to run linting on all source code. I guess the one thing that is advantageous with lint-staged is that it prevents any doggy commits from being in your commit history before pushing changes. Also it only performs linting on the files that someone has entered in the staging rather than the whole code base.

I agree, but I still believe it's better to run linting on push since:

we squash merge our PRs, so I think the commit history isn't a problem
it could be slightly annoying at times to run lint on each commit for the developer

A pre-push hook should suffice.

I agree, but I still believe it's better to run linting on push since:

we squash merge our PRs, so I think the commit history isn't a problem
it could be slightly annoying at times to run lint on each commit for the developer

A pre-push hook should suffice.

Thanks for the feedback. I understand your reasons. I will close this PR and raise another with just the pre-push hook functionality


filtering on value

Is this ready for review? Did you get the filter on value working?
Also, please include a rule comment at the top outlining the before and after state of the plan.

I need to remove some code but other than that it is ready...I fixed the problem with search query

Go ahead and remove the code and add the rule comments, then I will review the code.

All the test cases that have a prefix useindex test the new functionality....however, the only way to verify that the rule actually works is by debugging. I checked that and it works fine.

I see you added tests for attributes and indexing. Does this mean that the indexing works for attributes without any changes?

Yes, it works....the only change I made was that instead of storing the length of the attribute in the constructor of IndexAttributes (was always 0 since the constructor always initializes the lists) I store it one of the methods where I add elements in the lists.


Journal doesn't appear with collection tab

I've tried reloading the UI, closing the game, restarting my pc and nothing will make the soulshape journal appear. It shows up in the addon list but nowhere else. Please help.

Hello, the addon is currently not working with patch 10.0.
Resourceful users have provided a temporary fix in this thread : https://github.com/christopheml/soulshape-journal/issues/61


Update SPMSwiftDriverExecutor to Adopt TimePoint

In response to https://github.com/apple/swift-driver/pull/1171, adopt TimePoint over Foundation.Date in the interface of DriverExecutor.

apple/swift-driver#1171
@swift-ci test

https://github.com/apple/swift-driver/pull/1171
@swift-ci test

https://github.com/apple/swift-driver/pull/1171
@swift-ci smoke test

Creating working copy for https://github.com/apple/swift-driver.git
Working copy of https://github.com/apple/swift-driver.git resolved at main

So it appears the self-hosted builds don't truly support cross-repo PR testing yet.

@CodaFi yes, they are expected to fail in this case


Updating the uptime monitoring image

Now that we introduced the check for HTML body content so the image needs to be updated.

Can you run imageoptim over the file and amend it to this commit? So it's a smaller file size?


Event not showing up on the calendar

hey I am using the latest version in flutter app, trying to create an event. I see that the code return success, but when I go into the calendar, I see that event is not created. I would really appreciate any pointers as to why I am not seeing events in the calendar.
in a code  I am setting the calendar in a method, and calling another method to create the event.
       calendarsResult = await _deviceCalendarPlugin.retrieveCalendars();
       setState(() {
        _selectedCalendar = calendarsResult?.data[0];
        _addEventsToCalendar(_academicEvents);
       });

Future _addEventsToCalendar(List<AcademicEvent> events) async {
    if (_selectedCalendar != null) {
     if (events != null && events.length > 1) {
       for (var event in events) {
      final eventTime = new DateTime(event.eventStartDate.year, event.eventStartDate.month,   event.eventStartDate.day);
      final eventToCreate = new Event(_selectedCalendar.id);
      eventToCreate.title = event.eventSummary;
      eventToCreate.start = eventTime;
      eventToCreate.description = event.eventSummary;
      ;
      String eventId = event.eventUID.toString();
      if (eventId != null) {
        eventToCreate.eventId = eventId;
      }
      eventToCreate.end = new DateTime(DateFormat.yMMMd().format(eventStartDate));
      final createEventResult =
          await _deviceCalendarPlugin.createOrUpdateEvent(eventToCreate);
      if (createEventResult.isSuccess &&
          (createEventResult.data?.isNotEmpty ?? false)) {
        //print("events added to local calander");
      }
    }
  }
}

Which platform are you seeing this on?

I am building a flutter app, in android studio.

yes, but which platform (iOS or Android, or both) are you seeing this issue with?

oh..Both the platforms... calendar type is gmail

If you query the calendar using the API do you see the event?

do you mean after adding the event to the calendar?

yes - check that the API isn't seeing the event. Or is that what you meant by checking the calendar?
Also, please remove the line that sets the eventId - I suspect that might be preventing the event from being saved since this is auto generated.

I removed the id statement.
so this is interesting  in the createOrUpdateEvent(Event event), I see that the event is loaded.
but  when its invokes this method, at this line
await channel.invokeMethod('createOrUpdateEvent', mapobject);
channel is null...
but it does not through any exceptions also..

ok... I changed the calendar to look at the default one and also removed the setting of id...and now it seems to be adding the event. but there is  another issue..
When ever I go to certain page, I am loading and adding events, but I see that the event is being added to the calendar every time I open that page, i was expecting it to add only once and update it if there is change. how does it determine if the event is unique or not? I removed setting the eventId based on your previous comment. so how would it determine if its new event or old one now?

At this stage you'll need to retrieve the events and see if there is one that matches. You could use the Url property to store your event id and that way you can check the Url property

I see that retrieveevents functionality, but I see that the eventid is also required to be passed..so I am confused on how to retrieve events.

Hey i am also facing the same issue, But i figured it out where is the problem.
Problem is when we use build app(after flutter build apk) it cant gave us access to use user's calendar, we cant retrieve any calendars from user's devices. But when we use it with same code at debugging time it grant us to access to user's devices calendar there's no issue.
I dont know how is this happen.

Is this still an issue!?

event has been created successfully, I don't see it in my calendar


Diagnosis or useful alert for daemon.json broken

Tell us about your request
Diagnosis or useful tips for user's daemon.json file broken, the software quietly does not started.
Which service(s) is this request for?
Docker Desktop for Windows.
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
I accidentally break the C:\Users\<username>\.docker\daemon.json file while adding the "registry-mirrors" servers, it has a bad JSON. After a period of time, I tried to start the Docker Desktop again, it doesn't respond, no tray or alert. Sometimes it says Docker is exiting (meaning).
Are you currently working around the issue?

Rename the daemon.json file, re-start it, this file is rebuilt and the product is restored.
Use the C:\Program Files\Docker\Docker\resources\com.docker.diagnose.exe based on the doc, it collect many information and uploaded, no offline diagnostics options provided, and it only gives me a diagnostic ID (not available to the free plan), not helpful.

@yfdyh000 this is great feedback, we are currently working on some tools to help with diagnosis of issues and this is a great case we hadn't considered. I don't know if this will make it into the first release but we will 100% look at adding this to it!

Thanks for your idea! This is something we will consider as part of our wider efforts to improve error messages within Docker Desktop. I'll add to the 'Considering' column of our roadmap.


Don't HTML escape custom stylesheet

If you HTML escape the custom stylesheet, you can't have quotes in it.
You need quotes if you want to add something with url('foo') (in a
background-image by exemple).

:beers:


Support numpy arrays for categorical_features and feature_names

In https://github.com/marcotcr/lime/pull/132 I did not realize people were sometimes passing in numpy arrays for categorical_features and feature_names. While that's not technically following the documentation (both args ask for lists), I don't want to break people's workflows, so this brings back the more defensive behavior that supports lists or numpy arrays.
I added more tests to make sure this behavior is preserved!

thanks

I received an error passing a numpy array as categorical_features names in the <IP_ADDRESS> version for Tabular LIME. When I changed the array to python lists, things worked just fine.


HHH-15022 Test deleting all entities of a given type in a table-per-class hierarchy with a default schema

https://hibernate.atlassian.net/browse/HHH-15022
It turns out the problem only affects 5.6, not 6.0, but I figured I would still add a non-regression test in 6.0

Merging, since tests pass and  #4659 was approved.


Where is the ensembling of multi-scale results done ?

Hello,
I have a model that is being executed for inference on a GPU that does not have enough memory to handle the full multi-scale augmentation batch I am giving it. I was thinking it might work if I split the full multi-scale batch into sub-batches and then ensemble the results after the inference has been complete.
However, I have been unable to find where the ensembling of the results for multi-scale val is done so am not sure how the results are ensembled. Could someone point me in the right direction please ?

Nevermind, I figured it out.

Never mind, I figured it out.


Release Canvas Drawing

@shama Would you mind creating a new NPM release with #12 ?
Thanks for the tool. Great work.

Done! Thanks for the ping.


manifest: update hal_xtensa

Update hal_xtensa for NXP SoC overlay support.
Signed-off-by: Iuliana Prodan<EMAIL_ADDRESS>

Can somebody, please, merge this - is pending for a week now.
cc: @nashif @MaureenHelm @lgirdwood @dbaluta @dcpleung @dleach02


Update spark config defaults

@tnachen PTAL and confirm formatting for the defaults (then merge!)

:+1:

Please run scripts/build.sh to keep the index up to date.

@ConnorDoyle I did, but it didn't change anything. There weren't any required configs yet anyway, so maybe the index stayed the same?

@adam-mesos Hey sorry didn't see this PR. I already did the change and merged it. It did rebuilt the index for me. I'm closing this PR.


[Bug] Incorrect warning message from import_argparse_flags_main.py

I get this error when I try to run an operation specified with action='store_true' flag, e.g. main: main is fine bug main: main --verbose gives this error.
WARNING: cannot import flags from [my module]: usage: import_argparse_flags_main.py [-h]
                                     mod_path package base_args output_path
import_argparse_flags_main.py: error: the following arguments are required: output_path

My feeling is that import_argparse_flags_main.py:61 is written without the output_path arguement added at line:69

Thanks for the report @alabamagan - this is fixed in master and will be available in the next release.


Refreshing the page can delete checkmarks in quizzes

Describe the bug
When refreshing the page (with the Browser, e.g. by pressing F5) immediately after checking a checkbox in a quiz, the new check mark is gone on the refreshed page.
This issue might be related to Firefox. It did not occur on Chrome.
To Reproduce

Start a quiz in an exam.
Check a checkbox and immediately refresh the page (e.g. press F5).
After confirming the message on the start page again and navigating to the quiz, the new check mark is missing.

Expected behavior
Refreshing the page should automatically save all current input, so it will be there after the refresh.
Environment
Firefox on MacOS

It takes a few ms until the changes are sent to the server. I consider this as an edge case that does not happen in reality. I also don't think we can easily solve this issue.


freee access for lan and not for others

Attempted Debugging

[X] I have read the debugging page

Searched GitHub Issues

[X] I have searched GitHub for the issue.

Describe the Scenario
hi,
i haven't found the answer : is it possible to create a scenario where a specific ip range can access publicly the documentation, while others must login ?
Exact BookStack Version
last
Log Content
No response
Hosting Environment
docker

Hi @boscorelly,
No, that's not something which is supported at all.
It could be possible to hack on an instance in various ways using the logical theme system, but I'm hesitant to provide further guidance on that as it's easy to result in an insecure setup when it comes to customizations around authentication.

Hi,The aim is to let users freely access the docs when at the office, and force them to login through office 365 (where passwordless mfa is mandatory) when they are roaming

Envoyé à partir de Outlook pour iOS

De : Dan Brown @.>Envoyé : mardi, décembre 31, 2024 4:47 PMÀ : BookStackApp/BookStack @.>Cc : Camille Ollié @.>; Mention @.>Objet : Re: [BookStackApp/BookStack] freee access for lan and not for others (Issue #5377)
Hi @boscorelly,
No, that's not something which is supported at all.
It could be possible to hack on an instance in various ways using the logical theme system, but I'm hesitant to provide further guidance on that as it's easy to result in an insecure setup when it comes to customizations around authentication.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>
Envoyé à partir de Outlook pour iOS


Conversion rules for numpy [and ctypes] scalars

These are already in src/old/cpython/CPython.jl, just need migrating.

Actually ctypes scalars are mutable and expose the buffer interface, so just keep these as-is.

Done a while ago!


rowspan & colspan

STOP
We do not track feature requests through Github. Please use the Spectrum React Table Forum to talk about new ideas and features.

This will be possible in v7

Hi there! It looks as though you've submitted an issue, feature request, or question regarding v6 of React Table, which I am no longer maintaining or supporting. I invite you to try the latest alpha release of v7 and help me push towards a v7 release soon. Thankyou!
If you still wish to discuss v6, please use https://spectrum.chat/react-table.

Is colspan and rowspan already included in v7? what's the status?

colspan has been implemented as a render prop only for column headers. Support for colspan and rowspan in cells is coming soon.

is colspan / rowspan support merge cell as well?

hey, could this be re-opened to track implementing cell-level row/col-span?
Cheers

Please open a new issue and we’ll track it there.
Tanner Linsley
On Dec 2, 2019, 11:11 PM -0700, Jarrad Whitaker<EMAIL_ADDRESS>wrote:

hey, could this be re-opened to track implementing cell-level row/col-span?
Cheers
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.


onSelect() being executed 2 times.

Hi!
I updated the library to: @1.7.2 also together with React16. I've started to experience some strange issue that the onSelect()is being call also when I didn't select it.
Does anyone have the same issue? Thanks a lot,
Juan

Hi Juan! You'll have to provide some more information if we're going to be able to debug the problem. Can you create a reduced test case in a gist and/or jsfiddle?

Hi @CMTegner ... Sorry about my delay and thanks for your help.
I've just created an isolated test on this JSBIN: http://jsbin.com/kicocuj/2/edit?html,js,console,output.
And realized that work ok using React 16 and latest @1.7.2 version of react-autocomplete.
That's why checked and we had an issue with our code.
Thanks a lot for your help, and sorry for that.
Juan


Saving molfile doesn't expand monomers

This is fixed in 2016-10-11 update: saving molfile options is removed.

Closed.


Error Code: 4D78F526

Hi! What do I do wrong ? Can't connect , says no drivers , but I did everything as it was said.

This one? https://github.com/ClickHouse/clickhouse-java/releases/download/v0.4.4/clickhouse-jdbc-0.4.4-http.jar
Which directory is it in?

Yes
There is a directory in the screenshot
~Library/Tableau/Drivers

The ~/Library/Tableau/Drivers does exist when Tableau Desktop is installed.
A tilde ~ is used to represent the user's "home folder",
thus the absolute path would be
/Users/$USER/Library/Tableau/Drivers

Thanks
Works


Proposal: Improved UI for ink to text

Proposal: Improved UI for ink to text

Summary
Currently, the ink to text flyout looks a out of date, and is unintuitive to use. For example, it is hard to select suggestions, and there isn't a convenient way to undo. It is also annoying to have the text jump when it is immediately converted after writing. Recently, the Microsoft Journal app was released, which includes new (and in my opinion better styles) for ink-to-handwriting. I think it would be nice if this style, or a similar one, was included in WinUI 2.x.
Current:

Journal style ink-to-text:

Improvements over current version:

undo, redo, and clear
eraser tool
output preview instead of immediate conversion from ink to text
inline suggestions

Rationale

This will help improve the inking experience on windows to make it more intuitive

Scope

Capability
Priority

This proposal will update the ink to text ui to match new winui styles
Must

This proposal will place the word suggestions inline, instead of showing them in a separate flyout to make it easier to use
Must

This proposal will add undo, redo, and clear buttons
Should

This proposal will show a preview of the output, instead of converting it immediately
Should

This proposal will add an eraser tool for handwriting
Could

This proposal will improve the ink to text recognition
Won't

Important Notes

Journal app: https://www.microsoft.com/store/productId/9N318R854RHH
Open Questions

Should ink to text be in it's own flyout, or should it be inline?

@chigy fyi

Was wondering whether there are any updates on this?


"Already imported a file that will be measured" reported for coverage itself

Describe the bug
Admittedly, my configuration is probably ~slightly wrong -- but since this is a new message in coverage 5.x I figured I'd point it out.  The warning seems non-actionable for this particular case.
To Reproduce
How can we reproduce the problem?  Please be specific.
This .coveragerc (distilled from a larger example)
[run]
source = .

coverage installed into ./venv
$ ./venv/bin/pip freeze
coverage==5.0

any old simple python file, here's t.py
print('hello world')

run coverage run -m t
$ coverage run -m t
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/bin/coverage (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/__init__.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/version.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/control.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/env.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/annotate.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/files.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/backward.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/misc.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/report.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/collector.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/debug.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/disposition.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/pytracer.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/tracer.cpython-36m-x86_64-linux-gnu.so (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/config.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/tomlconfig.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/context.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/data.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/sqldata.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/numbits.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/html.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/results.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/templite.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/inorout.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/python.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/parser.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/bytecode.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/phystokens.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/plugin.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/jsonreport.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/plugin_support.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/summary.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/xmlreport.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/multiproc.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/cmdline.py (already-imported)
Coverage.py warning: Already imported a file that will be measured: /tmp/x/venv/lib/python3.6/site-packages/coverage/execfile.py (already-imported)
hello world

Expected behavior
the warnings are pretty noisy, and in my case not really actionable (this is a development env, I'll later run from tox where omit will exclude these particular files).
I realize I probably should add omit = venv*/* or something of the sort
Additional context
nothing really, just new noise :)
thanks again for coverage, I don't know what I'd do without it

Perhaps this is a variant of #876, that coverage should be smarter about ignoring local venv directories.

Wait, somehow i missed that this was about coverage itself...!

yep! and bin/coverage (sys.modules['__main__'] and site.py (which I assume is due to virtualenv shenanigans and doesn't get covered by the stdlib filter?)

(Looks like something is missing from your comment?)

This is similar to #876: when using --source=., the venv is part of that source tree, so all of venv is considered measurable.
You can fix it by adding this to .coveragerc:
[run]
omit = venv/*

BTW, I don't think this is a slightly wrong configuration, I think it's really common, unless I missed something?

--omit=venv/* worked for me, thanks nedbat.

Any update on this?

+1

@piranna @tlrh314 The solution is above.

This is now fixed in 0285af966a
Would you mind testing it out?
pip install<EMAIL_ADDRESS>

soooo close, most of the console spam is gone, just one thing remains:
$ coverage run -m t
Coverage.py warning: Already imported a file that will be measured: /tmp/q/venv/bin/coverage (already-imported)
hello hello world

Ugh, newb mistake not running the actual example from the bug report!  It's fixed now in 5c2f614e

awwww yiss -- thanks a ton!
$ coverage run -m t
hello hello world

This is now released as part of coverage 6.0.


Discussion: Handling Timeouts more gracefully

Background
The most likely request to ever run into a timeout is a batch request as that is basically a collection of import requests. With some request of the data-classification (cc @michaverhagen, @antas-marcin) repo we have seen that a batch size of 1000 can lead to timeouts depending on the data set. (Note that "batch size" in this case refers to the number of objects in a POST v1/batch/objects request. Not to be confused with the classification batching which the repo does).
If such a timeout occurs, the reason for the timeout (which in my opinion is the batch size which is too large) is very opaque to the user. I think we should achieve the following goals. I don't know how feasible they are in the client, hence this discussion.
Goals
1. Handle gracefully, make user aware
The current output when a timeout occurs looks something like this. Note that I have shortened the snippet to remove potentially sensitive data:
.... shortened to remove sensitive stuff ....
 File "/usr/local/lib/python3.9/site-packages/weaviate/batch/crud_batch.py", line 102, in create_objects
    return self.create(
  File "/usr/local/lib/python3.9/site-packages/weaviate/batch/crud_batch.py", line 64, in create
    response = self._connection.run_rest(
  File "/usr/local/lib/python3.9/site-packages/weaviate/connect/connection.py", line 221, in run_rest
    response = requests.post(
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='REMOVED.semi.network', port=443): Read timed out. (read timeout=20)

Technically, all of the info is present in the above. However, it's very hard to find the correct info and there is no suggestion how the user can fix it. I think it would be nicer if a clear error message (if possible without any kind of stack trace?) is printed such as
The request was cancelled because it took longer than the configured timeout of 20s. Try reducing the load or increasing the timeout if the load cannot be reduced any further.

Not sure if we should encourage the user to try to increase the timeout. Long-running HTTP requests are problematic, because at some point we will run into hard-coded (or default) limits in load balancers and proxies. From my experience these are usually around the 1min mark. Since the default is already 20s - which is a lot - I don't think we should encourage the user to just increase the timeout further, which brings me to the next point
2. Print current batch size and suggest reducing it
I have no idea if this is possible in the client at all, but it would be really nice if we could catch the fact that the timed out request was a batch request and make the user aware, such as
The batch request was cancelled because it took longer than the configured timeout of 20s. Try reducing the batch size (currently 1000) to a lower value. Aim to on average complete batch request within less than half (10s) of the currently configured timeout (20s).

The most important part is about reducing the batch size. The second sentence encourages the user the think about monitoring their requests. The idea behind that is that you could be in a very dangerous situation where your request take on average 19s. Then you change a tiny thing (maybe add slightly longer descriptions) and suddenly all requests run into timeouts - seemingly without changing anything considerable.
Feedback
Happy to hear your feedback on the UX suggestions and the feasibility of implementing both 1 and 2 in the client.
I think we should also implement something like this in the other clients, but given that the python client receives 99% of the usage we're aware of at the moment, it's where we should start :)

I like the suggestion to encourage the user to reduce the batch size. I have already incorporated batch size in much of the code, as (I understand) there are currently hard limits to batch size. If we want to abstract from the data that customers use (i.e. not knowing the amount of data), we should always include a parameter that limits the batch size for all actions (creation / cross referencing / classification).

I like the suggestion to encourage the user to reduce the batch size. I have already incorporated batch size in much of the code, as (I understand) there are currently hard limits to batch size. If we want to abstract from the data that customers use (i.e. not knowing the amount of data), we should always include a parameter that limits the batch size for all actions (creation / cross referencing / classification).

It should be possible to implement it very easily in python by catching the requests.exceptions.ReadTimeout. I will do it for the Batch and for the Batcher too.

It should be possible to implement it very easily in python by catching the requests.exceptions.ReadTimeout. I will do it for the Batch and for the Batcher too.

How can we achieve it with javascript?

@Chirag-cog afaik this is not implemented yet, but feel free to create an issue in the TS repository


Fix flakiness of migrate test

This test became flaky when the default write type became async-through. This is because when the migrate is configured to delete the source, sometimes, that inode is NOT persisted, so it is not removed from the UFS. Then, on the exists call, it will loadMetadata from UFS, and it will load the file from UFS and the deleted file will "return".

alluxio-bot, merge this please