Creating a database automatically upon startup

A couple of comments (4) in the DockerHub page suggest adding parameters to create a database with username and password automatically.

+1

is there a timeline for this. Also the missing tools in this container/image (see #8 ) make this hard to achieve for me on my own.

+1

FYI - We released CTP 1.4 today.  This release of the mssql-server-linux image now includes the mssql-tools package (sqlcmd and bcp) in it.
Executing sqlcmd as part of the entrypoint.sh script can be used for this kind of scenario for now.  Since this is such a commonplace requirement we want to make it easier in the future, but sqlcmd will provide a reasonable option until then.

How would you use that with docker-compose
That is what i've got now:

  sqlserver:
    container_name: 'sqlserver'
    image: microsoft/mssql-server-linux:2017-CU1
    environment:
      - ACCEPT_EULA=Y
      - MSSQL_PID=Developer
      - SA_PASSWORD=<pwd>
      - MSSQL_DB=mydb (was expected db to be created)
      - MSSQL_USER=me
      - MSSQL_PASSWORD=<pwd>
    ports:
      - "1433:1433"
    networks:
      - app-network

hi @kuncevic did this docker compose work for you? did you get a db and user on the container?

Creating databases and users at container run time is not implemented yet.

@pjpradeepjha with the method i described above - yes.

@kuncevic @twright-msft  thanks for the comments. appreciate the help. :)  I was constantly trying to create user and db from docker compose on the mssql docker image to no effect.

@kuncevic - We will continue to make the SQL image container smaller.  Just takes some time to minify something like SQL Server as you can imagine. :)

:) Thanks a lot @kuncevic

Bump. Both postgres and mysql images already support using environment variable to create an initial database when the image is run and the container is created (POSTGRES_DB: "mydb" or MYSQL_DATABASE). Would be great if this were also supported in the official mssql image, otherwise we need to rely on executing sqlcmd to create the db on container startup.

Any updates? Its been in the backlog for over 2 years. Doesnt seem like that big of a request. If we cant create one based on ENV is there a default DB created when the container boots up?

The naming convention in these env vars is inconsistent with industry standard database connectors.
MYSQL_DATABASE here actually refers to the database Instance in connector drivers, not the Database.
For example, MYSQL_DATABASE treated as an Instance variable would have users connect with the following (which is currently what it does):
sqlserver://localhost\Foo:1433
Where as MYSQL_DATABASE as a Database variable would have users connect with:
sqlserver://localhost:1433;database=Foo, which is currently does not.
I think this is why some folks have connection issues and others don't. It would probably be best to support both.

It is possible, here you are examples: https://github.com/microsoft/sql-server-samples/tree/master/samples/containers
And also my example, IMHO much more straight forward: https://github.com/lkurzyniec/netcore-boilerplate

How do we know sleep 30s will be the right delay ? I have been trying hacks like this, but nothing works...
Support for database creation is still needed.

@damien-git check this sample out: https://github.com/microsoft/mssql-docker/tree/master/linux/preview/examples/mssql-customize
this will ping the server, and if it is ready, it will run the startup script, setup.sql. This is where you want to put your create DB script

Sorry @vin-yu, but suggesting to create our own Docker image just so we can create a DB upon startup is not satisfiabile solution.
For me the idea to create a DB upon startup is because I want to use your own Docker image in our CI. I don't want to build a new image everytime you release a new image.
Would really be interested to know what the problem is to add such support to the official docker image? Why not move that example here and parameterise it?

I totally agree with @filiphr . Also the fact that basically EVERY other db container offers such a feature should be an indication that this is a VERY useful thing to have.

@vin-yu : thanks for the example, but it does not work for me. In configure-db.sh, ERRCODE is 1 before the server is done starting up, so it fails unless I add a sleep statement before (which makes the whole code useless). Also, there are several syntax errors on line 19.

@damien-git I am aware that the previous "solution" is neither complete nor appropriate, but after tweaking the configure-db.sh, I could (partly) replicate the desired behavior:
#!/bin/bash

TIMEOUT=60
DBSTATUS=1
ERRCODE=1
i=0

while [[ $i -lt $TIMEOUT ]] ; do
	i=$i+1
	DBSTATUS=$(/opt/mssql-tools/bin/sqlcmd -h -1 -t 1 -U sa -P $SA_PASSWORD -Q "SET NOCOUNT ON; Select SUM(state) from sys.databases")
	ERRCODE=$?
	sleep 1

	if [[ $DBSTATUS -eq 0 ]] && [[ $ERRCODE -eq 0 ]]; then
		break
	fi
done

if [[ $DBSTATUS -ne 0 ]] || [[ $ERRCODE -ne 0 ]]; then
	echo "SETUP: SQL Server took more than $TIMEOUT seconds to start up or one or more databases are not in an ONLINE state"
	exit 1
fi

sleep 2

# Run the setup script to create the DB and the schema in the DB
/opt/mssql-tools/bin/sqlcmd -S db -U sa -P $SA_PASSWORD -i /usr/config/setup.sql

I ended up making my own hack. It works in my case, for now, but that is still not satisfying...
#!/bin/bash
# (see https://github.com/microsoft/mssql-docker/issues/2 )
echo "Container initialization: waiting for the server to come up"
while [ ! -f /var/opt/mssql/log/errorlog ]
do
  sleep 1
done
FOUND=0
i=0
while [[ $FOUND -ne 1 ]] && [[ $i -lt 60 ]]; do
  i=$i+1
  FOUND=$(grep -cim1 "Service Broker manager has started" /var/opt/mssql/log/errorlog)
  if [[ $FOUND -ne 1 ]]; then
    sleep 1
  fi
done
if [[ $FOUND -ne 1 ]]; then
  echo "Container initialization: Error: waited for more than 60 seconds for the server to start. Trying to create the database now..."
fi

echo "Container initialization: creating the database if needed"
/opt/mssql-tools/bin/sqlcmd etc...
echo "Container initialization: done"

this shouldn't be a closed issue
are there no plans to implement this extremely useful feature?
if the benefits are not immediately apparent then I would be happy to help explain them

@twright-msft

I recently came into a situation where I had no choice but to use mssql server, and I am honestly completely baffled at how cumbersome it is to use this docker image. Must have feature imo!

I'm a bit baffled to come across this three-and-a-half-year-old closed issue for such an extremely obvious and expected feature that is supported by every other major RDBMS Docker image.
@vin-yu do you think you could explain again why you closed this issue? I realize it may have been accidental, since the "Comment" and "Close and comment" buttons are right next to each other.
I would join @agates4 in helping justify the feature, if it's still unclear.

@agates4 @patricklucas - Thank you for the feedback. We don't have plans to implement this in the near future, and might have closed this by accident. Reopening but please refer to this work around for now: https://github.com/microsoft/mssql-docker/tree/master/linux/preview/examples/mssql-customize
@damien-git - I'll look into why this doesn't work but got it working before we posted this.
There are multiple workarounds to create a database or run custom .sql scripts post start-up so we are focused on other container improvements/products at the moment.
Thank you.

Thanks for the reply @vin-yu. I think the main concern was that the issue was closed without a clear resolution, so I appreciate your quick response and reopening it.

@vin-yu I find it extremely baffling that you don't consider this a priority and that you have no plan to implement it in the near future.
All other database vendors (MySQL, Postgres, MariaDB, etc.) offer this feature out-of-the-box. Why? Because it significantly reduce both the time it takes to use it and the learning curve. I shouldn't have to search on google for 30 minutes on how to automatically create a database.
Frankly, I doubt any other container improvements/products you could come up with would be as beneficial as this one, and by a very huge margin. The amount of upvotes on this ticket should be a clear sign.
The very first comment I get from other devs trying to use the mssql docker image the first time is always the same: Why can't I quickly create a database, like the other DB vendors? It doesn't make any sense to ask your users to go customize a startup script and build a new image.
Unless your goal is not to promote the usage of SQL Server, this very basic feature should be added. There's a reason SQL Server isn't the go to database for most users, adding unnecessary barriers to its use is probably the biggest reason.

If anybody else had problems with the approaches here, this worked for me:
echo "Starting Sql Server"
( /opt/mssql/bin/sqlservr & ) | grep -q "Service Broker manager has started"
&& /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P $SA_PASSWORD -d master -i setup.sql
&& sleep infinity

For those using entityframework with asp.net core
I'm doing like this right only and I think this may help someone
Calling this before (EnsureCreatedAsync or MigrateAsync) will make sure that you have a database to work on that also accept your password
private static async Task InitDatabase(IConfiguration config)
        {
            string hostServer = config["HOST_SERVER"] ?? "(localdb)\\MSSQLLocalDB";
            string serverPort = config["HOST_PORT"] ?? "1433";
            string databaseName = config["DATABASE_NAME"] ?? "testdb";
            string userName = config["USERNAME"];
            string passward = config["SA_PASSWORD"];
            string connectionString;
            if (!string.IsNullOrEmpty(userName) && !string.IsNullOrEmpty(passward))
            {
                connectionString = $"Server={hostServer};User Id={userName};Password={passward};";
                var optionsBuilder = new DbContextOptionsBuilder<DataAccess>();
                optionsBuilder.UseSqlServer(connectionString);
                using (var dbContext = new DataAccess(optionsBuilder.Options))
                {
                    var query = @$"IF DB_ID('{databaseName}') IS NOT NULL
set noexec on
CREATE DATABASE [{databaseName}];";
                    await dbContext.Database.ExecuteSqlRawAsync(query);
                }
            }
        }

Using docker-compose, this works for me:
version: "3"
services:
        sql:
                image: "mcr.microsoft.com/mssql/server"
                container_name: sql
                ports:
                        - "1433:1433"
                environment:
                        - ACCEPT_EULA=Y
                        - MSSQL_PID=Enterprise
                        - SA_PASSWORD=Password1
                command: /bin/sh -c "(/opt/mssql/bin/sqlservr &) && sleep 10s && /opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P Password1 -d master -Q 'CREATE DATABASE [MyDatabaseName]' && sleep infinity"

Chiming in here from our conversation on Twitter. I think the main use case for Docker here, at least where this matters, is people want to spin this up as a service. For me and my teams, we are generally doing this in Actions or locally. For things like integration tests it's a primary use case - you're using Docker in the first place because you want to quickly spin it up and run. One of the critical components in doing this is to have a database.
We have developers, who may or may not have experience creating a database, but that shouldn't be a barrier to entry here. With an environmental-variable-based approach, we can lower that barrier. Please consider that even though this issue has workarounds with scripting to workaround this issue, some amount of users will give up, never having made it to this issue, or far enough through it for a working solution. Given the volume of input we have here, I'd suggest this be given some priority, since adoption is greatly hindered for any technology if users hit large barriers early.
If it helps, our main use cases are integration tests (e.g. for libraries like MiniProfiler, StackExchange.Exceptional, and Dapper), and also less public ones like internal software (also integration tests), or per-PR build environments we spin up services in for some time period.
Here's an example GitHub Actions config from a project spinning up services for some integration tests:
    services:
      mongo:
        image: mongo
        ports:
        - 27017/tcp
        env:
          MONGO_INITDB_DATABASE: test
      postgres:
        image: postgres
        ports:
        - 5432/tcp
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test
      sqlserver:
        image: mcr.microsoft.com/mssql/server:2019-latest
        ports:
        - 1433/tcp
        env:
          ACCEPT_EULA: Y
          SA_PASSWORD: Password1!
      mysql:
        image: mysql
        ports:
        - 3306/tcp
        env:
          MYSQL_ROOT_PASSWORD: root
          MYSQL_DATABASE: test

Note that  MySQL (and MariaDB) use MYSQL_DATABASE and PostgreSQL uses POSTGRES_DB. When we're consuming SQL Server, we're either scripting this out manually, or resorting to abusing tempdb overall.

In addition to @NickCraver comment, postgresql also consider files present in /docker-entrypoint-initdb.d/ directory (in the container). It helps to create tables and fill them when container is created.
Exemple :
services:
  pg:
    image: postgres
    ports:
    - 5432/tcp
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: test
    volumes:
    - ./pg.sql:/docker-entrypoint-initdb.d/init.sql

init.sql is not a keyword, multiple files can exist in this directory (postgres execute them in alphabetical order). Without this feature, an external script would be required to create tables and insert data.

Any plans to change this? Auto DB creation on startup is the standard for every other database container.

Come on @vin-yu, @twright-msft , how much longer are you guys going to ignore this very important issue? It's literally the single most requested feature and should be very quick to implement on your side.

DB Engine
Has init DB variables?

DB2
:white_check_mark:

InfluxDB
:white_check_mark:

MariaDB
:white_check_mark:

MongoDB
:white_check_mark:

MS SQL Server
:x:

MySQL
:white_check_mark:

Neo4j
:white_check_mark:

PostgreSQL
:white_check_mark:

If this feature is ever tackled, please add support for init.d directory to the entrypoint script.
i.e. run all shell / sql scripts in the init.d directory in order once the DB is up and ready to receive commands.

Thank you all for the suggestions and feedback. I am working on triaging this internally and will update this thread on the next action we take. Until then the options mentioned by @vin-yu is something that can be used as a workaround. While we work on this feedback internally.

I'm chipping in with a slightly modified entrypoint.sh which plays nicely with wait-for-it, dockerize, etc. The idea is to only make the server available to other containers once the init/setup scripts ran.
#!/bin/bash

_setup() {
  for _ in {1..60}; do
    if /opt/mssql-tools/bin/sqlcmd -S <IP_ADDRESS> -U sa -P "$SA_PASSWORD" -d master -i setup.sql; then
      break
    fi
    echo "Server is not ready..."
    sleep 1
  done
}

echo "Starting SQL-Server on <IP_ADDRESS>"
/opt/mssql/bin/mssql-conf set network.ipaddress <IP_ADDRESS>
/opt/mssql/bin/sqlservr &
_setup

echo "Setup finished. Stopping SQL-Server..."
pid=$(pgrep -o sqlservr)
kill "$pid"
tail --pid="$pid" -f /dev/null

echo "Starting SQL-Server on <IP_ADDRESS>"
/opt/mssql/bin/mssql-conf set network.ipaddress <IP_ADDRESS>
/opt/mssql/bin/sqlservr

Other containers then can simply dockerize -wait tcp://db:1433 ... for example and be sure that the database is only available once it's fully initialised.

@amvin87 its been a couple more months now and still no update, what is the status? i'm likewise amazed that this is still an open issue after almost four years...
what could possibly have a greater cost/benefit than this?
You can put "microsoft heart linux/containers/docker" on slides all day long but not supporting basic features like this what really tells the story :/ Dont ruin the good will dotnet is creating and make it that much harder for us to convince people Microsoft is not just full of hot air when it comes to this stuff

@amvin87 I don't understand why you going silent. A small feedback will be appreciated and a basic feature implementation will be honoured.

@amvin87 I don't understand why you going silent. A small feedback will be appreciated and a basic feature implementation will be honoured. Thnx in advance!

Hello @amvin87, do you have any updates or a timeline that you can share?
Thank you 🙇🏼

Honestly find it quite shocking that this isn't considered an obvious feature.

@twright-msft Is the TL;DR on this still that your images do not have a good story for spinning up a DB without needing to dig into a pile of version-specific scripting?
Literally every other DB has a story for this -- it's 2022 after all
I'm doing pretty unexotic testing; the typical prod database this runs against is an Azure SQL instance.
Can someone be assigned this please, or do you have a good one liner that explains why there's a need to differentiate here? Or is this something the Azure SQL team should be tasked with?
It makes it very hard to include MSSQL in a test matrix alongside sensible children like PG and Mysql as-is, and has thoroughly wasted my Saturday afternoon updating a test rig for an OSS project (with a lot of curse words along the way)

It's been 6 years and still not done? come on Microsoft

Any sort of update on this?
@amvin87

I am eager for an update as well 🙏🏼

Give this a look... just dumped it from another project, and minor tweaks... completely untested in current state.
https://github.com/tracker1/mssql-docker-enhanced

@Schlurcher NB, the last command in an entrypoint script should always be executed thourgh exec, e.g.
...
# NB NB The last (long running) command should always be started through exec, see
# https://stackoverflow.com/questions/39082768/what-does-set-e-and-exec-do-for-docker-entrypoint-scripts and
# https://stackoverflow.com/questions/32255814/what-purpose-does-using-exec-in-docker-entrypoint-scripts-serve
# for details.
exec /opt/mssql/bin/sqlservr

I generalized the script to support a generic /docker-entrypoint-initdb.d directory instead of a hardcoded setup.sql and also used it as CMD replacement instead of ENTRYPOINT replacement. The docker-compose.yml file then needs the following change:
   db-server:
     image: mcr.microsoft.com/mssql/server:2017-latest
     ...
+    command: [ "/cmd_wrapper.sh" ]
     volumes:
       ...
+      - ./volume/db-server/cmd_wrapper.sh:/cmd_wrapper.sh
+      - ./volume/db-server/docker-entrypoint-initdb.d/:/docker-entrypoint-initdb.d/

with cmd_wrapper.sh being the following:
#!/bin/bash

function finish() {
        local _ret=$?
        if [[ _ret -ne 0 ]]
        then
                echo "========================================================================"
                echo "$0: ERROR: Some command exited with exit code $_ret"
                echo "========================================================================"
        fi
}
trap finish EXIT

set -euo pipefail

# /opt/mssql/bin/sqlservr ought to match the CMD from the upstream docker image, e.g.
#
# $ docker inspect mcr.microsoft.com/mssql/server:2017-latest | jq '.[0].Config.Cmd'
# [
#   "/opt/mssql/bin/sqlservr"
# ]
# $

function wait_for_server_ready() {
        local rows_affected=0

        # Sql server has 4 system database, so when they are connected and
        # created, SQL server can be assumed to be running and working as
        # expected. Example output from sqlcmd:

        #name
        #--------------------------------------------------------------------------------------------------------------------------------
        #master
        #tempdb
        #model
        #msdb
        #some_other_database_001
        #some_other_database_002
        #some_other_database_003
        #some_other_database_004
        #some_other_database_005
        #
        #(9 rows affected)
        while [[ rows_affected -lt 4 ]]
        do
                echo "Waiting for sys.databases query to return 4 or more rows"
                sleep 1
                rows_affected=$(/opt/mssql-tools/bin/sqlcmd -S <IP_ADDRESS> -U sa -P "${MSSQL_SA_PASSWORD}" -Q 'SELECT name FROM sys.databases' 2>/dev/null | sed -n 's/(//; s/ rows affected.*//p;')
        done
}

function process_init_scripts() {
        local old_pwd=$PWD
        for dir in "$@"
        do
                cd "$dir" || { echo "$0; Error: unable to cd to $dir" 1>&2; exit 1;}
                for file in *.sql
                do
                        case "$file" in
                                "*.sql")
                                        echo "No *.sql files in $dir."
                                        ;;
                                *)
                                        echo "Processing $file"
                                        /opt/mssql-tools/bin/sqlcmd -S <IP_ADDRESS> -U sa -P "${MSSQL_SA_PASSWORD}" -d master -i "$file"
                                        sync
                                        ;;
                        esac
                done
        done
        cd "$old_pwd"
}

echo "Starting SQL-Server on <IP_ADDRESS>"
# Ignoring output from mssql-conf since it will complain
#   SQL Server needs to be restarted in order to apply this setting. Please run
#   'systemctl restart mssql-server.service'.
# which is not an issue here in this script.
/opt/mssql/bin/mssql-conf set network.ipaddress <IP_ADDRESS> > /dev/null
/opt/mssql/bin/sqlservr &
pid=$!

wait_for_server_ready
echo "========================================================================"
echo "Database ready for initialization"
echo "========================================================================"
process_init_scripts /docker-entrypoint-initdb.d

sync

echo "Setup finished. Stopping SQL-Server..."
kill "$pid"
wait "$pid"

sync
sleep 1

echo "Starting SQL-Server on <IP_ADDRESS>"
/opt/mssql/bin/mssql-conf set network.ipaddress <IP_ADDRESS> > /dev/null

# NB NB The last (long running) command should always be started through exec, see
# https://stackoverflow.com/questions/39082768/what-does-set-e-and-exec-do-for-docker-entrypoint-scripts and
# https://stackoverflow.com/questions/32255814/what-purpose-does-using-exec-in-docker-entrypoint-scripts-serve
# for details.
exec /opt/mssql/bin/sqlservr

Still no updates? This is turning into an irony now!

Does:
apt-get install sqlcmd
sqlcmd create mssql --accept-eula --user-database foo
sqlcmd query "SELECT DB_NAME()"

Help with this scenario?
https://aka.ms/sqlcmd

@stuartpa The point is that every other similar image has a way to request the creation of a database in a declarative manner.
It's the need to have a separated step (and being able to trust that it's been done) that's the problem.
Nobody is suggesting that the individual step is in any way hard (though your wiring happens to be OS-specific).

Just saying that your specific scripting will work, but assumes a specific environment (i.e. if the container simply did it, one would not need to have something that has apt-get and the associated permissions in the context of how one might be spinning things up in the consumption scenario)
Perhaps you can flesh out what you're hoping to accomplish with your suggestion? e.g. in my suggestion, I have a command that can can accomplish similar, leaning on the container's contents - but I am calling out that this is pretty useless as it is not integrated, and as a result I have to maintain special case documentation

wiring happens to be OS-specific).

I'm not tracking what is OS-specific, can you flesh this out a bit more.

The issue is I don't want to script it out, I just want to pass a file for the container to understand. I can script it out, but I can write assembly too and don't want to.
Every other DB container has this feature. It's entirely possible.
MS simply doesn't care.

I don't have a stake in one direction or another... but the container really should have the options to use/specify environment variables to create a database with a non-sa user that has full access only for that database.  This would align closer to other Database containers.

A workaround using docker-compose :
version: "3"
services:
  db:
    image: mcr.microsoft.com/mssql/server:2022-latest
    ports:
      - "1433:1433"
    environment:
      ACCEPT_EULA: y
      MSSQL_SA_PASSWORD: 01pass_WORD
  db-init:
    image: mcr.microsoft.com/mssql/server:2022-latest
    container_name: diaconat-sqlserver-init
    network_mode: service:db
    command: bash -c 'until /opt/mssql-tools/bin/sqlcmd -U sa -P 01pass_WORD -Q "CREATE DATABASE mydb"; do echo retry && sleep 1; done'
    depends_on:
      - db

Here we do not use an arbitrary sleep timeout, we loop until the sql command ends up in success (and wait 1s between retries).

Just another version of what @vbueb wrote that waits for db to properly start and executes all sql scripts in certain directory. It also exits because I believe that db-init contain does not need to keep running after db has been setup:
  db:
    image: mcr.microsoft.com/mssql/server:2022-latest
    restart: always
    ports:
      - "1433:1433"
    environment:
      ACCEPT_EULA: y
      SA_PASSWORD: $SA_PASSWORD
    healthcheck:
      test:
        [
          "CMD",
          "/opt/mssql-tools/bin/sqlcmd",
          "-U",
          "sa",
          "-P",
          "$SA_PASSWORD",
          "-Q",
          "SELECT 1",
        ]
      interval: 10s
      retries: 5
      start_period: 20s
  db-init:
    image: mcr.microsoft.com/mssql/server:2022-latest
    network_mode: service:db
    command: >
      bash -c '
        for file in ./sql-scripts/*.sql; do
          echo "Executing SQL file $$file"
          until /opt/mssql-tools/bin/sqlcmd -U sa -P $SA_PASSWORD -i "$$file" > /dev/null; do
            echo "Retry SQL execution for $$file"
            sleep 1
          done
        done
        exit 0
      '
    volumes:
      - ./sql-scripts:/sql-scripts
    depends_on:
      db:
        condition: service_healthy

Setup above executes all scripts inside sql-scripts directory after sqlserver becomes active.
If you want to execute another service after db init has finished a simple setup like this worked fine for me:
  api:
    build:
      context: ../
      dockerfile: ./someapi/Dockerfile
    ports:
      - "8080:80"
    environment:
      ConnectionStrings:MyDatabase: "Server=db;Database=MyDatabase;User Id=sa;Password=$SA_PASSWORD;"
    depends_on:
      db-init:
        condition: service_completed_successfully


protocol is currently bottable

One issue I see w/ the protocol as it is, is that there is no way for a deployer to control who mints it.
That is nice for free-for-all collections and the free market ethos of crypto but it does limit the usecases orc-721 can support.
Example: Lets say I want to do an orc-721 collection. I spend a lot of time creating the art and I want everyone to have a fair shot at minting.
I first inscribe the sprite file
Then the deploy json with a mint max of 1000.
a scanner spots the deploy json in the mempool and decides to claim 90% of the mint.
arguably this is a problem that solves itself as it can start to be a waste of money but it may mean hyped projects could not use this protocol to launch.

This is a good point @jokie88. Can bots do the same thing with BRC-20 right now, and do they? I really don't know.

yeah good point   one idea is to add a private key flag or such e.g.
"private": true

meaing only the original deployer can mint (or really only the mints from the owner address are accepted) another is to add a owner(s) key option or such e.g.
"owner":  "address here"  or
"owners": ["address 1","address 2",]
maybe only allow one owner bitcoin address - and let schnorr multi-sig in one sig handle it.  Anyways,  it's still early - let's collect / research possible extensions.  More suggestions welcome

you can sort of prevent it w/ brc-20 by deploying and minting the whole supply to yourself in the same block (2 inscriptions)
then you can transfer smaller quantities during the mint to the minters.
not as easy w/ orc-721 due to the fact that each one is a separate inscription

If you really wanted to couldn't you still initiate all the inscriptions in a batch right after the deploy confirms, similar to minting all of the brc20? I realize it's many more txns, but could still be done in one block most likely, right? Plus, if a bot's transactions confirm before the deploy, they would be invalid, so it is an even playing field after the deploy block confirms.
The bot would have to pay higher fees than the inscribers, which would be unlikely because the inscribers would be doing way smaller numbers and would likely be willing to pay higher fees than a bot whose transactions could end up invalid.

yes i guess brc-20 is bottable too but in that case not much is lost. you can just reinscribe the two inscriptions with a slightly different attribute.
with orc-721 you have to inscribe your entire collection which leave a lot of gaps for anyone watching the pool (or makes it expensive like you say -- to try to rush it all in.)

i think one trouble still to research / better understand for the validation rule is the diyordibots deploy that was possible done in one block BUT:

the first mints in the block are ordinals before the deploy inscription and
the spritesheet inscription is after the deploy inscription

not a memory pool expert or block ordering / mining specialist  - i might be wrong here (and it was not done in a block and not sure if the order within  a block is assigned random or you can determine if you submit).
for now the thinking for validation is  - to reject mints before deploy   -  still undecided about deploys before spritesheets  (for now the plan is to accept as valid if the inscription exists - maybe add the rule within the same block if in the future).


Allow download directory to be configurable

There are a couple goals here. The first is with respect to the failing
allspark tests. The issue seemed to be because the /tmp dir was in a
separate partition from the filestore and this caused linking to fail
when the dest_path option was specified. The sensible solution would be
to place the filestore root in the same directory as dest_path. But
dest_path was poorly named. So I changed it to download_dir. The other
download funcs could use something similar I ended up solving an
auxilary goal: adding a configurable download parameter all of the
download endpoints.
The download_dir option is currently undocumented because it will be
simpler to take care of during breaking API changes in #297

Codecov Report

Merging #305 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #305      +/-   ##
==========================================
+ Coverage   85.13%   85.15%   +0.01%
==========================================
  Files          37       37
  Lines        1709     1711       +2
==========================================
+ Hits         1455     1457       +2
  Misses        254      254

Impacted Files
Coverage Δ

hca/dss/__init__.py
90.2% <100%> (+0.06%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54c6405...97468e8. Read the comment docs.

This kind of fixes #303 but doesn't add any documentation. Since this PR is supposed to be a 'quick fix' it seemed appropriate to save the docs for #297

@amarjandu I think this may break the datastore's integration tests since it changes a command line option from --dir-name to --download-directory. Either way it's an easy fix.

@amarjandu I think this may break the datastore's integration tests since it changes a command line option from --dir-name to --download-directory.   But ill keep my 👀  peeled for issues.

Yes and that command line option is changed in the basic bundle download.
On Fri, Apr 19, 2019 at 9:17 AM amar jandu<EMAIL_ADDRESS>wrote:

@amarjandu https://github.com/amarjandu I think this may break the
datastore's integration tests since it changes a command line option from
--dir-name to --download-directory.  But ill keep my 👀 peeled
for issues.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/HumanCellAtlas/dcp-cli/pull/305#issuecomment-484946292,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACR4ZFGPAH5M5T3RDU3SCFLPRHWBDANCNFSM4HGPAI7Q
.

Yes and that command line option is changed in the basic bundle download.
…
On Fri, Apr 19, 2019 at 9:17 AM amar jandu @.***> wrote: @amarjandu https://github.com/amarjandu I think this may break the datastore's integration tests since it changes a command line option from --dir-name to --download-directory.  But ill keep my 👀 peeled for issues. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#305 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ACR4ZFGPAH5M5T3RDU3SCFLPRHWBDANCNFSM4HGPAI7Q .

i believe this is the only context its used in. Thanks for the heads up! 😃

@amarjandu ok coool. Looks like we're good then.


804 feat update icons

Issue Number
fix #804
Compliance

[X] I have verified that this change complies with our code and contribution policies.
[X] I have verified that this change does not cause regressions and does not affect other parts of the code.

Thanks @jAllanOli ! Amazing job!


Double jump does not work

Minecraft Version
1.20.1
Fabric Version
0.15.3
Fabric API Version
0.91.0
Hephaestus Version
<IP_ADDRESS>
Describe your issue
The double jump upgrade does not work. Other upgrades like the frost walker are working properly. I am NOT sure if my method of applying double jump is correct. I tried double-clicking the space bar in survival mode but nothing happened. I also tried pressing 'I'(The default key for Leggings Interaction) but still no response.
Crash Report
No response
Other mods
PortingLib 2.1.1163+1.20.1
Tried reproducing with just Tinkers?
Yes
Performance Enchancers
None of the above
Searched for known issues?
Checked pinned issues, Searched open issues, Searched closed issues, Checked the FAQ

It's because the info about the extra jumps is stored in a capability, which Hephaestus does not register:
https://github.com/Alpha-s-Stuff/TinkersConstruct/blob/8d64ca2073c528d4183dd6add172756c99bcb018/src/main/java/slimeknights/tconstruct/tools/modifiers/ability/armor/DoubleJumpModifier.java#L51-L74
(the PersistentDataCapability.CAPABILITY is null)

Fixed in https://github.com/Alpha-s-Stuff/TinkersConstruct/commit/406cfa581a47288c01b1a1d28aa614f9f54941ed


torch.cuda.DeferredCudaCallError when installing vllm

Hi,
I'm trying to run vllm on a 4-GPU Linux machine.
When I followed the Installation guide to pip install vllm, I got this error:
torch.cuda.DeferredCudaCallError: CUDA call failed lazily at initialization with error: device >= 0 && device < num_gpus INTERNAL ASSERT FAILED at "../aten/src/ATen/cuda/CUDAContext.cpp":50, please report a bug to PyTorch.

If I explicitly set the visible device to just device:0, via:
export CUDA_VISIBLE_DEVICES=0

then the installation worked fine.
However, when I switched back to the default (export CUDA_VISIBLE_DEVICES=0,1,2,3), the DeferredCudaCallError occurred.
Similar pattern was observed when I tried to do llm = LLM(model="facebook/opt-125m"); it'd only worked when visible device is pinned to just 1 GPU.
Has anyone run into similar issues?

Just an update. This issue was caused by MIG configuration, which by default is enabled on A100 GPUs.
Details in this PyTorch discussion: https://discuss.pytorch.org/t/cuda-initialization-error-with-torch-zeros-1-cuda/183460


Private Mode Launcher Shortcut for Specific Albums

A feature that allows users to create a launcher shortcut for a specific album in the gallery app. When launched from this shortcut, the app should open in a "private mode" where only the selected album is visible, and all other albums are hidden. To view other albums, users should launch the gallery app normally.
Details:

Functionality:

Allow users to select any album and create a dedicated launcher shortcut for it.
When the gallery app is opened through this shortcut, only the selected album should be accessible.
All other albums should be hidden from view unless the user restarts the app without album shortcut.

Use Case:

This feature is useful for users who wish to quickly access a specific album without exposing other albums.
It enhances privacy, especially when sharing the device or showing photos from a specific album.

User Flow:

The user selects an album and chooses the option "Create Private Shortcut."
A shortcut is added to the launcher for that album.
When the user taps the shortcut, the gallery app opens directly to the selected album in private mode.
To view other albums, the user must exit the private mode or launch the app through the normal app icon.

Benefits:

Improves user privacy by limiting access to specific albums.
Provides quick access to frequently used albums.
Enhances user experience with customizable shortcuts.

Provides quick access to frequently used albums.
Enhances user experience with customizable shortcuts.

For these two you can already create shortcuts to custom collections. For a specific album, open the album you want, then in the Collection page select in the menu "Add shortcut".

Improves user privacy by limiting access to specific albums.

I don't intend to provide sandboxing features. If other people have physical access to devices, superficial features like these won't cut it.


Installation: "future" module is missing

after standard install using "pip install -r requirements.txt" command, trying to run application using Python 2 I got error:
Traceback (most recent call last):
File "copier.py", line 11, in
from PythonConfluenceAPI import ConfluenceAPI
File "c:\python27\lib\site-packages\PythonConfluenceAPI_init_.py", line 6, in
from future import standard_library
ImportError: No module named future
so then I run command:
"pip install future"
and application started working
so probably installation of "future" is missing

Hm.. I can't find it in my local working copy:
grep "future" . -Rl

Also, there is no init.py yet. Are you sure it's not a local problem?

looks like error was thrown by PythonConfluenceAPI
so maybe it's out of scope here

Indeed, looks like originally it came from PythonConfluenceAPI


Build using CMake does not generate re2.pc and zlib.pc files

env \
    CC=/usr/bin/gcc \
    CXX=/usr/bin/g++ \
cmake \
    -S /opt/.source/grpc \
    -B /opt/.build/grpc-1.54.0 \
    -D CMAKE_BUILD_TYPE=Release \
    -D CMAKE_INSTALL_PREFIX=/opt/grpc-1.54.0 \
    -D BUILD_SHARED_LIBS=ON \
    -D gRPC_INSTALL=ON \
    -D gRPC_BUILD_TESTS=OFF

cmake \
    --build /opt/.build/grpc-1.54.0 \
    --target install \
    --config Release \
    --parallel 12

gRPC isn't supposed to generate pc files for targets that gRPC doesn't own such as re2.pc and zlib.pc. You need to install them separately if you want to have them.


Error while sourcing - add-zle-hook-widget

System: Win10 + WSL
zsh: 5.1.1 (x86_64-ubuntu-linux-gnu)
When sourcing plugin I get:
(anon):1: add-zle-hook-widget: function definition file not found
/home/qyadr/.antigen/bundles/softmoth/zsh-vim-mode/zsh-vim-mode.plugin.zsh:558: add-zle-hook-widget: function definition file not found

This is the line:
add-zle-hook-widget line-finish vim-mode-cursor-finish-hook

What is it for?
Thanks.

It's used to track the input mode (insert, command, search, etc.) and update the prompt / cursor style appropriately. The function is available since ZSH 5.3. http://zsh.sourceforge.net/Doc/Release/User-Contributions.html#Manipulating-Hook-Functions

I should say that it would be possible to degrade functionality gracefully if that feature isn't present, and only track viins / vicmd modes (ignoring search and visual modes).


Trouble with the SQL Alchemy strings

I'm stoked to have discovered csvkit.  It makes me sad to think how many hours have been wasted doing these types of data manipulations by hand in the past.  Thanks for your work on this & releasing it as open-source.
I'm having trouble connecting to a local database using the SQL Alchemy strings.  Everytime I do something like:

csvsql --db 'postgres://localhost:5432' --insert --tables table data.csv

I get:

You don't appear to have the necessary database backend installed for connection string you're trying  to use. Available backends include:

Postgresql:	pip install psycopg2
MySQL:		pip install MySQL-python

For details on connection strings and other backends, please see the SQLAlchemy documentation on dialects at:

http://www.sqlalchemy.org/docs/dialects/

I've installed both psycopg2 and MySQL-python, but neither option works for me.  I have no problem if I open a python prompt and type "import psycopg2" but I've had no luck getting the connection to work via csvsql.
Any tips on what I should try next?  I'm running python 3.5, btw.

Are you in a virtualenv?
I have csvkit installed system-wide, but not psycopg2. When I installed csvkit in a virtualenv alongside psycopg2, I could run ~/.virtualenvs/myvirtualenv/bin/csvsql  --db 'postgres://localhost:5432/mydatabase' --insert --tables mytable mycsv.csv If you install both system-wide, it will also probably work.

See #522 for discussion of same issue. Can re-open if the solution towards the end of #522 doesn't work.


Problem backing up large accounts

Using GYB 1.11 on windows 10 I'm having troubles to backup 2 accounts of a G Suite domain.
These 2 are both the biggest accounts in the list with 76K and 85K messages. All the others (smaller ones) complete without problems.
The command used:
D:\gyb_1.11\gyb.exe --email<EMAIL_ADDRESS>--service-account --local-folder "\path"
Outcome:
Got 84583 Message IDs
GYB needs to examine 84583 messages
GYB already has a backup of 84204 messages
GYB needs to backup 379 messages
Traceback (most recent call last):
File "gyb.py", line 1875, in
File "gyb.py", line 1301, in main
File "gyb.py", line 522, in callGAPI
File "googleapiclient_helpers.py", line 130, in positional_wrapper
File "googleapiclient\http.py", line 1432, in execute
File "googleapiclient\http.py", line 1364, in execute
File "oauth2client\transport.py", line 175, in new_request
File "oauth2client\transport.py", line 282, in request
File "httplib2_init.py", line 1511, in request
File "httplib2_init_.py", line 1261, in request
File "httplib2_init.py", line 1185, in _conn_request
File "http\client.py", line 1239, in request
File "http\client.py", line 1285, in _send_request
File "http\client.py", line 1234, in endheaders
File "http\client.py", line 1065, in _send_output
File "http\client.py", line 986, in send
File "ssl.py", line 972, in sendall
File "ssl.py", line 941, in send
File "ssl.py", line 642, in write
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
[4916] Failed to execute script gyb
What could I try to fix this?
Thanks in advance!

Does the same error get thrown every time you run the command?
On Fri, Sep 7, 2018, 7:13 AM rimeco<EMAIL_ADDRESS>wrote:

Using GYB 1.11 on windows 10 I'm having troubles to backup 2 accounts of a
G Suite domain.
These 2 are both the biggest accounts in the list with 76K and 85K
messages. All the others (smaller ones) complete without problems.
The command used:
D:\gyb_1.11\gyb.exe --email<EMAIL_ADDRESS>--service-account
--local-folder "\path"
Outcome:
Got 84583 Message IDs
GYB needs to examine 84583 messages
GYB already has a backup of 84204 messages
GYB needs to backup 379 messages
Traceback (most recent call last):
File "gyb.py", line 1875, in
File "gyb.py", line 1301, in main
File "gyb.py", line 522, in callGAPI
File "googleapiclient_helpers.py", line 130, in positional_wrapper
File "googleapiclient\http.py", line 1432, in execute
File "googleapiclient\http.py", line 1364, in
execute File "oauth2client\transport.py", line 175, in new_request File
"oauth2client\transport.py", line 282, in request File "httplib2_init.py",
line 1511, in request
File "httplib2_init_.py", line 1261, in
request File "httplib2_init.py", line 1185, in _conn_request
File "http\client.py", line 1239, in request
File "http\client.py", line 1285, in _send_request
File "http\client.py", line 1234, in endheaders
File "http\client.py", line 1065, in _send_output
File "http\client.py", line 986, in send
File "ssl.py", line 972, in sendall
File "ssl.py", line 941, in send
File "ssl.py", line 642, in write
ConnectionResetError: [WinError 10054] An existing connection was forcibly
closed by the remote host
[4916] Failed to execute script gyb
What could I try to fix this?
Thanks in advance!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/jay0lee/got-your-back/issues/182, or mute the thread
https://github.com/notifications/unsubscribe-auth/AEaMsBCU41lW3l7Zl4nSttKG5u0EYEC6ks5uYlS8gaJpZM4WepQV
.

I tried at least 5 times since this morning, to rule out local connection problems.
So far I get the error every time, but only for those 2 accounts. Other accounts go fine in the meantime.

Had no issue downloading 67,100 emails (over 26GB) from a G Suite account for the first time using the macOS binary contained in gyb-1.11-macos.tar.xz:
$ gyb --email<EMAIL_ADDRESS>--action backup

Using backup folder<EMAIL_ADDRESS>Got 67100 Message IDs
GYB needs to examine 67100 messages
GYB already has a backup of 0 messages
GYB needs to backup 67100 messages
backed up 67100 of 67100 messages

GYB needs to refresh 0 messages

These are generally network errors.

Don't backup over WiFi. It can be flaky
Try turning Security/Antivirus software off, they can interfere with GYB connections.
Ultimately GYB should be able to handle and recover from these issues as you found.


Native Crashes when running Smoke Tests

My pipeline https://dev.azure.com/vscode/VSCode/_build/results?buildId=96901&view=logs&j=15f6beb8-c39c-5fa4-f1fe-6e73c89d1002&t=f548697b-f1e3-56d7-b20f-10575b2a648b&s=96ac2280-8cb4-5df5-99de-dd2da759617d
PR that triggered it https://github.com/microsoft/vscode/pull/185658
Native crashes:
logs-linux-x64-smoke-1.zip
@deepak1556 let me know if you need more details from my side

Above are just the logs, attaching crash dumps:
crash-dump-linux-x64-smoke-1.zip

This would be https://github.com/microsoft/vscode-internalbacklog/issues/3454 affecting the OSS electron builds since they don't contain the fix. I will keep this issue open to upstream the fix.
The crash itself is just a side effect of the workbench terminating the connection with the extension host eagerly, might want to investigate if the message port was disconnected gracefully.

Change has been upstreamed https://github.com/electron/electron/pull/39335

Addressed in https://github.com/microsoft/vscode/pull/189904


Disabling UEFI booting

I need to make a custom iso for nixos that does not have UEFI enabled at all.  Old mother board that doesnt understand it can't boot with UEFI enabled ISO.  I can build the custom ISO, but who knows what setting I need in my configuration.nix that will build the nixos installert without UEFI support?

I recently installed with legacy BIOS support by following the manual. There is documentation on how to build ISOs. Together they form a solution to your problem. I have never heard of UEFI support being present have any impact. In the remote case this is the case, you could find all the kernel options related to UEFI and compile yourself a custom kernel (this too, is documented). See https://wiki.archlinux.org/index.php/Unified_Extensible_Firmware_Interface#Linux_Kernel_Config_options_for_UEFI.
Please close this issue with a comment of how it all went.

I turns out the issue was actually my cd burner.  I thought maybe it was a driver problem but even booted into windows the drive produced bad disks.  So I plugged in an external CD drive to burn the ISO and the stock ISO booted on the old system just fine.


[Unit Testing] Support primative types + objects and arrays in dbt-bigquery

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt-bigquery functionality, rather than a Big Idea better suited to a discussion

Describe the feature
Currently, the unit testing framework in core uses the safe_cast utils macro to convert a user-provided value for an input mock to the appropriate type. In some instances, this safe_cast may fail and need special handling depending on the type.
The acceptance criteria for this issue is to introduce a test that provides coverage across any non-STRUCT input types for unit testing input data, potentially updating the safe_cast macro as necessary.
Describe alternatives you've considered
No response
Who will this benefit?
No response
Are you interested in contributing this feature?
yes
Anything else?
STRUCT support is out of scope for this initial leg of work.

Hey @MichelleArk, is there a plan to support UDFs as well? I tried it and it doesn't work. It fails like that:
 Traceback (most recent call last):
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/base.py", line 376, in safe_run
    result = self.compile_and_execute(manifest, ctx)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/base.py", line 322, in compile_and_execute
    result = self.run(ctx.node, manifest)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/base.py", line 423, in run
    return self.execute(compiled_node, manifest)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/test.py", line 251, in execute
    unit_test_result = self.execute_unit_test(test, manifest)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/test.py", line 182, in execute_unit_test
    unit_test_manifest = self.build_unit_test_manifest_from_test(unit_test_def, manifest)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/task/test.py", line 176, in build_unit_test_manifest_from_test
    return loader.load()
           ^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/parser/unit_tests.py", line 54, in load
    self.parse_unit_test_case(unit_test_case)
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/parser/unit_tests.py", line 98, in parse_unit_test_case
    get_rendered(unit_test_node.raw_code, ctx, unit_test_node, capture_macros=True)
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt/clients/jinja.py", line 146, in get_rendered
    rendered = render_template(template, ctx, node)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt_common/clients/jinja.py", line 540, in render_template
    return template.render(ctx)
           ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/dbt_common/clients/jinja.py", line 145, in render
    return self.environment.handle_exception()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/anaconda3/envs/dbt-1.8/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
TypeError: sequence item 0: expected str instance, Undefined found

I couldn't find an opened issue for that need. Do you want me to make one?


ModelsList 'id' serialization

var Podium = Model.inherit({
attributes: {
models: Model.attributeTypes.ModelsList(FashionModel)
}
}),
podium = new Podium(data),
list = podium.get('models'),
kate = new FasionModel(1, {name: 'Kate'}),
list.push(kate),
podiumRestored;
kate.isNew(); // false
podiumRestored = new Podium(podium.toJSON());
kate  = podiumRestored.get('models').get(0);
kate.isNew(); // true

It's correct behaviour for current moment. In common case we can't know that field is storage id. For mongo it could be _id or any other key in collection. I some situations we could have few keys.
In BEM.MVC they have a solution: a special type of the field https://github.com/bem/bem-mvc/blob/master/common.blocks/i-model/__field/_type/i-model__field_type_id.js
Should we have the same? What you think?

Yes I think we definitely should get rid of internal "id" and make storage key normal serializable attribute. It may be special attribute type or just flag (e.g. storageId: true).
isNew may look like iterating through all storage attributes and check for emptiness.


Re-enable post-Sapling sync tests

Motivation
We disabled our post-Sapling stateful sync tests when the state format changed and required a re-caching of state.
Solution
This re-enables the mainnet test with the newly synced v3 state disk, when testnet is also re-synced I'll enable that in another PR.
Review
Not urgent, will test with a manual trigger of the 'Test' workflow.
Related Issues

Follow Up Work
Re-enable the testnet variant when the re-sync disk is ready. @dconnolly

This re-enables the mainnet test with the newly synced v3 state disk,

I think we're at database format version 4 right now?
https://github.com/ZcashFoundation/zebra/blob/main/zebra-state/src/constants.rs#L14

This re-enables the mainnet test with the newly synced v3 state disk,

I think we're at database format version 4 right now?
https://github.com/ZcashFoundation/zebra/blob/main/zebra-state/src/constants.rs#L14

You're right! 👀

(The macOS failure is a GitHub PR git ref error.)


Adds Apache2 as new Proxy Webserver option

@flightcom please add an example and some documentation on how to use the volume, the sections using NGINX as example should provide an example for Apache as well.


Add another maxretry handler which does not create additional exchanges

I bet to get this merged, it'll be helpful to convert from rspec back into minitest like. If you'd like some help with this, I could either pair with you on it or I might be able to take a look and try to convert it myself. If you're not merged in by then, I'll pull request to your branch.

Hi, I don't have much experience with minitest. To be honest, I don't know what you mean.

Cool, how does this differ in terms of the existing max retry exchange?
It does not create extra exchanges but does this have a performance penalty? Any change you can draw some topology out?
thanks looks interesting!

After some experimenting a couple of pointers for other people that want to try:
when making a worker you must add the following arguments:
class Worker
  from_queue :foo, handler: Sneakers::Handlers::RoutingMaxretry,
                              arguments: { 'x-dead-letter-exchange' => "exchange",
                                                   'x-dead-letter-routing-key' => "queue.foo.retry" }

  # ....
end

Notice that its .retry and not -retry as with the current maxretry handler
Next, wildcard route matching is quite dangerous. I had a general routing key from my exchange to foo using # but this creates an endless feedback loop. If any of your defined routing_keys also match the following then you might have problems (I'm using example queue name foo):

queue.foo.error
queue.foo.requeue
queue.foo.retry

I am working on using this, but when an error goes to the error queue bunny errors and gives me a IOError: not opened for reading error. I am investigating but just thought I would post here if anyone has got an idea. :)


A redemption with zero value is allowed by tbtc but rejected by the mempool

Description
Bitcoin rejects transactions that send "dust", and an output with zero sats very comfortably meets the definition of dust. Such a transaction isn't rejected by tbtc, creating a discrepancy which can be exploited to exchange BTC for signer ETH.
Exploit Scenario
Eve creates a deposit and requests a redemption with 0 value. The signers are unable to get this transaction included in a block without going directly to a miner. Assuming they fail to do so, Eve gets 150% of her deposit back in the form of ETH.
Suggested Remediation
A simple solution would be to implement an upper cap on fees. A better one might be to reimplement the dust calculations, although the dust threshold is set on a per-node basis. As both solutions are slightly brittle, a third solution might be to redesign the system such that invalid transactions that can't be accepted into the mempool can be used to penalize the depositor in some way.

The current thought is to set an upper bound of lotSize - initialRedemptionFee, That would make the value always >= minimumRedemptionFee (currently 2000 sat)
@prestwich would love to run this by you for a sanity check
@Shadowfiend, in case I missed anything

What's the failure behavior here? Who loses funds if the tx never makes it on chain, or the UTXO is 100% sent to fees?

Signers do (as it manifests as signer fraud, since the signers cannot further bump the fee and they become subject to the redemption timeout).
On the other hand, there's nothing stopping the signers from broadcasting the tx with a lower fee… The fee in the deposit acts as an upper bound. So actually I don't think this is a viable attack.

Ah, there is something stopping them: the signature must be authorized or the signers risk bond loss. So if the redemption request is submitted with a starting fee at a high value, the deposit may be unredeemable, fees unchangeable, and the signers may forfeit their bonds. Signers can still take the BTC with whatever fee value they want once they lose their bond, but it's not a great scenario.
With this in mind, it seems like the right move here is to set an upper bound on the fee (as suggested by @NicholasDotSol ) but only in requestRedemption. Thoughts?

Stepping back from mechanics for a second, the motivation for the current design was that the loop would NEVER exceed the UTXO value, because we considered the odds that fees would eclipse lot size (1 BTC) to be negligible. like LN, the security model breaks if fees are higher than the amount of BTC being secured
So it's not as simple as just adding a maximum. we have to decide who should pay if the maximum is reached. I'd lean towards having the redeemer pay

Hmmm… I'm not sure I'm following here. In a standard scenario where the fee starts low and is fee-bumped to be high, the redeemer does pay, insofar as we expect the redeemer to broadcast the tx as soon as it can be accepted so they can get their funds with minimum loss, or the signers to do so in order to free up their bonds and avoid punishment. In that scenario, if the fee is eventually bumped to any maximum, be it the total UTXO value or some amount less than that, all fee values between the initial value and the max value have valid, authorized signatures, and any of those signatures can be used to broadcast + prove the transaction safely, by signers or redeemer.
The specific scenario in this issue is the redeemer's full control over what the initial fee is, and the fact that a high enough initial fee allows for the initial transaction, without any fee bumps, to have too small a value and therefore be rejected. If we set an upper limit on the initial fee, the redemption flow won't start in the first place.

if the fee is eventually bumped to any maximum

my question is "then what happens." After the fee has been bumped to maximum, what happens?

Two options: anyone (e.g., signers or redeemer) broadcasts the transaction to Bitcoin with any of the fee steps and proves the outcome (or short-circuits the fee bumping loop in the first place by doing this before any max is hit), or the redemption proof timeout expires and the redeemer has the option of claiming signer bonds.

or the redemption proof timeout expires and the redeemer has the option of claiming signer bonds.

So if the maximum is lower than the network fee rate, the signers lose. If you want to bound the fee bumping loop, I would make it the opposite to avoid punishing signers for something outside their control

If the max is lower than the network fee rate everyone loses, though the signers lose more due to overcollateralization---at this point the redeemer has already given up their TBTC to initiate redemption.
I think I wasn't clear, though: I'm not suggesting bounding the fee bumping loop anymore. I'm only suggesting bounding its starting point, since that's the attack vector here.

That does leave us with the effective bound that already exists, of course, which is UTXO value - initial fee. As such, if the initial value of the fee is >1/2 lot size, the fee bump loop is effectively disabled (because the first bump would result in a fee > UTXO value, since fee bumps must be in steps = initial fee).
I think there's room to handle the situation you're describing by saying that IF the latest fee is >= (UTXO value - initial fee) (or whatever upper bound) AND a signature has been provided for that fee, THEN the signers are allowed to close the deposit unilaterally without a transaction proof and reclaim their bond. That would require tracking one additional boolean for whether the last fee has a corresponding signature (currently provided signatures are only emitted as events, they aren't tracked in the deposit), and it would add a bit of extra complexity to the state machine for redemption. I'm a bit loath to change the mechanism of proof timeout this late in the game for a limitation that already existed, but if we're feeling like this is an attack vector I think it's fairly manageable. It needs a little more study to see if any new attacks open up by having this proofless exit available for signers, though.

That does leave us with the effective bound that already exists, of course, which is UTXO value - initial fee. As such, if the initial value of the fee is >1/2 lot size, the fee bump loop is effectively disabled (because the first bump would result in a fee > UTXO value, since fee bumps must be in steps = initial fee).

Yes, this is exactly what I'm saying. If the next fee is ever > than the UTXO size we've entered a failure case. We previously assumed this could never happen because the UTXO was 1 BTC (but that may not have been a good assumption without a maximum initial fee). This failure case would result in some valid txn being signed sending BTC to the user, and then potentially a slashable failure to bump the fee after that. That would cause the signers to forfeit BTC and bond despite behaving honestly.

Ok, here's an updated proposal which handles some of the concerns, but not all of them.
First, define a minimum post-fee UTXO value, minimumUtxoValue. This is the minimum value that the contracts will accept as utxoValue - fee.
Second, disallow an initial fee that is < utxoValue / 2.
Third, if a fee bump would result in a post-fee UTXO value < the minimum, permit the fee bump, clamped to the minimum.
Fourth, if needed, adjust the minimum permitted lot size (currently 50k sats) if needed.

This approach doesn't really fix the problem---there is still a failure case where the minimumUtxoValue prevents a high enough fee from being used, and in this case the signer will still forfeit their bond and (presumably when fees settle down) the BTC. However, it also doesn't require a proofless exit, which feels like it needs more study (and may be a good idea for v2), and with appropriate parameters it should minimize the probability of such an event, even if it doesn't completely eliminate it.
In particular, if the minimum UTXO value is 2000 sats, and the minimum available lot size is 50000 sats, the occurrence would require a fee event that would,  at minimum, cause loss of 96% of the UTXO value in fees. An honest redeemer can be expected to start their redemption at the lowest reasonable fee value, and to avoid high-fee scenarios if possible. A dishonest redeemer could push for high fees to grief, but assuming the minimum lot size of 50k sats is available and the minimum UTXO value is 2k sats, my understanding is that would require fees of ~480 sats/vbyte, sustained for ~5 hours (due to the limited initial fee + allowing an additional fee bump, and assuming an hour to get the requisite confirmations). If this feels insufficient, we can raise the minimum allowed lot size to also raise the max fee.
Worth noting: there is an effective (non-governance) minimum lot size of over 100k sats at the moment due to the ETH/BTC price and #634 .

Second, disallow an initial fee that is < utxoValue / 2.

Do you mean >= utxoValue / 2?  i.e. it must be possible to bump at least 1 time?
Other than that this seems pretty reasonable.

Hah, yep, updating.


message view: Selecting text over multiple messages looks messy.

E.g. the following looks broken, even though if I copy it it's actually going to do something great:

I think highlighting only the message text and the name is probably what we want?
Not sure if this is easy or hard. At minimum we should make all the little vertical lines and the "EDITED" unselectable.

@rishig Is it just the message text and the sender name you want or event the topic and stream names.

Like This -

@zulipbot claim.

@rishig I believe we just fixed this; can you test in Firefox to make sure it works correctly there (I only carefully tested Chrome)?

Minor thing: It selects the "Edited" in Chrome.

I just fixed "Edited"; apparently stopped working in my last rebase.  By "eats the newline", do you mean "doesn't provide a blank line", or did you actually have things without a line between?
The timestamp is complicated; it actaully gets copied or not depending on how you copy things (ctrl-C vs. using the middle-click on Linux).  I'm going to consider that out-of-scope for this issue (we should open a new issue for the inconsistency, but I think it's actually unclear what we should do, since anyone doing operations work really really wants the timestamps included, but other users probably would be happy enough without them).  Solution might involve some documentation or a new setting or something.

by "eats the newline" I just mean "doesn't provide a blank line"
Currently with Ctrl-C it prints "Yesterday" or "Today" (i.e. the date in the floating recipient bar), even though it is not printing the time. I think it should either print the date and time, or neither.

I can open a new issue for this (I thought there was an issue, but I don't see it), but I think the main thing in the discussion at the time was for the highlighted part to indicate what is being copied. So e.g. if one drags the mouse over half of two messages, it should select all of both messages.
I also think that this would look better if it didn't select the date change

or the stream/topic in the recipient bar:

Solution might involve some documentation or a new setting or something.

I'd be in favor of not highlighting the timestamps, and having documentation that explains the middle click behavior. I'd also be fine with a setting.

Issues specific to firefox

The highlight over the stream name is inverted

Highlighting two messages in different recipient blocks doesn't provide a blank line (even in vanilla/unformatted messages)
Timestamp is copied with ctrl-C (and in a confusing way; the timestamp of message n is shown in the same block of text as message n-1)
The "select the entire message when > 1 message" isn't working it all

I think date changes are more important than timestamps, and you basically always want to know when 3 days passed in between two parts of a conversation, since that almost always has a significant impact on the meaning of the discussion.  So I think that means:

Ctrl-C should include the dates
The date of at least today should be highlighted.  We could potentially make it be the copy that's in the recipient bar, not the date change separator, though.  (Arguably, we don't need the dates to appear on those date-change separators at all, now that they're on the recipient bar, and we should just make them into unadorned divider lines).  If that seems like a reasonable plan, let's open an issue for that detail.

I don't think there's a middle-click equivalent for Mac, so I'm not sure just documenting that is satisfactory.  I think we should open a separate issue for figuring out copy-paste and timestamp interactions; that alone is probably more complicated than everything else.  I can imagine solving that in a way we actually like taking someone a week of work, and seems best to separate that from these tactical things (like the highlighting looking ugly) that take like 5 minutes each.

The highlight over the stream name is inverted

@brockwhittaker can you investigate how to fix this?  It's kinda weird.

Highlighting two messages in different recipient blocks doesn't provide a blank line (even in vanilla/unformatted messages)
Timestamp is copied with ctrl-C (and in a confusing way; the timestamp of message n is shown in the same block of text as message n-1)

I think these are happening in Chrome too, and seems like the main priority issue for 1.7 here.

The "select the entire message when > 1 message" isn't working it all

I think the actual logic is that we use the browser's native copy-paste when selecting <1 message, and our code for >1.  I am not confident there's a "Select entire message" feature.  So is this actually a problem?

Ctrl-C include dates: https://chat.zulip.org/#narrow/stream/design/topic/copying.20messages
Date separators: https://chat.zulip.org/#narrow/stream/design/topic/date.20separators
Copy-paste and timestamp interactions: issue opened
Various firefox copy bugs: should be fixed, but probably don't need to hold the release for them

I think the actual logic is that we use the browser's native copy-paste when selecting <1 message, and our code for >1. I am not confident there's a "Select entire message" feature. So is this actually a problem?

ah, the issue then is that selecting multiple messages is not triggering our code, and is instead using the browser's native copy-paste.


Fix Issue 23940 - std.getopt does not assert with options that only d…

…iffer in case with config.caseInsensitive

@CyberShadow  This is a huge breaking change for getopt users! getopt with no config options whatsoever now asserts, ... at runtime.
This broke upgrades to recent compiler at Weka. CC @JohanEngelen (our traces instrumentor)
I can understand the breakage in a pedantic standpoint tho, as this is documented as such, although not entirely clear about short vs long options, but we should have announced this in the changelog, at bare minimum, and, imo, a better discussion.
A common valid use-case:
-d --debug
-D --directory

I would expect short options to be case insensitive here. This exact options now fail with this patch. Should we reconsider or/and document this better?

Ahh, sorry about that :(
To be fair, this is one of those "thing that silently never worked in the first place is now correctly reported as an error" types of situations.

I would expect short options to be case insensitive here.

I don't understand what you mean by this. How can they be case insensitive if you have two options that differ only in case?

@ljmf00-wekaio wouldn't this be fixed for weka by explicitly setting config.caseSensitive ?

@ljmf00-wekaio wouldn't this be fixed for weka by explicitly setting config.caseSensitive ?

It does, but given that was not the case, now only if it reaches the getopt call, it asserts at runtime. (given the nature of getopt, its not possible to catch it at compile-time). I think this can be a huge breakage since it may affect a lot of projects (and more silent breakage if they have multiple getopt, for subcommands).
I think, at bare minimum should have been in changelog.

@thewilsonator Do we have any precedents for what to do in this situation?
More generally, what to do about breaking changes that can't be detected at compile-time? E.g. do we make built programs print a message to stderr? We could also do nothing and continue silently accepting wrong/self-contradictory parameters...
Should we document this breaking change retroactively by updating that version's changelog on dlang.org?

Should we document this breaking change retroactively by updating that version's changelog on dlang.org?

I agree with it.
Regarding the other suggestion, my opinion would be to ease that for -d and -D situations. I interpret the documentation as if you could do --debug or --Debug or --DEBUG, but if there's d|debug and D|documentation, -d and -D should have the possibility to be sensitive while the long option insensitive, but as far as I understand, there's no specific mention on short and long options, regarding the case sensitive option.

In anyway, this is something that I would really like to discuss. I also believe that, in the long term for Phobos v3, we should look at projects like https://github.com/andrey-zherikov/argparse from @andrey-zherikov. I generally agree with the design choices (some things I would change), and I'm sure Phobos folks would also generally agree with them. I think this being an adoption to Phobos would modernise our standard library.
In some projects I saw the necessity of using it because the Phobos getopt is simply not good/powerful enough.

(some things I would change)

Could you please create issue(s) with your thoughts so we can discuss? I'm living in my own bubble without feedback from others. This is a good time for new ideas because I'm working on new major version where I can accommodate as much breaking changes as possible.


Update mkl fpk GPU and compiler version for 2024.2.0

Cherry pick commits from main.

/intelci: run

It seems this is a duplicate of https://github.com/oneapi-src/oneDAL/pull/2699
And #2699 is already merged.

@Vika-F it's the same update as in #2699 but for rls/2024.2.0-rls branch.

@Vika-F it's the same update as in #2699 but for rls/2024.2.0-rls branch.

Sorry, hadn't noticed that.


Add ability to load library (like Django) before running

Is your feature request related to a problem? Please describe.
I'm trying to use mkdocstrings to document an API that I wrote in Django. However, if you try to access anything in the library without first running django.setup(), you run into an AppRegistryNotLoaded error. For the overall mkdocs library, this would be fine, because I can write a script that sets up django and then runs mdkocs. However, due to this line
https://github.com/pawamoy/mkdocstrings/blob/b23daed3743bbd2d3f024df34582a317c51a1af0/src/mkdocstrings/handlers/python.py#L147
that method doesn't work. The context isn't transferred to your subprocess, so even if I setup django before running mkdocs, I run into the error.
Describe the solution you'd like
I'd love to have a pre_process hook in the mkdocs.yml config for this plugin that allows me to specify a list of scripts I want to run when the subprocess gets kicked off

Hello @rossmechanic, thanks for taking the time to write this :slightly_smiling_face:
I totally agree we should do something about this. However, even with a pre_process key in the plugin configuration, I'm not sure how we could make the pytkdocs subprocess execute it. Maybe instead of calling pytkdocs directly, we could call python and make it execute some Python code? I think poetry is doing something along these lines.

That seems like a great solution. Think it's something we could add and release quickly? Happy to help. Trying to get a nice doc setup for our django app but ran into this issue.
Was going to monkeypatch it for now, but I'd be happy to have a fix in an official release.
Another hack I found that could work is to have dummy directory called setup_django/
with an __init__.py file that sets up django, and then in one of the index.md files, reference ::: setup_django, but I'm unsure of the order in which the files are processed. Any thoughts on that in the meantime?

Nice hack! From my debugging session, I think pages are always processed in the same order by mkdocs, though it might not be the same order as the YAML nav one. I'm not sure about the order the markdown will be processed in, but my guess is from top to bottom :slightly_smiling_face:  Feel free to experiment with this and report back to tell us if it works :slightly_smiling_face:
I'll also gladly review a PR if you want to send one. We already have some things that need to be released so I could release a new version quickly.
The subprocess command would look like
cmd = custom_code  # to be retrieved from the config
                   # it must be a single string, and each Python command
                   # must be separated with a semi-colon

# finally, add pytkdocs code
cmd += "from pytkdocs.cli import main; main('--line-by-line')"

self.process = Popen(
            ["python", "-c", cmd],
            universal_newlines=True,
            stderr=PIPE,
            stdout=PIPE,
            stdin=PIPE,
            bufsize=-1,
            env=env,
        )

Special care should be put into making sure the custom code does not read any input or write any output! Not sure how :sweat_smile:
If you decide to work on this, before you start coding I'd like you to describe in details how the custom code would be retrieved (list of scripts in a YAML array? Python code as a YAML multi-line string / array?) so we can discuss about it :slightly_smiling_face:

Oh, I realize this would not work if pytkdocs is not available in the default python environment, like when installed with pipx...

Hey @pawamoy playing around with it on my local now. It looks like the Collector doesn't have access to any sort of config object until collect is called, but we would need it at __init__, which would mean updating the get_handler function to take the list of python commands that need to run. Obviously I'd call it something else like setup_commands,  so that it's not python specific. Then those optional setup_commands could be passed to any handler, and the handler could choose how it wants to execute them. Does that make sense?
In terms of the YAML config, I was thinking something like:
handlers:
    python:
        rendering:
            show_source: true
       setup_commands:
            - "import os"
            - "import sys"
            - "import django"
            - "django.setup()"

Thoughts?
Also, when just trying the base case you wrote above
cmd += "from pytkdocs.cli import main; main('--line-by-line')"

without any custom additional code, I seem to be getting a BrokenPipe error. Any ideas?

Update: realized I was building the args the wrong way. Got it to work

for the record, the args passed to the main command has to be a sequence rather than a str:
cmd += "from pytkdocs.cli import main; main(['--line-by-line'])"

but we would need it at init, which would mean updating the get_handler function to take the list of python commands that need to run

Oh, indeed :confused:
This is a bit more complex than I thought haha.
I like the idea of the setup_commands key, however I'm starting to think that writing lines of code directly in mkdocs.yml is not a good idea. Users could have complex setup code, and this would be very impractical (python code in YAML array :scream:? no syntax checks/highlighting? not testable/debuggable?). What do you think? If we parse the list as a list of scripts instead, then I'm not sure how they would be executed in the cmd we're building. It boils down to: "what's the best way to pre-execute code in a (non-interactive) Python interpreter"

Then those optional setup_commands could be passed to any handler, and the handler could choose how it wants to execute them. Does that make sense?

It makes sense! I would generalize and say that any key present in a handler configuration that is not "rendering" or "collection" would be passed to its respective Handler class. However each handler would be completely free to accept any extra key it wants, and here, our Python handler would accept setup_commands (therefore you could give it a more Python-specific name if you want).

Hi @pawamoy, I posted a PR (#90) that can serve as a POC of this functionality. I have it working locally. Want to move our conversation to that PR so I can make any changes you deem necessary, and we can hopefully get it to a mergeable state? Thanks

Users could have complex setup code, and this would be very impractical (python code in YAML array 😱? no syntax checks/highlighting? not testable/debuggable?). What do you think? If we parse the list as a list of scripts instead, then I'm not sure how they would be executed in the cmd we're building. It boils down to: "what's the best way to pre-execute code in a (non-interactive) Python interpreter"

If someone has a more complex setup than the one I mention in the PR doc, they can always separate it into another python file and import and call that python file (as long as the file is in the PYTHONPATH, this should be fine). Thoughts?


[RFC] Improve compatibility with legacy versions

Because why not… :-)
Now more seriously: This project is a low level lib that is used as a building block for quite a few higher level abstractions on top of it. As such, compatibility (even with significantly outdated versions) is a major concern to me.
Note that I'm not suggesting putting significant amount of work into this. The patch is already here and, personally, I see little harm in supporting this.
Also note that I'm not suggesting we need to keep support indefinitely. Should this ever turn out to be a burden in the future, e.g. because we actually require any new language features or some external lib, then I'm all for dropping support again.

:+1:  from me, seeing how little effort goes into it it's a non issue for me. Personally I don't like supporting EOL versions but is doesn't require any real effort to keep it up

Sounds good, little reason not to. I agree if we ever need new language features we can drop this (and 5.4) again.


refactor(NODE-3337): unwrap-WriteConcernError

Description
Unwrapped the nested errors seen in handleMongoWriteConcernError to be clearer and readable. The nesting was made possible through prop flexibility in MongoServerError. Previously, MongoServerError assigned any prop from its parameter onto itself which is why nesting occurred. Moving forward, MongoServerError should explicitly capture the props it needs to prevent future nesting.
Currently, there is an expected test failure at test/functional/core/error.test.js:39. The test checked if any props in the params for MongoServerError's constructor would be assigned onto the object, which is now a removed behavior.

@andymina we should be careful about removing the server error behavior of assigning all input object props to the error object; the function of the server error class is to wrap errors returned from the server, and I believe the original design was based around the idea that we might not always know what those properties are, so we just want to pass them on to the user so that they have that information from the server; removing this behavior is a potentially breaking change. The goal of the ticket was just to remove the explicit rewrapping of the write concern error, not necessarily change the actual implementation of the MongoServerError class.

@andymina we should be careful about removing the server error behavior of assigning all input object props to the error object; the function of the server error class is to wrap errors returned from the server, and I believe the original design was based around the idea that we might not always know what those properties are, so we just want to pass them on to the user so that they have that information from the server; removing this behavior is a potentially breaking change. The goal of the ticket was just to remove the explicit rewrapping of the write concern error, not necessarily change the actual implementation of the MongoServerError class.

Ahh, didn't realize that there were potentially more/new props coming in from the server. Initially I figured that the shape of MongoServerErrors that the driver needs would remain the same so I just explicitly retrieved them rather than relying on the loop. So sorry for any confusion! Will revert this change, ty :)

Neal requested reviews from my laptop lol


Use gradle-build-action for CI builds

Continuation of PR #6509 (by @Goooler) that ended up being auto-closed when I accidentally created and merged PR #6510 in this repository instead of the one I use for testing.
I made some additional changes to get better Gradle cache usage.

Thanks, @Goooler :+1:


Installing coverage deletes Pipfile.lock without any warning

Issue description
Installing coverage.py with pipenv removes Pipfile.lock without any warning.
Expected result
Pipfile.lock updated with new package, and still existing.
Actual result
$ pipenv install coverage --dev
Installing coverage…
Looking in indexes: https://pypi.python.org/simple
Collecting coverage
  Using cached https://files.pythonhosted.org/packages/2c/c0/8047b7cbbcdbd7d21f8d68126196b7915da892c5af3d1a99dba082d33ec0/coverage-4.5.1-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: coverage
Successfully installed coverage-4.5.1

Adding coverage to Pipfile's [dev-packages]…
Pipfile.lock (c2b9f3) out of date, updating to (7ba9e7)…
Locking [dev-packages] dependencies…

Steps to replicate
The Pipfile:
[[source]]
verify_ssl = true
name = "pypi"
url = "https://pypi.python.org/simple"

[dev-packages]
django-cors-headers = "*"
"flake8" = "*"
pycodestyle = "*"

[packages]
django = "*"
lti = "*"
"psycopg2-binary" = "*"
oauthlib = "*"
djangorestframework = "*"
"4acce62" = {path = "./apiwrapper-0.2.1-py3-none-any.whl"}

[requires]
python_version = "3.5"

The command:
pipenv install coverage --dev

My environment:

Python 3.5.2
pipenv, version 11.9.0
Ubuntu 16.04

This has been fixed for quite a while. You need to upgrade Pipenv.


Sidebar search no longer a feature?

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.91.1
OS Version: Windows_NT x64 10.0.22631

Steps to Reproduce:

Open the "Open Editors" in the sidebar of vscode.
Click on a folder.
Press ctrl+f
The search bar for the sidebar doesn't popup. It used too.

I used to be able to really quickly search for a file in the sidebar, but now I can't. Maybe that is a setting that got marked as off as default in a recent update or something. Idk.

A couple of releases back its shortcut became Ctrl+Alt+F

A couple of releases back its shortcut became Ctrl+Alt+F

Awesome, thanks!


Rule: check if initialization files does not contain restricted element

Typically __init__.robot file cannot contain Test Cases section and some settings are not supported

It's already covered by ParsingErrorChecker. Before scanning files we run recognize_file_types() which detects if the file is Resource type (it's only if it's imported by other file). File type is used to select Robot parser type -> if we select resource parser for file that contains test cases, it will throw Robot parsing error and it will be caught by ParsingErrorChecker


Remove API directory

We will now use github.com/giantswarm/policy-api as a centralized API

Is the build fail expected right now?

It is passing now 😄


Infinite loop bug in deliveryAck

There is an infinite loop bug in deliveryAck as retryCount isn't decreased. If RabbitMQ is down then deliveryAck will retry indefinitely and consume a lot of CPU resources.

Thanks for the PR! LGTM!


Search: filter result dropdowns should stack on mobile?

 Makes the filtering experience a little difficult as the visitor has to zoom out.
Device: iPhone 6
Browser: Chrome/Safari

Good find! Fixed in 27be0c91f2000729f1a730e802a203a5fccebe26.

Looks good!


Add rotate_toward to Vector2, Vector3, Basis and Quaternion

closes https://github.com/godotengine/godot-proposals/issues/7965
continuation of https://github.com/godotengine/godot/pull/82699
Implements a rotate_toward to Vector2, Vector3, Basis and Quaternion, working practically the same in all of them. The method works like slerp (and uses it internally), except taking a radian increment as an amount instead of a percentage. As such the behavior should be consistent with it.
Vector2 and Vector3 have two edge cases:
When both inputs are colinear (also counts when the length of p_from is equal to zero), it will perform a move_toward, with delta multiplied by the length of p_to.
When the length of p_to is equal to zero, the best we can do is fallback to a lerp, in order to still try reaching the destination.
I wrote a few tests for the Vector2 and Vector3 implementations, only specific ones as the main logic happens in a slerp, which is already tested extensively. I cannot write tests for Quaternion and Basis, as I am not good with working with them, and tests for their slerp functions would have to be written first anyway.

Please add co-authorship as the code is largely the same as #82699

Please use rebasing to update your branch instead of merge commits, it messes with the history (and you'll need to squash it at the end anyway), see here

This was done as last resort as I couldn't update the branch, I still struggle with git workflows.

I'm trying to get some review on this as this was partially my code someone else has to review.

In summary, it should be fixed the following:

[ ] Add float rotate_toward()
[ ] delta must only be treated as radian
[ ] Negative delta must be rotated by longest path

@TokageItLab What does it mean to rotate a float?

[ ]  Add float rotate_toward()

I already added that last year https://github.com/godotengine/godot/pull/80225.

What does it mean to rotate a float?

The method was originally called move_toward_angle(), since it's the lerp_angle() equivalent of move_toward(). After some discussion it was renamed to rotate_toward() because it's only useful when dealing with rotations, and you "can't move towards an angle".

@aaronfranke It treats from, to, and delta as radians and rotates to the target in the TAU for the radian.

I already added that last year https://github.com/godotengine/godot/pull/80225.

@ettiSurreal Yes, I think this behavior is correct. So if delta is negative, the rotation should rotate to select the longest path, so if rotate_toward is implemented for other classes, it should be consistent with that.

@ettiSurreal I was unable to do that review https://github.com/godotengine/godot/pull/80225 at the time. The direction of rotation is correct, but the target seems to be implemented incorrectly I sent the issue, see https://github.com/godotengine/godot/issues/99083.

Updated relevant methods based on comments and discussion in https://github.com/godotengine/godot/issues/99083. Will update the patch hopefully tomorrow when I have time to mess with git again.
func rotate_toward_vec2(from: Vector2, to: Vector2, delta: float, keep_length: bool = false) -> Vector2:

		var angle: float = absf(from.angle_to(to));

		if keep_length:
			if angle < delta:
				return to.normalized() * from.length()
			return from.normalized().slerp(to.normalized(), delta / angle) * from.length()
		if angle < delta:
			return to
		return from.slerp(to, delta / angle)

func rotate_toward_quaternion(from: Quaternion, to: Quaternion, delta: float) -> Quaternion:
		if delta < 0.0:
			from = Quaternion(-from.x, -from.y, -from.z, -from.w)
		var angle: float = from.angle_to(to)
		if angle < delta:
			return to
		return from.slerp(to, delta / angle)

After trying to find how to find the longest path in a quaternion, apparently you can achieve that by simply negating one of the inputs? This seems to work after testing, at the point where it starts to oscillate due to switching which path is the longest, the dot product between the quaternions is around 0.
Though there is an issue with both methods, when you try to pass in a negative delta when from and to are the exact same, the method will return nan on all returned components.

Inverting the phase may indeed indicate a rotation of the longest path, but there may be a problem if the user arbitrarily inverts the phase and passes it as an argument.
You can align the phases by interspersing the Basis transformation once, as we did in https://github.com/godotengine/godot/pull/63593/files#diff-484517e0d5814647ff6ed5a31f35e6c6b758895e77acea104fe0515ff2694034R196-R200. However, from a performance standpoint, it may be better to invert only the first check of slerp.

@ettiSurreal Yes, I think this behavior is correct partially. So if delta is negative, the rotation should rotate to select the longest path, so if rotate_toward is implemented for other classes, it should be consistent with that.

when delta is negative, once the angle crosses the 180º limit, now the longest path is the oposite direction, so it would lock pointing to the oposite direction forever 🤔

@MarianoGnu As I mentioned in https://github.com/godotengine/godot/issues/99083#issuecomment-2468990393, if you want that behavior, just invert the target, not delta


delete edge with subquery does not work nor gives exception

on GratefulDeadConcerts database, orientdb version 2.1.3
I've tried the following actions in sequence:
SELECT FROM #11:5
DELETE EDGE FROM (SELECT FROM #11:5)
SELECT FROM #11:5
none of them issued an error.
first returned as expected, edge #11:5
second returned:
{
"result": [
{
"@type": "d",
"@version": 0,
"value": 0
}
],
"notification": "Query executed in 0.019 sec. Returned 1 record(s)"
}
the third returned again the edge #11:5 which to me looks like the second statement did not work as I would have expected since there was no error message that would have implied it is not supported.
Please let me know if there is something else I can do to help reproducing/understanding the issue.
by the way, works fine with DELETE VERTEX variant.

Hi @mcculls
In REMOVE EDGE syntax, the FROM indicates the starting point vertex of the edge you are trying to delete, so for example the following is valid:
DELETE EDGE FROM (SELECT FROM #9:1) TO (SELECT FROM #9:2)

To do what you are trying to do, the right syntax is
DELETE EDGE E where @rid in (SELECT FROM #11:5)

I guess some docs improvement is needed here http://orientdb.com/docs/last/SQL-Delete-Edge.html , I'm doing it right now
Thanks
Luigi


Is there a way to use electron.remote methods?

They all seem to be blocked for security reasons.
Is there any way to use them, through vscode extensions maybe? Loading securely?

No, extensions cannot use them  and we are actively moving away from using them anywhere in VS Code

No, extensions cannot use them and we are actively moving away from using them anywhere in VS Code

why? moddability is core to open-source.
You just end up with hacks like injecting into main.js of vscode .. ?


Text Detection doesn't work at all

In the following block of code. The app never moves into the 'if block' always returns  Log.d(TAG, "text data is null");
Note: Using the sample project ocr-reader as exact
if (graphic != null) {
            text = graphic.getTextBlock();
            if (text != null && text.getValue() != null) {
                Intent data = new Intent();
                data.putExtra(TextBlockObject, text.getValue());
                setResult(CommonStatusCodes.SUCCESS, data);
                finish();
            }
            else {
                Log.d(TAG, "text data is null");
            }
        }
        else {
            Log.d(TAG,"no text detected");
        }

Device info

Android version: 7.1.2
Baseband version: 7_GENNS_PACK-1.105949.1.106939.3
Kernel version: 3.18.31-perf-g8ea9a50 queen@scmbuild #1
Build number: HOLLAND2_999A_V1_0_73
Manufacturer: 10OR, E

Device Log
10.0r-device-log.txt
Object of scan [My keyboard]  :P

How to unlock bootloader of this device? It's bootloader is unlockable or not? Please tell!


Remove Public egress example because it is NOT supported

As the title says.
Remove public egress example as it's not supported and will continue to be misused.

Any idea how to make it work ? @timoreimann


Stable doc search is broken

Before submitting the issue

[X] I have checked for Compatibility issues
[X] I have searched among the existing issues
[X] I am using a Python virtual environment

Description of the bug
Search in stable doc is broken, see https://dpf.docs.pyansys.com/version/stable/index.html
The loading icon keeps turning but nothing ever appears.
Dev doc and previous versions docs are working fine.
Steps To Reproduce
The deployment job to look at is: https://github.com/ansys/pydpf-core/actions/runs/8661875996/job/23752909974
A reference deployment job (for 0.11.0) can be found here: https://github.com/ansys/pydpf-core/actions/runs/7989894623/job/21817574944
I think the problem may be with the meilisearch for stable.
Which Operating System causes the issue?
Windows, MacOS, Linux
Which DPF/Ansys version are you using?
DPF Server 2024.2.pre1
Which Python version causes the issue?
3.11
Installed packages
NA

Hi @Revathyvenugopal162, the meilisearch bar stopped working on our stable doc, (it hangs indefinitely) do you know what could be causing this?
The stable index deployment job seems to have worked fine as far as I can tell... Is it possible it is a problem with the Meilisearch service?

Hi @Revathyvenugopal162, the meilisearch bar stopped working on our stable doc, (it hangs indefinitely) do you know what could be causing this? The stable index deployment job seems to have worked fine as far as I can tell... Is it possible it is a problem with the Meilisearch service?

sure, let me check.

This is working now. Should we close the issue?

Hi @jorgepiloto , @Revathyvenugopal162 , I guess the latests releases fixed that, most likely due to upgrades in the pyansys actions used. Closing this issue? Thanks both!


Group can have multiple primary admins.

Coverage increased (+0.04%) to 73.436% when pulling 352e004ff7d5a60db0222d0e4e077d3f054a45cb on group-admins into b3d1fb224b9fb82889b5a867a370984150f2c898 on master.

Coverage increased (+0.04%) to 73.459% when pulling b7703caa90b07357fb725642d575726cc7bcf04a on group-admins into c97fb126b8dae5ac53b1f84b8566e8ffa722d7aa on master.