Build error

It looks the Ocean.js isn't being built correctly when it's used with Ocean sub graph:
 Error: Cannot find module<EMAIL_ADDRESS>    at createEsmNotFoundErr (internal/modules/cjs/loader.js:907:15)
    at finalizeEsmResolution (internal/modules/cjs/loader.js:900:15)
    at resolveExports (internal/modules/cjs/loader.js:432:14)
    at Function.Module._findPath (internal/modules/cjs/loader.js:472:31)
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:867:27)
    at Function.Module._load (internal/modules/cjs/loader.js:725:27)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/home/runner/work/ocean-subgraph/ocean-subgraph/test/integration/Datatokens.test.ts:5:1)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Module.m._compile (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/ts-node/src/index.ts:1056:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Object.require.extensions.<computed> [as .ts] (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/ts-node/src/index.ts:1059:12)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.exports.requireOrImport (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/mocha/lib/esm-utils.js:42:12)
    at Object.exports.loadFilesAsync (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/mocha/lib/esm-utils.js:55:34)
    at Mocha.loadFilesAsync (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/mocha/lib/mocha.js:473:19)
    at singleRun (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/mocha/lib/cli/run-helpers.js:125:15)
    at exports.runMocha (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/mocha/lib/cli/run-helpers.js:190:10)
    at Object.exports.handler (/home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/mocha/lib/cli/run.js:362:11)
    at /home/runner/work/ocean-subgraph/ocean-subgraph/node_modules/yargs/build/index.cjs:443:71
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>test-integration: `TS_NODE_PROJECT='test/integration/tsconfig.json' mocha --config=test/integration/.mocharc.json --node-env=test --exit 'test/integration/**/*.test.ts'`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>test-integration script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2021-04-19T01_07_27_888Z-debug.log
Error: Process completed with exit code 1.

Full build logs: https://github.com/oceanprotocol/ocean-subgraph/runs/2376793860?check_suite_focus=true

In ocean-subgraph, it's standard ocean.js usage:

https://github.com/oceanprotocol/ocean-subgraph/blob/main/package.json#L56
https://github.com/oceanprotocol/ocean-subgraph/blob/main/test/integration/Datatokens.test.ts#L5

I think the issue could be caused by the wrong name in the "exports" field within package.json: https://github.com/oceanprotocol/ocean.js/pull/754/files

I re-ran the build job in the Ocean Sub Graph and it's still giving the same error: https://github.com/oceanprotocol/ocean-subgraph/runs/2380269955?check_suite_focus=true

We need to release a new ocean.js first and bump it in subgraph.  Doing that now

Ok cool, thank you

https://github.com/oceanprotocol/ocean-subgraph/pull/89/checks?check_run_id=2381209889

if it helps @jamiehewitt15 👍
/ocean/ocean-subgraph$ ls node_modules/@oceanprotocol/lib/dist/node/
balancer/                   exchange/                   lib.d.ts                    lib.module.js               lib.umd.js.map              ocean/
datatokens/                 index.d.ts                  lib.js                      lib.module.js.map           metadatacache/              provider/
ddo/                        Instantiable.abstract.d.ts  lib.js.map                  lib.umd.js                  models/                     utils/
I can see lib.module.js  in node_modules/@oceanprotocol/lib/dist/node/ and not in node_modules/@oceanprotocol/lib/dist/

@alexcos20 Thank you for pointing that out. I have updated the paths here: https://github.com/oceanprotocol/ocean.js/pull/756/files

and still fails:  https://github.com/oceanprotocol/ocean-subgraph/pull/89/checks?check_run_id=2382858822
this time with another error

This is what I was suggesting on the calling, potentially trying to use the Common JS bundle rather than the ESM module: https://github.com/oceanprotocol/ocean.js/pull/761/files
I'll try and test it locally with Ocean sub graph

@alexcos20 I tested out pull request #761 with Ocean sub graph using npm link and the integration tests passed


robotjs does not install

Expected Behavior

for the module to install
Current Behavior

Possible Solution

fix
Steps to Reproduce (for bugs)

1.make it with a folder with unicode characters()
2.have same version of npm, nodejs, node-gyp and windows
3.install the module
4. get the error
Context

I cannot install it
Your Environment

RobotJS version:latest for 28(d).10(m).2020
Node.js version:v12.18.2
npm version:v6.14.5
Operating System:Windows 10 build 18362 x64

This issue will be closed, i found out its a node-gyp issue, not a robotjs issue

How to solve it ?

How to solve it ?

this is so old i actually forgot how i fixed it.


Fix Configure death handler

A couple of details weren't quite right.

I applied the patch and it still doesn't work.

Well, uhmmm, I didn't really expect it to.  However, this change makes Configure play nicer with Carp::Always, which can then help writing stack traces any time die is called, and thereby maybe help us figure out what your issue is.
For everyone else listening, this is related to this discussion on openssl-project

... which, incidently, wasn't a breakage at all, just [ahem] rogue output ;-)

Merged.
master / 1.1.1:
eb807d5383fd228a5c4cf9afc2fec487e0d22cee Configure death handler: instead of printing directly, amend the message
88accfe6dccf904fec5a17db4a59cd2c4c480382 Configure death handler: remember to call original death handler
1a6c30029802179ebe0ec1eedfdc9d78bb6dc4dd Configure death handler: bail out early when run in eval block
1.1.0:
36e732b00c8a22d50f88ee59d8790d5667ff72e2 Configure death handler: instead of printing directly, amend the message
f65389acaad600e816dcd9c231ea70d505d027a4 Configure death handler: remember to call original death handler
64eae7475e6caa669cd6369770ba4853ecaff8d1 Configure death handler: bail out early when run in eval block


An example to use it for any log file.

Hi, Thanks for making such an amazing project.
I have trying to use it for my log files. I could parse log files to its equivalent csv files using Logparser by LogPAI, but I have no idea how to convert logs to sequence of number as you have in you ~/data/hdfs/ directory. Also, then how to use it for inference real time log file.
Could please you help me with it?

I will update the code and documentation on how to generate the sequence of number in the next few days.It will include two methods which depends on your own logs: Time sliding window and hard disk ID sequence window.

Thanks for your response.
Also, thanking you in advance for sharing your methods to generate sequence of numbers from any log file as almost every log anomaly detection repo uses hdfs logs with pre-computed sequence of numbers from log message and I could not figure out how to use it for every log file case.

Hope it will help you :)
Example of how to sample your own log

Thanks alot for sharing your expertise.
I've gone through your code, I get your idea as (I am putting minute details so that it could be helpful to someone in future.)

First gather all the logs which was obtained from normal execution of application, i.e., logs without errors.
Combine these logs and convert to _structured.csv and _template.csv file using drain from logpai.
Train the model using obtained _structured.csv from step 2.
After successful training and saved model, it's time to test the model' s accuracy using a abnormal log file (log file with anomaly) and normal log file followed by inference of the model for the new logs files.
To implement step 4, since log files will be different in the sequence of events, so obtaining _structured.csv and _template.csv file using drain will not make any sense as randomly generated event_id will be completely different for an event from generated event_id for same event from the log file used for training. So, you proposed structure_bgl.py, using which I can generate event_id for completely new logs based on the event_id of the logs used for training using the generated event_template. Further, sample_bgl.py will convert the structured log into sequence of event_id which can further be replaced by its equivalent integer and thus testing can be performed.
Further to inference the model, new log line or logs lines in particular time window can be mapped with training file's event_template to obtain event_id.

Could I figured it out correctly?
Please feel free to correct me if I failed to describe your approach.
Thanks for your time.

What I do in structure_bgl.py is just to use the template extracted by drain to map the log file to event_id and extract the time and other information to be used next part.
This part can be said that it is just the meaning of data cleaning?
It seems to be no problem with the other parts you mentioned.

TBH, this is exactly what I meant, using structure_bgl.py, one can use the template extracted by drain as used during training to map new log files to event_id  of that template.

Hi! Thank you for posting this amazing project and thank you @kartikeyporwal  for opening this issue here! I also got a couple of questions about using the model on my own datasets. And here is my understanding of the workflows here:

Take HDFS raw log dataset as an example, I first need to transform it into a structured log dataset using the LogParser. And I will ended up getting this two files _structured.csv and _template.csv
And to get the training data and test data that look like the hdfs_train, hdfs_test_normal and hdfs_test_abnormal from the structured log dataset that I got from step 1, I will need to first do the sampling to generate the sequence of number as you stated in the example of how to sample your own log.  Then after having the event sequences, I will need to do the train test split mannually to get the three datsasets that listed above.
After having the datasets, we can perform the model (e.g. deeplog.py) training on the hdfs_train file that uses the sliding window sampling methods to generate sequence vector, count vector and semantic vector to train the deep learning model. And we can choose our own combination of the feature vectors that we wanna use.
Lastly, use the saved model to do inference on the test dataset.

And the questions I have based on the workflow that I describe above are:

When generating the semantic vector, it used a event2semantic_vec.json file that contains total 0-28 events number indicator mapping to different vectors. I guess this file is specifically generated for HDFS dataset to correspond to each eventID in HDFS, right? And how can we generate such json file if we are using our own log data?
And I'm also a little bit confused about the two sampling parts in step 2 and step 3. My understanding is that the sampling method happened in step2 is to generate sequence of event like we see in the hdfs_train file. And I believe it also depends on which window type you choose, right? And based on your sampling example for HDFS, I think that is session window sampling which is the same method used in loglizer dataloader.py. And for the sampling method in the sample.py, this is mainly for generating the feature vectors, right?

Feel free to correct me if there is anything wrong!! Look forward to any feedback!! :)

hi @cherishwsx
1.
I just use the Facebook open source fasttext pre-trained word vector model to extract the Word vector and use tf-idf method to generate Sentence vector for a log(correspond to each enentID).
If you are interested I can upload my code as a reference
2.
LSTM training process input sequence length needs to be fixed
You are right, first sampling origin HDFS dataset by session window, then use sliding window for generating the feature vectors(count vector and sequence vector)
If you use BGL dataset, just sampling origin data by sliding window and can just use it for generating the feature.
In robustlog (supervised learning), I just use a fixed sequence length method(crop and pad) to train lstm in the code......

Thank you for the reply!!

It would be great if you can upload your code! I really appreciate that!
Do you mean the sequence length (28 in your case since this number will be different depend on which parser tool) that you use to initialize the some of the feature vector length?

I mean the length of sequence [5 5 5 22 11 9 11 9 11 9] is fixed as 10 in deeplog and loganomaly.
Example:
sequence [5 5 5 22 11 9 11 9 11 9]
sequence_vector=[5,5,5,22,11,9,11,9,11,9]
count_vector=[0]*28
count_vector[5] = 3
count_vector[9] = 3
count_vector[11] = 3
count_vector[22] = 1

28 is just the number of template(Ground truth, not parsing by myself) of the HDFS dataset.

In this case, I think you are refering to the winow_size parameter (default is 10), correct? For example, if the window_size is set to 20, then the length of sequence will be 20. And the sequence_vector and count_vector (like the code chunk you showed above) will be created based on the length 20 sequence.

I just use the Facebook open source fasttext pre-trained word vector model to extract the Word vector and use tf-idf method to generate Sentence vector for a log(correspond to each enentID).
If you are interested I can upload my code as a reference
@donglee-afar I'm very interested about how to get the event2semantic_vec.json file. Could you please upload the code? Thank you very much!

hi @cherishwsx
1.
I just use the Facebook open source fasttext pre-trained word vector model to extract the Word vector and use tf-idf method to generate Sentence vector for a log(correspond to each enentID).
If you are interested I can upload my code as a reference
2.
LSTM training process input sequence length needs to be fixed
You are right, first sampling origin HDFS dataset by session window, then use sliding window for generating the feature vectors(count vector and sequence vector)
If you use BGL dataset, just sampling origin data by sliding window and can just use it for generating the feature.
In robustlog (supervised learning), I just use a fixed sequence length method(crop and pad) to train lstm in the code......

Hi,
Can you please upload the code that is used to generate the event2semantic_vec.json file?
Thanks,
Deep

Hi @donglee-afar,
Please give me a hint for creating the event2semantic_vec.json file. Thank you

Hi @donglee-afar,
How to use fasttext generate event2semantic_vec.json file?  Look forward to any feedback!  Thank you!!!

# -*- coding: utf-8 -*-
"""
Created on Mon Dec 23 10:54:57 2019

@author: lidongxu1
"""
import re
import spacy
import json

def data_read(filepath):
    fp = open(filepath, "r")
    datas = []  # 存储处理后的数据
    lines = fp.readlines()  # 读取整个文件数据
    i = 0  # 为一行数据
    for line in lines:
        row = line.strip('\n') # 去除两头的换行符，按空格分割
        datas.append(row)
        i = i + 1
    fp.close()
    return datas

def camel_to_snake(name):
    """
    # To handle more advanced cases specially (this is not reversible anymore):
    # Ref: https://stackoverflow.com/questions/1175208/elegant-python-function-to-convert-camelcase-to-snake-case
    """
    name = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
    return re.sub('([a-z0-9])([A-Z])', r'\1_\2', name).lower()

def replace_all_blank(value):
    """
    去除value中的所有非字母内容，包括标点符号、空格、换行、下划线等
    :param value: 需要处理的内容
    :return: 返回处理后的内容
    # https://juejin.im/post/5d50c132f265da03de3af40b
    # \W 表示匹配非数字字母下划线
    """
    result = re.sub('\W+', ' ', value).replace("_", ' ')
    result = re.sub('\d',' ',result)
    return result
# https://github.com/explosion/spaCy
# https://github.com/hamelsmu/Seq2Seq_Tutorial/issues/1
nlp = spacy.load('en_core_web_sm')
def lemmatize_stop(text):
    """
    https://stackoverflow.com/questions/45605946/how-to-do-text-pre-processing-using-spacy
    """
#    nlp = spacy.load('en_core_web_sm')
    document = nlp(text)
    # lemmas = [token.lemma_ for token in document if not token.is_stop]
    lemmas = [token.text for token in document if not token.is_stop]
    return lemmas

def dump_2_json(dump_dict, target_path):
    '''
    :param dump_dict: submits dict
    :param target_path: json dst save path
    :return:
    '''
    class MyEncoder(json.JSONEncoder):
        def default(self, obj):
            if isinstance(obj, bytes):
                return str(obj, encoding='utf-8')
            return json.JSONEncoder.default(self, obj)

    file = open(target_path, 'w', encoding='utf-8')
    file.write(json.dumps(dump_dict, cls=MyEncoder, indent=4))
    file.close()

data = data_read('template.txt')
result = {}
for i in range(len(data)):
    temp = data[i]
    temp = camel_to_snake(temp)
    temp = replace_all_blank(temp)
    temp = " ".join(temp.split())
    temp = lemmatize_stop(temp)
    result[i] = temp
print(result)
dump_2_json(result, 'eventid2template.json')

# 单独保存需要用到的fasttext词向量
template_set = set()
for key in result.keys():
    for word in result[key]:
        template_set.add(word)

import io
from tqdm import tqdm

# https://github.com/facebookresearch/fastText/blob/master/docs/crawl-vectors.md
def load_vectors(fname):
    fin = io.open(fname, 'r', encoding='utf-8', newline='\n', errors='ignore')
    n, d = map(int, fin.readline().split())
    data = {}
    for line in tqdm(fin):
        tokens = line.rstrip().split(' ')
        data[tokens[0]] = map(float, tokens[1:])
    return data

fasttext = load_vectors('cc.en.300.vec')

template_fasttext_map = {}

for word in template_set:
    template_fasttext_map[word] = list(fasttext[word])

dump_2_json(template_fasttext_map,'fasttext_map.json')

import os
import json
import numpy as np
import pandas as pd
from collections import Counter
import math

def read_json(filename):
    with open(filename, 'r') as load_f:
        file_dict = json.load(load_f)
    return file_dict

eventid2template = read_json('eventid2template.json')
fasttext_map = read_json('fasttext_map.json')
print(eventid2template)
dataset = list()
with open('data/'+'deepLog_hdfs_train.txt', 'r') as f:
    for line in f.readlines():
        line = tuple(map(lambda n: n - 1, map(int, line.strip().split())))
        dataset.append(line)
print(len(dataset))
idf_matrix = list()
for seq in dataset:
    for event in seq:
        idf_matrix.append(eventid2template[str(event)])
print(len(idf_matrix))
idf_matrix = np.array(idf_matrix)
X_counts = []
for i in range(idf_matrix.shape[0]):
    word_counts = Counter(idf_matrix[i])
    X_counts.append(word_counts)
print(X_counts[1000])
X_df = pd.DataFrame(X_counts)
X_df = X_df.fillna(0)
print(len(X_df))
print(X_df.head())
events = X_df.columns
print(events)
X = X_df.values
num_instance, num_event = X.shape

print('tf-idf here')
df_vec = np.sum(X > 0, axis=0)
print(df_vec)
print('*'*20)
print(num_instance)
# smooth idf like sklearn
idf_vec = np.log((num_instance + 1)  / (df_vec + 1)) + 1
print(idf_vec)
idf_matrix = X * np.tile(idf_vec, (num_instance, 1))
X_new = idf_matrix
print(X_new.shape)
print(X_new[1000])

word2idf = dict()
for i,j in zip(events,idf_vec):
    word2idf[i]=j
    # smooth idf when oov
    word2idf['oov'] = (math.log((num_instance + 1)  / (29+1)) + 1)

print(word2idf)
def dump_2_json(dump_dict, target_path):
    '''
    :param dump_dict: submits dict
    :param target_path: json dst save path
    :return:
    '''
    class MyEncoder(json.JSONEncoder):
        def default(self, obj):
            if isinstance(obj, bytes):
                return str(obj, encoding='utf-8')
            return json.JSONEncoder.default(self, obj)

    file = open(target_path, 'w', encoding='utf-8')
    file.write(json.dumps(dump_dict, cls=MyEncoder, indent=4))
    file.close()

dump_2_json(word2idf,'word2idf.json')

import json
import numpy as np
from collections import Counter

def read_json(filename):
    with open(filename, 'r') as load_f:
        file_dict = json.load(load_f)
    return file_dict

event2template = read_json('eventid2template.json')
fasttext = read_json('fasttext_map.json')
word2idf = read_json('word2idf.json')

event2semantic_vec = dict()
# todo :
# 计算每个seq的tf，然后计算句向量
for event in event2template.keys():
    template = event2template[event]
    tem_len = len(template)
    count = dict(Counter(template))
    for word in count.keys():
        # TF
        TF = count[word]/tem_len
        # IDF
        IDF = word2idf.get(word,word2idf['oov'])
        # print(word)
        # print(TF)
        # print(IDF)
        # print('-'*20)
        count[word] = TF*IDF
    # print(count)
    # print(sum(count.values()))
    value_sum = sum(count.values())
    for word in count.keys():
        count[word] = count[word]/value_sum
    semantic_vec = np.zeros(300)
    for word in count.keys():
        fasttext_weight = np.array(fasttext[word])
        semantic_vec += count[word]*fasttext_weight
    event2semantic_vec[event] = list(semantic_vec)
def dump_2_json(dump_dict, target_path):
    '''
    :param dump_dict: submits dict
    :param target_path: json dst save path
    :return:
    '''
    class MyEncoder(json.JSONEncoder):
        def default(self, obj):
            if isinstance(obj, bytes):
                return str(obj, encoding='utf-8')
            return json.JSONEncoder.default(self, obj)

    file = open(target_path, 'w', encoding='utf-8')
    file.write(json.dumps(dump_dict, cls=MyEncoder, indent=4))
    file.close()

dump_2_json(event2semantic_vec,'event2semantic_vec_sameoov.json')

Hi @donglee-afar
What is the format of the "template.txt" file? Is the same as "templates.csv" file in the repository, which include EventId and EventTemplate, or the hdfs.log file?
Thanks

Hi @donglee-afar
What is the format of the "template.txt" file? Is the same as "templates.csv" file in the repository, which include EventId and EventTemplate, or the hdfs.log file?
Thanks

Hi,
These are the templates, you can dump the templates from the templates.csv file into this file as .txt. I am not sure but I think you can even use the templates.csv file but only templates column needs to be kept in that file as well as header row to be removed

@ZanisAli Thank you for your reply
I have another question: In the testing phase to predict "test_normal" and "test_abnormal", the "template.txt" file becomes update from the training phase or not? In the other words, is the "template.txt" file in the train the same as the test step?

@ZanisAli, Thanks for your helping
Is it possible to edit the "LogAnomaly" demo code to have semantic information?

Thanks @donglee-afar for this fantastic project and all the good work. also for @cherishwsx  the good summary.
" And to get the training data and test data that look like the hdfs_train, hdfs_test_normal and hdfs_test_abnormal from the structured log dataset that I got from step 1, I will need to first do the sampling to generate the sequence of number as you stated in the example of how to sample your own log. Then after having the event sequences, I will need to do the train test split mannually to get the three datsasets that listed above."
Could you please help elaborate the above a  bit more in regard with how to generate these three files hdfs_train, hdfs_test_normal and hdfs_test_abnormal ?
What I am trying to achieve here is to apply robustlog to Linux logs (e.g. syslogs). I first parse the syslogs with Drain to get the template, and then labelled the original syslogs by placing the "-" or otherwise(for abnormal) in the first field, after that, apply structure_bgl.py and sample_bgl.py to structure and sample the logs respectively, and I am stuck in the next step to train and validate the model, and do prediction with it, and that is where the above question comes from.
Would you please help here ?  Thanks a lot!

Sorry for the late reply， These are the three code snippets I wrote before, run them in orderI hope it will be useful to you！ @huhui ,@arunbaruah ,@nagsubhadeep, @Magical66 1.
# -*- coding: utf-8 -*-
"""
Created on Mon Dec 23 10:54:57 2019

@author: lidongxu1
"""
import re
import spacy
import json

def data_read(filepath):
    fp = open(filepath, "r")
    datas = []  # 存储处理后的数据
    lines = fp.readlines()  # 读取整个文件数据
    i = 0  # 为一行数据
    for line in lines:
        row = line.strip('\n') # 去除两头的换行符，按空格分割
        datas.append(row)
        i = i + 1
    fp.close()
    return datas

def camel_to_snake(name):
    """
    # To handle more advanced cases specially (this is not reversible anymore):
    # Ref: https://stackoverflow.com/questions/1175208/elegant-python-function-to-convert-camelcase-to-snake-case
    """
    name = re.sub('(.)([A-Z][a-z]+)', r'\1_\2', name)
    return re.sub('([a-z0-9])([A-Z])', r'\1_\2', name).lower()

def replace_all_blank(value):
    """
    去除value中的所有非字母内容，包括标点符号、空格、换行、下划线等
    :param value: 需要处理的内容
    :return: 返回处理后的内容
    # https://juejin.im/post/5d50c132f265da03de3af40b
    # \W 表示匹配非数字字母下划线
    """
    result = re.sub('\W+', ' ', value).replace("_", ' ')
    result = re.sub('\d',' ',result)
    return result
# https://github.com/explosion/spaCy
# https://github.com/hamelsmu/Seq2Seq_Tutorial/issues/1
nlp = spacy.load('en_core_web_sm')
def lemmatize_stop(text):
    """
    https://stackoverflow.com/questions/45605946/how-to-do-text-pre-processing-using-spacy
    """
#    nlp = spacy.load('en_core_web_sm')
    document = nlp(text)
    # lemmas = [token.lemma_ for token in document if not token.is_stop]
    lemmas = [token.text for token in document if not token.is_stop]
    return lemmas

def dump_2_json(dump_dict, target_path):
    '''
    :param dump_dict: submits dict
    :param target_path: json dst save path
    :return:
    '''
    class MyEncoder(json.JSONEncoder):
        def default(self, obj):
            if isinstance(obj, bytes):
                return str(obj, encoding='utf-8')
            return json.JSONEncoder.default(self, obj)

    file = open(target_path, 'w', encoding='utf-8')
    file.write(json.dumps(dump_dict, cls=MyEncoder, indent=4))
    file.close()

data = data_read('template.txt')
result = {}
for i in range(len(data)):
    temp = data[i]
    temp = camel_to_snake(temp)
    temp = replace_all_blank(temp)
    temp = " ".join(temp.split())
    temp = lemmatize_stop(temp)
    result[i] = temp
print(result)
dump_2_json(result, 'eventid2template.json')

# 单独保存需要用到的fasttext词向量
template_set = set()
for key in result.keys():
    for word in result[key]:
        template_set.add(word)

import io
from tqdm import tqdm

# https://github.com/facebookresearch/fastText/blob/master/docs/crawl-vectors.md
def load_vectors(fname):
    fin = io.open(fname, 'r', encoding='utf-8', newline='\n', errors='ignore')
    n, d = map(int, fin.readline().split())
    data = {}
    for line in tqdm(fin):
        tokens = line.rstrip().split(' ')
        data[tokens[0]] = map(float, tokens[1:])
    return data

fasttext = load_vectors('cc.en.300.vec')

template_fasttext_map = {}

for word in template_set:
    template_fasttext_map[word] = list(fasttext[word])

dump_2_json(template_fasttext_map,'fasttext_map.json')

import os
import json
import numpy as np
import pandas as pd
from collections import Counter
import math

def read_json(filename):
    with open(filename, 'r') as load_f:
        file_dict = json.load(load_f)
    return file_dict

eventid2template = read_json('eventid2template.json')
fasttext_map = read_json('fasttext_map.json')
print(eventid2template)
dataset = list()
with open('data/'+'deepLog_hdfs_train.txt', 'r') as f:
    for line in f.readlines():
        line = tuple(map(lambda n: n - 1, map(int, line.strip().split())))
        dataset.append(line)
print(len(dataset))
idf_matrix = list()
for seq in dataset:
    for event in seq:
        idf_matrix.append(eventid2template[str(event)])
print(len(idf_matrix))
idf_matrix = np.array(idf_matrix)
X_counts = []
for i in range(idf_matrix.shape[0]):
    word_counts = Counter(idf_matrix[i])
    X_counts.append(word_counts)
print(X_counts[1000])
X_df = pd.DataFrame(X_counts)
X_df = X_df.fillna(0)
print(len(X_df))
print(X_df.head())
events = X_df.columns
print(events)
X = X_df.values
num_instance, num_event = X.shape

print('tf-idf here')
df_vec = np.sum(X > 0, axis=0)
print(df_vec)
print('*'*20)
print(num_instance)
# smooth idf like sklearn
idf_vec = np.log((num_instance + 1)  / (df_vec + 1)) + 1
print(idf_vec)
idf_matrix = X * np.tile(idf_vec, (num_instance, 1))
X_new = idf_matrix
print(X_new.shape)
print(X_new[1000])

word2idf = dict()
for i,j in zip(events,idf_vec):
    word2idf[i]=j
    # smooth idf when oov
    word2idf['oov'] = (math.log((num_instance + 1)  / (29+1)) + 1)

print(word2idf)
def dump_2_json(dump_dict, target_path):
    '''
    :param dump_dict: submits dict
    :param target_path: json dst save path
    :return:
    '''
    class MyEncoder(json.JSONEncoder):
        def default(self, obj):
            if isinstance(obj, bytes):
                return str(obj, encoding='utf-8')
            return json.JSONEncoder.default(self, obj)

    file = open(target_path, 'w', encoding='utf-8')
    file.write(json.dumps(dump_dict, cls=MyEncoder, indent=4))
    file.close()

dump_2_json(word2idf,'word2idf.json')

import json
import numpy as np
from collections import Counter

def read_json(filename):
    with open(filename, 'r') as load_f:
        file_dict = json.load(load_f)
    return file_dict

event2template = read_json('eventid2template.json')
fasttext = read_json('fasttext_map.json')
word2idf = read_json('word2idf.json')

event2semantic_vec = dict()
# todo :
# 计算每个seq的tf，然后计算句向量
for event in event2template.keys():
    template = event2template[event]
    tem_len = len(template)
    count = dict(Counter(template))
    for word in count.keys():
        # TF
        TF = count[word]/tem_len
        # IDF
        IDF = word2idf.get(word,word2idf['oov'])
        # print(word)
        # print(TF)
        # print(IDF)
        # print('-'*20)
        count[word] = TF*IDF
    # print(count)
    # print(sum(count.values()))
    value_sum = sum(count.values())
    for word in count.keys():
        count[word] = count[word]/value_sum
    semantic_vec = np.zeros(300)
    for word in count.keys():
        fasttext_weight = np.array(fasttext[word])
        semantic_vec += count[word]*fasttext_weight
    event2semantic_vec[event] = list(semantic_vec)
def dump_2_json(dump_dict, target_path):
    '''
    :param dump_dict: submits dict
    :param target_path: json dst save path
    :return:
    '''
    class MyEncoder(json.JSONEncoder):
        def default(self, obj):
            if isinstance(obj, bytes):
                return str(obj, encoding='utf-8')
            return json.JSONEncoder.default(self, obj)

    file = open(target_path, 'w', encoding='utf-8')
    file.write(json.dumps(dump_dict, cls=MyEncoder, indent=4))
    file.close()

dump_2_json(event2semantic_vec,'event2semantic_vec_sameoov.json')

Thank you so much, @donglee-afar, for the excellent project and the code snippets for preprocessing.  I have used your example code snippets to create a gist for preprocessing (in my case is was Ubuntu system logs and I used a parser project based on SPELL as well as my own text normalization method).  This was mainly to create the event2semantic_vec.json semantics file for use with LogAnomaly method you've implemented.
Here is the gist in case it can help anyone (pls give feedback as you wish):  https://gist.github.com/michhar/388d037439da6114d67aa8f793293870
Best regards.

Hello buddy, I have the same problem as you. Can you help me? I hope you can reply to me when you see it. This is very important to me. Thank you!

谢谢@donglee-afar为了这个出色的项目和所有出色的工作。也为了了@cherishwsx 好的总结。
““为了从我从1”获得的结构化日志数据集中获得看起来看起来像hdfs_train，hdfs_test_test_normal和hdfs_test_abnormal的的示例。然而在获得事件序列之后，我将需要手动进行火灾车辆测试拆分以获取上面列表的三个数据集。”
关于如何生成这三个文件 hdfs_train、hdfs_test_normal 和 hdfs_test_abnormal，您能否帮助详细说明以上内容？
（syslogs syslogs）syslogs syslogs得到得到得到得到日志日志日志日志structure_bgl.py和sample_bgl.py应用于结构和样本分别是日志，我被困在下一步训练和验证模型，并用它做预测，这就是上面提到的问题的来源。
你能帮忙吗？多谢！

@gavine Hello buddy, I have the same problem as you. Can you help me? I hope you can reply to me when you see it. This is very important to me. Thank you!


Support for mongo 3

When possible, support for mongo version 3 and higher would be excellent.

Closing this as mongo 3 is now supported as of 0.3.0 of diagnostic-channel-publishers (from #50 ) and 1.2.0 of application insights


Proplems with initial setup on 1.3 rc1

I'm coming a long road of failures and help from openoms now here:
https://termbin.com/u0s1
Hardware:
PowerSupply:
Original Raspberry 4 power supply is to weak - it never passed the test so I sorted it out and only used the Apple USB-C - Powersupply - 29 Watt (MJ262Z/A) - cable = Apple usb-c cable that came with the powersupply.
Hardware is a Rasperry Pi 4 with 4GB Ram (produced in June)
USB 3.1 Conector Sabrent (EC-SS31)
SSD Samsung 1TB Pro 950
Hardware is a Rasperry Pi 4 with 4GB Ram (produced in Mai)
USB 3.0 Conector Sabrent (EC-SSHD)
SSD SanDisk SSD PLUS 1TB
Also try now a HDD Conector
USB 3.0 Inateck UA 1001 with external power supply
All ssd had been surfaced checked by Minitool Partition Wizard
SD Cards are several Samsung microSDHC pro Endurance 32gb
As the time I'm writing this... I have one bigger power supply from apple which I will switch just to be sure.
I build the software with:
-1.3 rc1 iso from rootzoll github
-1.3 from openoms github
all from scratch with the original raspian buster image
Flashing is done with balenaEtcher 1.5.52
Sync through Tor
IBD only from other nodes
In the blockchain sync I also get a flashing failure like another one from the Raspibolt telegram channel gets:

So I'm out of ideas what to do - it just get stucked in every combination of the setup process with a "bitcoind is not running" at some time.

one more issue pop up

The read-only file system error is probably caused by your previously insufficient power supply.  You may have to reformat your external hard drive and install/setup from scratch.

Sadly not. The last one was a fresh build with the biggest usb c power supply from apple.

Sadly not. The last one was a fresh build with the biggest usb c power supply from apple.

You're running Tor. And I see lots of warnings in your log like this:

Aug  4 08:29:25 CryptoSigmund tor[676]: Aug 04 08:29:25.000 [warn] /run/tor is not owned by this user (bitcoin, 1002) but by debian-tor (111). Perhaps you are running Tor as the wrong user?
Aug  4 08:29:25 CryptoSigmund tor[676]: Aug 04 08:29:25.000 [warn] Before Tor can create a control socket in "/run/tor/control", the directory "/run/tor" needs to exist, and to be accessible only by the user and group account that is running Tor.  (On some Unix systems, anybody who can list a socket can connect to it, so Tor is being careful.)

This warning/problem was reported here a while ago. Perhaps and old problem that cropped up again: https://github.com/rootzoll/raspiblitz/issues/402#issue-417282446

It's also an open issue here: https://github.com/rootzoll/raspiblitz/issues/592#issuecomment-516017690

Never tried it without tor couse this is not an option. But it is reasonable.
Could this also be related to the rpi 4 usb Problem https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=245931 ?

Pls close this.


Release/1.10

Testing

Trade-offs

Dependencies

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?


How to install on linux x86 ?

Hello!
Is it possible to run mkcert in x86 OS?
There is no pre-built binary for x86: https://github.com/FiloSottile/mkcert/releases
Brew needs x64.
Building from source code does not work also.
I tried to make pem-files on x64 system and then copy them to the x86 system,
but that does not work - browser does not like it very much: NET::ERR_CERT_AUTHORITY_INVALID
Adding certificate to the trusted also not work.
So, as I understand you need to run "mkcert -install" to make it work properly.
But how to run "mkcert -install" on the x86 ? :)
My OS is "Ubuntu 14 x86"

search "install cert into " in your favorite search engine.

How to install Mkcert:-
1)From your home folder, Need to wget the ?latest? release & specify where to put it ->
sudo wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.3/mkcert-v1.4.3-linux-amd64 -O /usr/local/bin/mkcert
2)Then change permissions:
sudo chmod +x /usr/local/bin/mkcert
3)Now cd into the folder if need be and run:
mkcert -install
4)Need to install local certificates for localhost:
mkcert localhost
In your VSCode Json file, the location of certificates may be specified otherwise they have to be home in a local folder.


[Event Request] report 1305 "Standard Sales - Order Conf." - OnHeaderOnAfterGetRecordOnAfterUpdateNoPrinted

🕮 Describe the request
Could you please add the following event and include the sender in this event?
trigger OnAfterGetRecord()
begin
    FirstLineHasBeenOutput := false;
    Clear(Line);
    Clear(SalesPost);
    [...]

    if not IsReportInPreviewMode() then
        CODEUNIT.Run(CODEUNIT::"Sales-Printed", Header);
    //+EVENT
    OnHeaderOnAfterGetRecordOnAfterUpdateNoPrinted(IsReportInPreviewMode, Header)
    //+EVENT

    CurrReport.Language := Language.GetLanguageIdOrDefault("Language Code");
    FormatAddr.SetLanguageCode("Language Code");

     [...]
end;

[IntegrationEvent(true, false)] //Include the sender (that we can access our own global variables of the request page)
local procedure OnHeaderOnAfterGetRecordOnAfterUpdateNoPrinted(ReportInPreviewMode: Boolean; var SalesHeader: Record "Sales Header")
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 11755.


Mastadon share button

Closes https://github.com/elifesciences/enhanced-preprints-issues/issues/585

Should these icons be on the same line?

@chris-huggins are you able to approve?
https://pr-533--epp.elifesciences.org/reviewed-preprints/80494

@nlisgo Looks good to me, thanks.

Should these icons be on the same line? This is at 320 breakpoint

@chris-huggins please raise this as a ticket if not working as expected with guidance on how it should be working. Thanks

@nlisgo Yes Dave is right. I've raised a ticket here:  https://github.com/elifesciences/enhanced-preprints-issues/issues/586


Following from the new installation from a clean OS - beside the policy error

Hi There,
I am not sure if this issue is critical or not
Running "createMultiTask:audioEvents" (createMultiTask) task
Creating lambda function DragonConnect-audioEvents

Creating lambda function failed
Message:  1 validation error detected: Value 'arn:aws:Liam::xxxxxxxxxxx:role/DragonConnect-Lambda-14b0' at 'role' failed to satisfy constraint: Member must satisfy regular expression pattern: arn:aws:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@-_/]+
Warning: Task "createMultiTask:audioEvents" failed. Use --force to continue.

Aborted due to warnings.
Please do help to look into the issue.
Thanks and Regards, Henry

I believe I know the issue - One MUST login as root account to perform the installation tasks... Not as an IAM account....


⚠ Thumbnails API Endpoint has degraded performance

In 2ae05f2, Thumbnails API Endpoint (https://thumbnails.roblox.com/v1/assets?assetIds=82738847&format=Png&isCircular=false&size=30x30) experienced degraded performance:

HTTP code: 200
Response time: 3848 ms

Resolved: Thumbnails API Endpoint performance has improved in aaa353b.


Add isArrayOfHashes feature

As title, I jsut implement the isArrayOfHashes method to help us to validate hashedArrays easily.
And it's related to issue #64.
And the test cases are for isArrayOfHashes method has been ready.

I think it can use here :).
It can validate the isArrayOfHashes before setting the transaction hashes.
I will keep searching where it can be use this method :smile: .


[Bug] [datasophon-service] yarn ha status acquisition

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
Operation failed: User root doesn't have permission to call 'getServiceState'
at org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.verifyAdminAccess(RMServerUtils.java:420)
at org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.verifyAdminAccess(RMServerUtils.java:385)
at org.apache.hadoop.yarn.server.resourcemanager.AdminService.checkAccess(AdminService.java:229)
at org.apache.hadoop.yarn.server.resourcemanager.AdminService.getServiceStatus(AdminService.java:382)
at org.apache.hadoop.ha.protocolPB.HAServiceProtocolServerSideTranslatorPB.getServiceStatus(HAServiceProtocolServerSideTranslatorPB.java:148)
at org.apache.hadoop.ha.proto.HAServiceProtocolProtos$HAServiceProtocolService$2.callBlockingMethod(HAServiceProtocolProtos.java:5415)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:528)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1086)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1029)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:957)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2957)
What you expected to happen
Regardless of whether acl is enabled or not, yarn ha status should be obtained normally.
How to reproduce
Get the value of yarn.admin.acl and su to this user to execute the command
Anything else
No response
Version
dev
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

if yarn.acl.enable is set to false , is it necessary?

We cannot control user behavior

HDFS status acquisition also involves similar issues

We cannot control user behavior
if yarn.acl.enable is set to false, Can your code run normally?


Input buffer has corrupt header: source input: Old-style JPEG compression support is not configured

Possible bug
Is this a possible bug in a feature of sharp, unrelated to installation?

[X] Running npm install sharp completes without error.
[X] Running node -e "require('sharp')" completes without error.

If you cannot confirm both of these, please open an installation issue instead.
Are you using the latest version of sharp?

[X] I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.
If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.
What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?
  System:
    OS: macOS 11.6.2
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 985.53 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 14.17.1 - ~/.nvm/versions/node/v14.17.1/bin/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 6.14.13 - ~/.nvm/versions/node/v14.17.1/bin/npm

What are the steps to reproduce?
Just tried to convert a tiff to png.
What is the expected behaviour?
Expecting the tiff image to be converted into png image
Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this problem
        const imgbuffer = "............"; // Buffer
        const buffer = Buffer.from(imgbuffer, 'base64');
        const bitmap = await sharp(buffer).tiff().png().toBuffer();

Please provide sample image(s) that help explain this problem

It looks like the TIFF input file you're trying to convert uses the "old style" JPEG compression, which has been deprecated since sometime in the 1990s, well over 20 years ago.
You might be able to compile your own libtiff with support for this, then compile libvips from source to use it, then compile sharp from source to use it - see https://sharp.pixelplumbing.com/install#custom-libvips


chore: fix release workflow

includes an arbitrary change to the values to trigger the release workflow

:tada: This PR is included in version 1.0.1-alpha.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add reopen method in PerThreadPKLookup

Description
Reuse loaded segments' termsEnum and postingsEnum.

@jpountz
Please take a look when you get a chance.

@jpountz
Please take a look when you get a chance.

I can't figure out  what can we reuse from prior PostingsEnum.
If we want reuse prior PostingsEnum's docBuffer( if it still stay in first block), we should not reset docBufferUpto to BLOCK_SIZE (maybe set to 0?) when prior docStartFP equals current's,  which will result in refillDocs.
For the situation of PK, every PK has different PostingsEnum, prior PostingsEnum is mostly useless if reuse means reuse same PostingsEnum loaded docBuffer.
@jpountz Please correct me if I am wrong.

This change looks clearer, Thanks @jpountz .


Fix Windows compilation warnings

There are warnings in Windows while compiling in CircleCI. It may require changes in the binding potentially.
https://app.circleci.com/pipelines/github/Consensys/jc-kzg-4844

No more Windows compilation warnings after https://github.com/Consensys/jc-kzg-4844/pull/38


:heart_eyes: load order after scan barcode

Are you sure that both updates are just :heart_eyes:  and not  :ambulance: or :zap: (depending on whether is it supposed feature or not)
:heart_eyes: =
before the commit it worked in expected way, but now it works better (quicker, nicer)


Support mmark markdown parser as a middleware

Hi Matt, thanks for this great project you've built.
I have an idea to build a static blog using Caddy, there is pretty much everything to do that well and it's not bloated in any way. However, the standard blackfriday Markdown parser has a vanilla set of Markdown features, I'd like to use much powerful (in terms of set of features included) implementations such as https://github.com/miekg/mmark by @miekg. Since it's a fork of blackfriday, I could merge it into Caddy just in a few hours, even the tests are pretty much the same.
https://github.com/xlab/caddy/tree/f/mmark — this branch works perfectly and along with built-in markdown middleware there is the mmark middleware. An example of Caddyfile:
http://localhost:8080

internal /templates

mmark /blog {
    ext     .md
    template blog /templates/blog.html
}

However, I've got some concerns about merging this branch into Caddy and sending a pull-request. This, in fact, should be a perfect case when it's better to have MMark separated as an add-on. But I can't figure out how to build such an add-on while keeping it integrated well as the original markdown middleware is. I'd like to have an advice :)
Or somebody could volunteer and separate this thing out into an add-on. I'll keep forked version until then. By the way, I can send a pull-request, if this is OK to have two integrated markdown implementations.

Thanks for your comments Maxim! And thanks for opening an issue before a PR for something like this.
Mmark looks neat, I hadn't seen it before. I can definitely see its utility, but I feel like it's more of an edge use case, and would make a better add-on than built into the web server.
Add-ons can't change built-in middleware, so the duplicated code (given that it's a blackfriday fork) would be unfortunate, and maintaining it will be your responsibility. But if you think more people than just yourself would really need/use mmark with Caddy, you can make it an add-on and add it to the website. https://github.com/mholt/caddy/wiki/Extending-Caddy
However, keep in mind that you can also just keep using your fork locally. Caddyext makes it easy to manage plugins on your local installation. This way there's no maintenance burden since it only has to work for you, and if this feature gets requested enough, we can point them to you.
Hopefully that makes sense :smile: I'll close this now but let me know if you have any other questions!


Do not hardcode ports in test.js

Currently test.js defaults to port 8000 if a port is not given.
But if I have something else running at port 8000, then I cannot use node make test because there is no way to change the port.
Can we set the port to 0, so that Node.js automatically chooses a free port? Then we can also run multiple tests next to each other (in theory).

There is not problem in moving test server to a different port, e.g. 5-digit number (auto-choosing a free port might introduce many unless code)

@yurydelendik Choosing a random port a la https://xkcd.com/221/ is not going to fly. I'll let the system choose a port (see PR #6620).


regression: table chart columns order and visibility

currently table chart display all columns which exist in response.
-> at the same time order is organized chaotically
Also earlier we had possibility to show limited number of columns from single huge query. It would be nice to restore somehow for table chart as well (maybe wizard template editor part)

fixed in 3.4.51+


Update test for Ruby 3.1 compatibility

pulled out of https://github.com/ManageIQ/manageiq/pull/22698

use qpid 0.37.0
fix queue spec to be more resilient

Backported to quinteros in commit 91d6aa2e0e839b49e1d426bfb1a0bcedd7ab6e14.
commit 91d6aa2e0e839b49e1d426bfb1a0bcedd7ab6e14
Author: Jason Frey<EMAIL_ADDRESS>Date:   Wed Nov 1 15:15:22 2023 -0400

    Merge pull request #22767 from kbrock/ruby31-tests

    Ruby31 tests

    (cherry picked from commit 6e66c89940f2378fe658fbc5834632f0bce9b959)


Custom messages in Chat

Can we get custom messages in chats, I mean block of html-code

I have solution with slots


Super User Logging Problem for Tenants

User logging in Successful for public tenant how to create view  for individual tenats I restricted in my django settings app can u have any solution for individual tenant login with separation from other tenants, Here is my settings app
SHARED_APPS = (
'tenant_schemas',  # mandatory, should always be before any django app
#'customers', # you must list the app where your tenant model resides in
'shared',
#'django.contrib.auth',
#'django.contrib.contenttypes',
#'django.contrib.sessions',
#'django.contrib.messages',
#'django.contrib.staticfiles',
#'django.contrib.admin',
'classroom',
'river',
# everything below here is optional
)
TENANT_APPS = (
#'django.contrib.auth',
'django.contrib.contenttypes',
#'django.contrib.sessions',
#'django.contrib.messages',
#'django.contrib.staticfiles',
#'django.contrib.admin',
#'tenantexample.tenant',
'classroom',
'shared',
'river',
#'accounts',
# your tenant-specific apps
)
INSTALLED_APPS = [
'tenant_schemas',
'django.contrib.auth',
'django.contrib.admin',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.humanize',
'crispy_forms',
'classroom',
'river',
#'accounts',
'shared',
]
In this any changes are required please do the need i am struggling in this issue

What is your specific question?

Don't understand the question?

What is your specific question?

I tried to login admin but it shows errors, and now i cleared this issue,


gitlint/git lint commits selection working wrong

SUMMARY

https://github.com/oasislabs/oasis-core/pull/2706/checks?check_run_id=461288544#step:7:184
that commit shouldn't be included. when I do gitlint --debug --commits origin/master...HEAD locally, it doesn't. something's wrong on CI where it is including it
ISSUE TYPE

Bug Report

COMPONENT NAME

github workflows
OASIS NODE VERSION

(n/a)
OS / ENVIRONMENT

(n/a)
STEPS TO REPRODUCE

push certain PRs. possibly related to merge commits
ACTUAL RESULTS

gitlint checks all historical commits

(too verbose. see action output in link)
EXPECTED RESULTS

should run on b05113a7f4862a31dbe6d62fc334d66817738cde, 269279fb77415d0cba4e8537fab944d6b91cba6e, 39f0a8432ac6314b971c4680d8e12f9329d912b5, 1e39860c6953fec12393f36e1535855f93f19dc3, and a069f99d0e64c603dfe079d596d4eebd643dfc37

oh it's because the changelog check truncates the worker's git history, which breaks the commit selection


Add -skip-cleanup option for publish commands.

Fixes #570.
Description of the Change
When  publishing multiple large snapshot (e.g. Ubuntu security snapshots) under the same prefix aptly becames very slow when switching and dropping publishes in that prefix, and the slow part is the "Cleaning up prefix "[...]" components main...".
In case of it is necessary to update/switch/drop multiple published repos/snapshots, a significant amount of time can be saved by performing the cleanup phase only after the last one operation.
This change allow skipping the cleanup of unreferenced files on publish switch/update/drop via the -skip-cleanup command line option. It also adds support for the API SkipCleanup parameter.
Checklist

[X] unit-test added (if change is algorithm)
[X] functional test added/updated (if change is functional)
[ ] man page updated (if applicable)
[X] bash completion updated (if applicable)
[ ] documentation updated
[X] author name in AUTHORS

Thanks!

Thank you for considering this change for merge!
I have used the web UI to resolve the conflict on AUTHORS.


Restart due to Google Drive Backup breaks iCloud integration

The problem

Every other day Home Assistant restarts without any visible issue, and after the restart every entity linked with the iCloud integration are not available: I have to delete the icloud folder in /hassio/homeassistant/.storage and restart in order to solve.
I tried to read the logs to find some errors, and I found that the first error (as you can see below) comes from binary_sensor.snapshots_stale which is created by the Google Drive Backup add-on for Hass.io (https://github.com/sabeechen/hassio-google-drive-backup).
I can't understand if and how these two problems are linked!
Environment

Home Assistant release with the issue: 0.105.2
Last working Home Assistant release (if known):
Operating environment (Hass.io/Docker/Windows/etc.): Hass.io in Docker on RPi4
Integration causing this issue: iCloud / Hass.io Google Drive Backup add-on
Link to integration documentation on our website: https://www.home-assistant.io/integrations/icloud/

Problem-relevant configuration.yaml

icloud:
  - username: USERNAME
    password: PASSWORD

Traceback/Error logs

From supervisor -> system
20-02-08 06:16:24 ERROR (MainThread) [hassio.homeassistant] Error on call https://<IP_ADDRESS>:8123/api/states/binary_sensor.snapshots_stale: Cannot connect to host <IP_ADDRESS>:8123 ssl:None [None]
20-02-08 06:16:26 ERROR (MainThread) [hassio.api.proxy] Error on API for request states/binary_sensor.snapshots_stale
20-02-08 06:32:07 ERROR (MainThread) [hassio.homeassistant] Error on call https://<IP_ADDRESS>:8123/api/states/binary_sensor.snapshots_stale: Cannot connect to host <IP_ADDRESS>:8123 ssl:None [Connection reset by peer]
20-02-08 06:32:07 ERROR (MainThread) [hassio.api.proxy] Error on API for request states/binary_sensor.snapshots_stale
20-02-08 06:32:08 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
20-02-08 06:32:09 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
20-02-08 06:32:09 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
20-02-08 06:32:13 WARNING (MainThread) [hassio.tasks] Watchdog found a problem with Home Assistant Docker!
20-02-08 06:32:13 INFO (SyncWorker_14) [hassio.docker.interface] Start homeassistant/raspberrypi3-homeassistant
20-02-08 06:32:14 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
20-02-08 06:32:14 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
20-02-08 06:32:14 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
20-02-08 06:32:19 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
20-02-08 06:32:19 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
20-02-08 06:32:19 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
20-02-08 06:32:24 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
20-02-08 06:32:24 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
20-02-08 06:32:24 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
20-02-08 06:32:29 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
20-02-08 06:32:29 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
20-02-08 06:32:29 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
20-02-08 06:32:34 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
20-02-08 06:32:34 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
20-02-08 06:32:34 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].
20-02-08 06:32:39 INFO (MainThread) [hassio.api.proxy] Home Assistant WebSocket API request initialize
20-02-08 06:32:39 INFO (MainThread) [hassio.api.proxy] WebSocket access from a0d7b954_nodered
20-02-08 06:32:39 ERROR (MainThread) [hassio.api.proxy] Client error on WebSocket API Cannot connect to host <IP_ADDRESS>:8123 ssl:False [Connection refused].

From developer tools -> logs
2020-02-08 07:32:25 WARNING (Recorder) [homeassistant.components.recorder] Ended unfinished session (id=124 from 2020-02-06 11:03:34.341178)
2020-02-08 07:32:33 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry<EMAIL_ADDRESS>for icloud
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 215, in async_setup
    hass, self
  File "/usr/src/homeassistant/homeassistant/components/icloud/__init__.py", line 124, in async_setup_entry
    await hass.async_add_executor_job(account.setup)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/icloud/account.py", line 124, in setup
    self.update_devices()
  File "/usr/src/homeassistant/homeassistant/components/icloud/account.py", line 168, in update_devices
    self._fetch_interval = self._determine_interval()
  File "/usr/src/homeassistant/homeassistant/components/icloud/account.py", line 239, in _determine_interval
    int(min(intervals.items(), key=operator.itemgetter(1))[1]),
ValueError: min() arg is an empty sequence

From the Hass.io Google Drive Backup add-on
02-06 09:38:08 INFO Starting server on port 8099
02-06 09:38:08 INFO Server started
02-06 09:38:08 INFO Syncing Snapshots
02-06 10:38:09 INFO Syncing Snapshots
02-06 11:38:11 INFO Syncing Snapshots
02-06 12:06:50 ERROR Trouble updating Home Assistant sensors.
02-06 12:06:50 ERROR
Traceback (most recent call last):
  File "/app/backup/haupdater.py", line 41, in update
    self._requests.updateSnapshotStaleSensor(self._stale())
  File "/app/backup/harequests.py", line 178, in updateSnapshotStaleSensor
    self._postHaData("states/binary_sensor.snapshots_stale", data)
  File "/app/backup/harequests.py", line 139, in _postHaData
    self._client.post(self.config.get(Setting.HOME_ASSISTANT_URL) + path, headers=self._getHaHeaders(), json=data).raise_for_status()
  File "/usr/lib/python3.8/site-packages/requests/api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3.8/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='hassio', port=80): Max retries exceeded with url: /homeassistant/api/states/binary_sensor.snapshots_stale (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xb4c046e8>: Failed to establish a new connection: [Errno 111] Connection refused'))

02-06 12:38:12 INFO Syncing Snapshots
02-06 13:38:14 INFO Syncing Snapshots
02-06 14:38:15 INFO Syncing Snapshots
02-06 15:38:17 INFO Syncing Snapshots
02-06 16:38:19 INFO Syncing Snapshots
02-06 17:38:20 INFO Syncing Snapshots
02-06 18:38:21 INFO Syncing Snapshots
02-06 19:38:22 INFO Syncing Snapshots
02-06 20:38:23 INFO Syncing Snapshots
02-06 21:38:25 INFO Syncing Snapshots
02-06 22:38:26 INFO Syncing Snapshots
02-06 23:38:28 INFO Syncing Snapshots
02-06 23:38:29 INFO Requesting a new snapshot
02-06 23:38:34 INFO Deleting 'Full Snapshot 2020-01-30 23:53:00' from Home Assistant
02-06 23:38:34 INFO Backup directory was written to, we'll reload snapshots from Hassio soon
02-06 23:38:39 INFO Backup directory changed
02-06 23:38:39 INFO Syncing Snapshots
02-06 23:40:05 INFO Backup directory was written to, we'll reload snapshots from Hassio soon
02-06 23:40:18 INFO Snapshot finished
02-06 23:40:18 INFO Syncing Snapshots
02-06 23:41:59 INFO Deleting 'Full Snapshot 2020-01-30 23:53:00' From Google Drive
02-06 23:42:00 INFO Backup directory changed
02-06 23:42:00 INFO Syncing Snapshots
02-07 00:42:01 INFO Syncing Snapshots
02-07 01:42:03 INFO Syncing Snapshots
02-07 02:42:04 INFO Syncing Snapshots
02-07 03:42:06 INFO Syncing Snapshots
02-07 04:42:07 INFO Syncing Snapshots
02-07 05:42:08 INFO Syncing Snapshots
02-07 06:42:09 INFO Syncing Snapshots
02-07 07:42:11 INFO Syncing Snapshots
02-07 08:42:12 INFO Syncing Snapshots
02-07 09:42:13 INFO Syncing Snapshots
02-07 10:42:15 INFO Syncing Snapshots
02-07 11:42:16 INFO Syncing Snapshots
02-07 12:42:18 INFO Syncing Snapshots
02-07 13:42:19 INFO Syncing Snapshots
02-07 14:42:20 INFO Syncing Snapshots
02-07 15:42:22 INFO Syncing Snapshots
02-07 16:42:24 INFO Syncing Snapshots
02-07 17:42:25 INFO Syncing Snapshots
02-07 18:42:27 INFO Syncing Snapshots
02-07 19:42:28 INFO Syncing Snapshots
02-07 20:42:29 INFO Syncing Snapshots
02-07 21:42:31 INFO Syncing Snapshots
02-07 22:42:32 INFO Syncing Snapshots
02-07 23:42:33 INFO Syncing Snapshots
02-07 23:42:34 INFO Requesting a new snapshot
02-07 23:42:39 INFO Deleting 'Full Snapshot 2020-01-31 23:51:41' from Home Assistant
02-07 23:42:40 INFO Backup directory was written to, we'll reload snapshots from Hassio soon
02-07 23:42:45 INFO Backup directory changed
02-07 23:42:45 INFO Syncing Snapshots
02-07 23:44:15 INFO Backup directory was written to, we'll reload snapshots from Hassio soon
02-07 23:44:29 INFO Snapshot finished
02-07 23:44:30 INFO Syncing Snapshots
02-07 23:46:14 INFO Deleting 'Full Snapshot 2020-01-31 23:51:41' From Google Drive
02-07 23:46:15 INFO Backup directory changed
02-07 23:46:15 INFO Syncing Snapshots
02-08 00:46:16 INFO Syncing Snapshots
02-08 01:46:17 INFO Syncing Snapshots
02-08 02:46:18 INFO Syncing Snapshots
02-08 03:46:19 INFO Syncing Snapshots
02-08 04:46:20 INFO Syncing Snapshots
02-08 05:46:21 INFO Syncing Snapshots
02-08 06:46:22 INFO Syncing Snapshots
02-08 07:32:07 ERROR Unable to reach Home Assistant (HTTP 502).  Is it restarting?
02-08 07:46:23 INFO Syncing Snapshots
02-08 08:46:24 INFO Syncing Snapshots
02-08 09:46:26 INFO Syncing Snapshots
02-08 10:46:27 INFO Syncing Snapshots
02-08 11:46:29 INFO Syncing Snapshots
02-08 12:46:30 INFO Syncing Snapshots
02-08 13:46:31 INFO Syncing Snapshots

Additional information
I can reach Hass.io from the internet with DuckDNS add-on

Hi @PRES1005 !
Are you sure you are on 0.105.2 ?
This error should already be fixed with #31533 🤔
Is it the first time you are using the iCloud integration ?
How many devices do you have into your iCloud account ? (with and without family sharing)
Do they have location activated ?
Are one of your device in zone when it happens ?
Thanks

You are right, I updated to 0.105.2 after the problem but before opening the issue, so I was on 0.105.1 when the problem occurred.
I had the same problem yesterday but I think it is only related to Google Drive Backup Add-on as the iCloud integration didn’t stop working this time...
I have only a device (7th gen iPad WiFi), now it’s working correctly but I have multiple 450 errors in the logs.

This issue #31465 was created to follow 450 error.
So maybe we can close this issue ?
Sent with GitHawk

Yes, sure. Sorry for the inconvenience and thank you for your help!


Fix the margin for the browse page

This PR fixes the padding of the browse page.
This PR also adds the nav bar with a hidden search to the browse page.
Fixes #312

I think there's a slight issue where the search bar gets too narrow in small-to-medium screen:
https://user-images.githubusercontent.com/17035406/186262754-bd20c516-8891-4b86-9cdf-1423fce11f13.mov


Invalid module specifier in package.json

In the package.json the "module" and the "exports.import" should both point to ./dist/vue3-mq.mjs instead of to ./dist/vue3-mq.js. The latter is not in ES module format.
In my case, this leads to problems when bundling with vite to the "system" output format.

This should be resolved in version 4 since the package uses "type": "module" which defaults to .js for ESM files.

This should be resolved in version 4 since the package uses "type": "module" which defaults to .js for ESM files.


Using old template

Flag for use program main doesn't appear in help and does not work.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 447e23ae-0256-d8c1-4dca-d31fe21602a5
Version Independent ID: 926db74c-be79-f7cd-a4e9-e156dafa6287
Content: C# console app template changes in .NET 6+ - .NET
Content Source: docs/core/tutorials/top-level-templates.md
Product: dotnet-fundamentals
GitHub Login: @tdykstra
Microsoft Alias: tdykstra

What version of the SDK are you using?


Bug: Typo in original localization

Description
The ToggleInfoPaneDescription has a typo inside the localization. The backslash should be a slash
  <data name="ToggleInfoPaneDescription" xml:space="preserve">
    <value>Toggle the info pane to view the detail\preview panes</value>
  </data>

Steps To Reproduce
See EN-US localization.
Requirements

Fix typo in EN-US resource file.

Files Version
3.0.14
Windows Version
10.0.19045.3324
Log File
N/A

I'm fine with it either way, do you want to open a PR?

This is resolved in main.


Use more concrete error class on plugin not found

Signed-off-by: Hiroshi Hatake<EMAIL_ADDRESS>
Which issue(s) this PR fixes:
None.
What this PR does / why we need it:
Not found plugin is basically config error but we need to use another error class to customize this error for config validation.
In general, cloud service does not permit to install plugins via user operations.
This patch does not affect for error message for ordinary cases.
Perhaps, it might affect error handling but this should be a corner case.
Docs Changes:
No needed.
Release Note:
Same as title.

Thanks!


fix: update post types and include system removals in skip logic

I believe that previously, these post types were grouped in these three categories, but have since been broken out to be more verbose.
I ran into this today as we have join/part as well as adding/removal from teams turned off, but found that when an Admin removes a user from the team, a different message type is generated.

Thanks!


Bug fix and some helper functions

bug fix in setPosition()
introduce rightTurn() / leftTurn() / clearScreen() (I remember suing these functions as a kid. )

Great!


generated code not compiling when inspected field is identical to history table field

I have a table that has a field named "status". the generated code by enthistory does not compile due to type mismatch
ent/history_from_mutation.go:139:29: cannot use status (variable of type a.Status) as dubshistory.Status value in argument to create.SetStatus

@Ohadbasan Thanks for letting us know, can you create an example repo to share with us so I can get a better idea about what's going on?

https://github.com/Ohadbasan/enthistory_example_bug
history_from_mutation won't compile

thank you! missed in the readme. sorry for that!


Tighten and IPv6-ify gen_subnet() etc

Tightens, canonicalises and improves for IPv6, the functions
gen_subnet(), gen_subnetv6(), gen_subnet_max(), gen_subnetv6_max()
Changes are transparent to calling code.
Issues:

gen_subnet() and gen_subnet_max() will validate both IPv4 and IPv6 as valid args, but will then try to process an IPv6 subnet bitwise as x32 LONG without further checking, causing erroneous but apparently valid responses.

None of the functions properly sanitise their input: if $bits is >32 or >128, or even a non-integer, erroneous results will be passed back to the calling code as valid data without checking, again causing erroneous but apparently valid responses.

3 of the 4 functions return an empty string for invalid but gen_subnetv6_max() returns a numeric value for invalid. Both responses loose-evaluate as False, but consistency is better.

Fixes and improvements:

The unspecified functions gen_subnet() and gen_subnet_max() now handle all args correctly, and don't mishandle if unexpectedly passed IPv6 or bad data.

Names are now canonical: gen_subnet(), gen_subnet_max() are now IPv4/v6 agnostic, and IPv4-only versions gen_subnetv4() and gen_subnetv4_max() are added as expected to exist, to match existing functions gen_subnetv6() and gen_subnetv6_max().

The return value for bad args is made consistent (empty string = False).

gen_subnetv6_max() now uses Net_IPv6's Ip2Bin() and Bin2Ip() functions and simple string manipulation rather than bitwise operations, so it's guaranteed 32-bit safe (compared to 128-bit bitwise operations in current code which seem less certain?)

Changes are transparent - the canonical functions still work exactly as before on IPv4 (only with proper bad arg validation) but also now work on IPv6 transparently, and on arbitrary IPv4/IPv6 data, similar to other functions like is_ipaddr().

Tested and handles valid but uncommon edge cases of /0, /32 (IPv4) and /128 (IPv6) correctly. Also avoids inet_ntop/pton if that's a real issue (previous PR comment had asked to avoid these functions)

I'll merge it and fix the typo myself. Thanks!

Looks like the fork is gone, then I needed to merge it manually. It can be closed now


fixed prettier/eslint conflicts

ES-Lint wants there to be single quotes for each string, but prettier's default config makes it so that each string has double quotes. By forcing prettier to accept single quotes for strings in the .prettierrc file, we fix this issue if devs have prettier installed globally.

@matthewcn56 this looks good to me, thanks for making the changes!!
Do you wanna maybe add in changes for #34 in this PR as well? Here's the editor's example if you wanted to yoink it https://github.com/uclaacm/TeachLAFrontend/blob/master/.prettierrc.json


 The provider confluentinc/confluent does not support resource type "confluent_schema_registry_acl".

My Bad!  I apologize for releasing the code before confirming that it would work properly.


when we use

What problem does this feature solve?
<script lang="ts">
export default {
  name: "Name"
}
</script>
<script lang="ts" setup>
// code
</script>

What does the proposed API look like?
<script lang="ts" setup name="Name">
// code
</script>

Please refer to: https://github.com/vuejs/rfcs/blob/master/active-rfcs/0040-script-setup.md#automatic-name-inference


Use plant UML for class diagram

For internal development we created a high-level design class diagram so new developers can get an overview to get started.
This was created with DrawIO as Design.drawio and then exported as Design.png.
The diagram should be updated. Therefore we need to pull the latest drawio file, edit it with DrawIO desktop app, save changes, export as PNG and overwrite the PNG. Then push all these binary changes to git.
An alternative is to use plant UML that is a markup language for UML and other diagrams as wiki text.
This has many advantages and even allows to understand the diff of changes in git.
The only disadvantage is that it can get quite cumbersome to make the layout of a complex diagram nice since you cannot move boxes/items around like in DrawIO.
There is also an IntelliJ Plantuml plugin that shows a live-preview of your puml file while editing.
You can even include a puml from adoc to embed it as an image and asciidoctor can render it into the PDF or website.
Therefore this story is about the following changes:

[ ] Add IntelliJ Plantuml plugin to ide-settings/intellij/plugins (with active=true)
[x] Create a new Design.puml file in our documentation showing the contents of our current Design.png
[x] Extend the Design.puml to reflect the latest changes (add IdeLoggerImpl, GitContext, LocalToolCommandlet, GlobalToolCommandlet)
[x] Remove the old Design.png and Design.drawio files
[ ] Remove/update links to the old files and instead include puml to dev getting started.

To not break the installation of intellij by failing plugin installations, the first point (add the plugin file with plugin_active=true to ide-settings) first requires the following issue to be solved:

#550

Reopening until first task is completed.


Pulse schedule construction benchmarks fail

What is the current behavior?
Traceback (most recent call last):
                 File "/usr/lib/python3.8/site-packages/asv/benchmark.py", line 860, in disc_modules
                   module = import_module(module_name)
                 File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
                   return _bootstrap._gcd_import(name[level:], package, level)
                 File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
                 File "<frozen importlib._bootstrap>", line 991, in _find_and_load
                 File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
                 File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
                 File "<frozen importlib._bootstrap_external>", line 783, in exec_module
                 File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
                 File "/home/mtreinish/git/qiskit/qiskit/test/benchmarks/pulse/schedule_construction.py", line 19, in <module>
                   from qiskit.pulse import Schedule, Gaussian, DriveChannel, SamplePulse, Play
               ImportError: cannot import name 'SamplePulse' from 'qiskit.pulse' (/home/mtreinish/git/qiskit/qiskit/.asv/env/76391772e92136ec87b9940d70226329/lib/python3.8/site-packages/qiskit/pulse/__init__.py)

Steps to reproduce the problem
Run benchmarks
What is the expected behavior?
They work
Suggested solutions
Update for breaking api changes in pulse

Hai, I want to solve this

Hai, I want to solve this

@vishwas1234567 There is already a PR open for fixing this issue - #1147

@vishwas1234567 There is already a PR open for fixing this issue - #1147


Added tests

Added ImportCommand Test
Added ImportCommandParser Test
Added ImportCommandIntegration Test
Added ExportCommand Test
Added ExportCommandParser Test
Refractored ImportCommandParser and ExportCommandParser to better support future extension

Closes #158, #159, #160

Pull Request Test Coverage Report for Build 380

23 of 29 (79.31%)  changed or added relevant lines in 6 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+3.8%) to 58.733%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/main/java/seedu/address/logic/converters/fileformats/csv/CsvFile.java
0
1
0.0%

src/main/java/seedu/address/logic/parser/ImportCommandParser.java
5
6
83.33%

src/main/java/seedu/address/logic/commands/ExportCommand.java
5
7
71.43%

src/main/java/seedu/address/logic/parser/ExportCommandParser.java
7
9
77.78%

Totals

Change from base Build 371:
3.8%

Covered Lines:
1725

Relevant Lines:
2937

💛  - Coveralls


Add availability method in companies and write test for it. Closes #5

Let's definitely refactor that Faraday call into it's own method


Switch Username to GitUsername and update UG

Update GitUsername to replace Username and all related dependencies. UG updated to current, Command Summary updated as well.

Codecov Report

Merging #126 (72c8904) into master (fe45101) will not change coverage.
The diff coverage is 63.41%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe45101...72c8904. Read the comment docs.


[BUG] Default BOM report shows incorrect part.

Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

Describe the bug*
Using the default BOM report generation is showing the wrong parts: I expected this:

But the report generates this:

The quantities look correct but the parts are wrong.
Steps to Reproduce

Locate an assembly.
Click Bill of Materials.
Generate a BOM report in the Bill of Materials section (using the default template).
See the result.

Expected behavior
The BOM report should show the correct parts. It is showing the correct quantities but not the correct parts.
Deployment Method

[ ] Docker
[X] Bare metal

Version Information
Version Information:
InvenTree-Version: 0.9.0
Django Version: 3.2.16
Database: sqlite3
Debug-Mode: False
Deployed using Docker: False
Active plugins: []
Relevant log output
No response

Hi there @method1020! Thanks for the report. This seems to be a problem with the default report. I am not an expert in this area myself but I am sure we will find the problem.
@SergeoLacruz do you have any insights on the BOM report or could look at it? You seem to be well acquainted with the structure. Would appreciate any insights.

Hi,
The default BOM report is here:
https://github.com/inventree/InvenTree/blob/e730b5c24ce88b7b3d72d39903815359b5b66fdb/InvenTree/report/templates/report/inventree_bill_of_materials_report.html#L144-L158
The for loop refers to line.part which constantly refers to the parent. Should be line.sub_part instead.
That refers to the BOM entry. Please refer to the example here:
https://docs.inventree.org/en/latest/report/bom/
We should correct this in the source.
Michael

Thank you @SergeoLacruz !

Just tested with the latest template, works as expected. Thank you!


PicoTorrent preferences lost when recovering from crash

Running v0.12.0 on Windows 10x64.
I had updated my PicoTorrent preferences but before I could properly exit the app, my computer crashed. Upon restarting however, my changes did not save and I had to re-apply.

Sorry that happened - I'll put up a fix.


Adding WordCounter project to the repository. [Hacktoberfest]

Issue #105
Adding a front end project to the repo, which calculates typing speed of the user and helps in practicing speed typing.
This website is created using HTML, CSS, javascript.

@VidhiBhatt01 Please review this Pull request.

@VidhiBhatt01 Open index.html file in your browser to view the website

@VidhiBhatt01 Open index.html file in your browser to view the website

You should deploy it somewhere and attach the deployed link in the PR description


:goto is broken with text properties

Describe the bug
When textprops are set, :goto goes to the wrong position in some cases. The ifdef'd calculations in ml_find_line_or_offset seem to be incorrect.
To Reproduce

vim --clean and create the following buffer

abc
..........
..........

:call prop_type_add('foo', {})
:call prop_add(1,1,{'length':2, 'type':'foo'})
:goto 4 goes to line 1 col 3 (correct)
:goto 5 goes to line 2 col 1 (correct)
:goto 6 goes to line 2 col 1 (incorrect)
any  :goto from 7 - 15 also goes to line 2 col 1 (incorrect)
:goto 16 goes to line 3 col 1 (correct)

Expected behavior
:goto 6 should go to line 2 col 2
:goto 7 should go to line 2 col 3 etc
Environment (please complete the following information):

Vim version: v8.2.0579 (master)
OS: debian (glinux)
Terminal: xfce4-terminal <IP_ADDRESS>

Additional context
The relevant call to ml_find_line_or_offset stores a negative offset in *offp which seems suspicios (e.g. :goto 5 -> -20, :goto 6 -> -19).
But I don't understand the code well enough to proceed further myself.
My distro-installed vim is 8.1.2269 and shows quite different incorrect behavior for the same :gotos, jumping to the start or end of buffer. My guess is b413d2e6a8cc7b1611a41bfa9462b986393ca5fe was intended to fix this but was not correct.

Workaround: since things work at line boundaries, line2byte and byte2line both seem to work. For a quick hack
:command! -nargs=1 Goto :call cursor(byte2line(<args>), <args>-line2byte(byte2line(<args>))+1)

Has the bug in vim been fixed?

@dbarnett wrote:

Has the bug in vim been fixed?

The bug has simple reproducible steps.
I could still reproduce it with vim-8.2.1313 on Linux x86_64.
So it is not fixed yet.
:help todo also has this:
Text properties:
- :goto does not go to the right place when test properties are present.
  (#5930)

Ran into the same bug and did some debugging before finding this issue. To add to the conversation, here's a regression test I added to src/testdir/test_textprop.vim that replicates the issue:
func Test_prop_go()
  enew

  call setline(0, 'one')
  call setline(1, '')
  call setline(2, 'two')
  call setline(3, '')
  call setline(4, 'three four')

  call prop_type_add('testprop', {'highlight': 'Directory'})
  call search('^two')
  call prop_add(line('.'), col('.'), {
        \ 'length': len('two'),
        \ 'type':   'testprop'
        \ })

  call search('three \zsfour')

  let expected_byte_position = line2byte(line('.')) + col('.') - 1
  exe expected_byte_position 'goto'
  let actual_byte_position = line2byte(line('.')) + col('.') - 1

  eval actual_byte_position->assert_equal(expected_byte_position)

  call prop_type_delete('testprop')
  bwipe!
endfunc

If the prop_add call is commented out, this test passes just fine, otherwise I get the error Test_prop_go line 23: Expected 13 but got 7.

Ran into the same bug and did some debugging before finding this issue. To add to the conversation, here's a regression test I added to src/testdir/test_textprop.vim that replicates the issue:
func Test_prop_go()
  enew

  call setline(0, 'one')
  call setline(1, '')
  call setline(2, 'two')
  call setline(3, '')
  call setline(4, 'three four')

  call prop_type_add('testprop', {'highlight': 'Directory'})
  call search('^two')
  call prop_add(line('.'), col('.'), {
        \ 'length': len('two'),
        \ 'type':   'testprop'
        \ })

  call search('three \zsfour')

  let expected_byte_position = line2byte(line('.')) + col('.') - 1
  exe expected_byte_position 'goto'
  let actual_byte_position = line2byte(line('.')) + col('.') - 1

  eval actual_byte_position->assert_equal(expected_byte_position)

  call prop_type_delete('testprop')
  bwipe!
endfunc

If the prop_add call is commented out, this test passes just fine, otherwise I get the error Test_prop_go line 23: Expected 13 but got 7.


Homebrew install is broken if not rebuilt from source with Xcode 7.3

You will get an error saying that a symbol expected from PathKit to be found in libswiftcore.dylib could not. I assume this is because the pre-build binaries are building against Swift 2.1.1. With Xcode 7.3/Swift 2.2, this symbol (SequenceBox something or another, sorry I don't have the log anymore.) no longer exists.

Thanks for letting me know.
An interesting question to ask @kylef is to know if PathKit needs any migration to Swift 2.2 (does it compile with Xcode 7.3 alright?) before regenerating a new version for SwiftGen.
In the meantime I might suggest to install either via the Rakefile or check of homebrew has some --no-bottle flag or similar to force it to rebuild until we find a more suitable solution.

An interesting question to ask @kylef is to know if PathKit needs any migration to Swift 2.2 (does it compile with Xcode 7.3 alright?) before regenerating a new version for SwiftGen.

Swift 2.2 is a patch release and should not contain breaking changes, I would not expect PathKit to break.

It does. I've been using --build-from-source for now.

Swift 2.2 is a minor release and should not contain breaking changes, I would not expect PathKit to break.

@kylef Yeah I agree, but then I don't understand why the homebrew pre-build bottles of SwiftGen should break, as the ZIP file I bundle in Homebrew contains the swiftgen binary but also all its dependent frameworks, including Stencil, Commander, PathKit… and the Swift dylibs it has been built with.
So I'm not sure to understand how this could break SwiftGen as it doesn't rely on the locally installed Xcode version and has all its dependencies copied along with the binary… any idea about that?

Perhaps because I'm running it inside of a Swift "script" environment (from a build script in Xcode) it's messing with the version of Swift it's using and not a general issue with using it, say, on the command line.

@ChristopherRogers Mmmh maybe but I would be surprised if it did.
Could you run otool -L swiftgen in your Script Build Phase and see what is printed in the Build Log output? Thanks!
Also, how do you refer to swiftgen in your Script Build Phase? Simply using swiftgen? Or using its full path /usr/bin/swiftgen? Anything else?

I'll have to get back to you later on that (when I can check it at work).

It seems to work fine when run standalone, directly from a build script in Xcode, and even from the REPL.
Here's the otool output by the way.
/usr/local/bin/swiftgen:
	@rpath/PathKit.framework/Versions/A/PathKit (compatibility version 1.0.0, current version 1.0.0)
	@rpath/Stencil.framework/Versions/A/Stencil (compatibility version 1.0.0, current version 1.0.0)
	@rpath/Commander.framework/Versions/A/Commander (compatibility version 1.0.0, current version 1.0.0)
	@rpath/GenumKit.framework/Versions/A/GenumKit (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1256.1.0)
	@rpath/libswiftAppKit.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftCore.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftCoreData.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftCoreGraphics.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftCoreImage.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftDarwin.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftDispatch.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftFoundation.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libswiftObjectiveC.dylib (compatibility version 0.0.0, current version 0.0.0)

After reinstalling from source, the versions for the Swift libraries change (as expected).
/usr/local/bin/swiftgen:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1258.1.0)
	@rpath/PathKit.framework/Versions/A/PathKit (compatibility version 1.0.0, current version 1.0.0)
	@rpath/Stencil.framework/Versions/A/Stencil (compatibility version 1.0.0, current version 1.0.0)
	@rpath/Commander.framework/Versions/A/Commander (compatibility version 1.0.0, current version 1.0.0)
	@rpath/GenumKit.framework/Versions/A/GenumKit (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1258.0.0)
	@rpath/libswiftAppKit.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftCore.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftCoreData.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftCoreGraphics.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftCoreImage.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftDarwin.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftDispatch.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftFoundation.dylib (compatibility version 1.0.0, current version 703.0.18)
	@rpath/libswiftObjectiveC.dylib (compatibility version 1.0.0, current version 703.0.18)

And here is the actual error:
dyld: Symbol not found: __TMPdCSs12_SequenceBox
  Referenced from: /usr/local/Cellar/swiftgen/0.8.0/bin/../lib/PathKit.framework/Versions/A/PathKit
  Expected in: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx/libswiftCore.dylib
 in /usr/local/Cellar/swiftgen/0.8.0/bin/../lib/PathKit.framework/Versions/A/PathKit

Thanks for the follow-up!

Ha that makes much more sense indeed. If you use any Swift library, which depends on the Swift runtime libraries themselves, you'll have to use the same Swift runtime version for everything.
Using SwiftGen in a Swift script means your Swift script itself will use the system-installed Swift (2.2) while depending on a library (PathKit) which was built with 2.1 (from the homebrew bottle) hence the issue :wink:
So to run everything with the same version you can either rebuild everything from source as you did (probably the best way) or maybe you might have been able to use xcrun to make your own Swift script run using the 2.1 runtime (not worth it IMHO), as long as everything uses the same runtime.


Invariant Violation: Native module cannot be null

First installed the library and using it prompts an error

Steps taken:

Deleted node_modules & yarn install
pod deintegrate & pod install

Error still appears

You should probably check on this side: https://github.com/react-native-community/push-notification-ios/issues/43


Request for testing Adafruit Feather ESP32 with Adafruit LoRa Radio FeatherWing configuration

Please test the Adafruit Feather ESP32 with Adafruit LoRa Radio FeatherWing configuration and provide feedback!
Deep-sleep current consumption would be interesting, too!

Please refer to https://github.com/matthias-bs/BresserWeatherSensorTTN#software-build-setup and select the following board:

Thingpulse ePulse Feather & Adafruit LoRa Radio FeatherWing - RFM95W 900 MHz
Test o.k.
Deep-Sleep Mode Current @ 5 V: 3.5 mA (+/-1.2 %, +1 Digit)


setup_heroku fails on Windows/MSYS terminal

Some notes from running bin/setup_heroku in MINGW64 GIt bash terminal on Windows

First off,  I had to gem install activesupport
Windows heroku client stores auth credentials in ~/_netrc so had to login in windows cmd and then in terminal, cp ~/_netrc ~/.netrc

After this the script ran fine all the way up until:
And what username would you like to login as? foo
Finally, what password would you like to use? C:/users/foo/git/huginnz/lib/setup_tools.rb:77:in `noecho': Bad file descriptor (Errno::EBADF)
        from C:/users/jmn/git/huginnz/lib/setup_tools.rb:77:in `ask'
        from C:/users/jmn/git/huginnz/lib/setup_tools.rb:83:in `nag'
        from bin/setup_heroku:138:in `<main>'

Related

Hi @jmn,
you can try to remove , noecho: true from bin/setup_tools.rb, but I can not guarantee that it will work. We generally recommend using a linux VM to set up heroku.

I am closing this issue. Please feel free to reopen if you need further assistance!


Suggestion: ARVR mode for editor

Unreal Engine has a in-editor VR mode that allows users to build content directly in VR using a headset like Occulus or Vive
https://docs.unrealengine.com/en-US/Engine/Editor/VR/index.html
Having a feature to create and edit scenes while in VR could be very useful for VR developers, as they wouldn't have to keep switching in and out of VR in order to test. And since Godot supports visual scripting, a majority of content in a scene could be created directly in the VR editor pretty intuitively (minus assets like textures and models)

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.
The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.
If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!


Notebook showtabs false results in accidental multiline title

Version: 1.53.0-insider (user setup)
Commit: 2ce26643d85bf985d3d3ce6ce0ff5d78866f9b5f
Date: 2021-01-28T13:54:01.170Z
Electron: 11.2.1
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19042

Set "workbench.editor.showTabs": false
Go into a notebook and click into a cell
🐛 Part of the title seems unaligned, as though it's multiline when it's not supposed to be

The breadcrumb entry for a code cell is misaligned.

The breadcrumb entry for a code cell is misaligned.

Likely wraped the line and therefore shows like that...

Doesn't happen with document symbols, so adding for Feb

This is actually more severe than I thought... It happens with all notebook breadcrumbs when a file icon theme is used...


Transform roll/pitch commands with current yaw instead of setpoint

@tumbili @LorenzMeier
VTOLs really suffer from this currently, would be great if we can get to the bottom of it soon.
For reference, a similar previous attempt: https://github.com/PX4/Firmware/pull/3397

There is something inadequate in the YAW handling in the PX4 for the VTOL. I dont know what, it might be the issue here and the #3397 as well.
Benchmark is our simple KK2.0 build, which handles the YAW much much more adequate in hover ie quad.
If I want to really have good YAW handling in all modes with PX4 I need to crank so much the PID's that the osculations are obvious. When I tune the frame to be steady and without osculations, then the YAW is very very mellow and soft and not strong enough.  Using POS Ctl in windy weather in a hover, becomes kind of challenging for the machine.
Also we've had a corner case, which I will explain in another new issue in detail, in which If we are in Stabilized flight and we make a back transition in a 20-30% throttle,  the VTOL starts sliding like it is with skates on ice. And most noticeably it starts spinning (YAW) left to right or sometimes 180. And in this condition even if we apply some 60 - 70% throttle to recover the recovery is very slow and sometimes fatal.
Of coerce the reason for this is, we don't give enough throttle to the quad and after the transition the quad spins very slow, the VTOL continues in a very steady and fast descend with a fast forward motion from the inertia of the FW flight and with yaw spin. If you delay the application of the throttle with just 2 seconds this condition is almost unrecoverable after that with the quad.
I don't know what it is but there is something with the Yaw.

@tubeme Have you yaw feedforward (FF) set? Sounds like you're missing it.

@LorenzMeier Yes the FF is not enough now when you pointed out. Will try more FF and will report.
Though how's the back transition YAW problem related to FF gain? Isn't it that the FF gain is applied to my input form the RC?

Oh, I thought there was already an issue for that, ref: https://github.com/PX4/Firmware/issues/3620

Fixed in #4564


fix: adjust patch rule for Java LTS strategy

The LTS branch will not bump the minor version - it will preserve the
major/minor/patch of the version it was patched from.

I think it would be good to explain the nuance that requires the new lts-snapshot.

Added a note in the docs/java-release.md and in the java-lts releaser