Recycling heuristics have false positives and false negatives

Currently, vnodes are added to a recycling pool if:

they are not a component (typeof vnode.tag === "string")
they are not a fragment or trusted (vnode.domSize == null)
there are no hooks that can modify vnode.dom in vnode.attrs, at the time of removal

This excludes nodes that could be recycled (fragments and components), and includes nodes that may have had problematic vnode.attrs in the past.
These heuristics are only applied to vnode that are detached (those where onbeforeremove can fire). However, once a vnode has joined the pool, its children are recycled as well even if they don't meet those criteria, thus possibly recycling DOM nodes that have been modified in the hooks.
Proposed solution:

Add a vnode.reuse = true field.
Set it to false for trusted nodes and when a problematic hook fires (by modifying initLifecycle() and friends)
Only add nodes that have vnode.reuse to the pool;
In updateNodes(), when recycling === true, verify that vnode.reuse === true as well.

AFAICT these heuristics would allow to recycle components, fragments and don't have the holes described above.

@isiahmeadows I'm a bit confused by the tag, AFAICT, the only thing the proposal would break is buggy behavior.
Unless you refer to the addition of a field to vnode?

@pygy I swapped the labels. While I was on the bus, I had a similar idea myself, as a basis for other optimizations. (I'll detail them later in a gist.) It was mostly the part regarding changing pooling, though.
I have a better idea, though, although it's breaking in some places:

Add a new optional vnode.state.pooled boolean. This can even be specified as a property on object components, so you don't need oninit to specify it when you're using oncreate and onupdate.
Copy this to vnode.pooled after init, documented as read-only.
Check vnode.pooled when recycling.
If vnode.state.pooled is undefined, keep the current default.

That way, we are using a constant that we can track for our own purposes.

Note that we shouldn't pool custom elements by default, since they aren't purely host objects (they do usually execute JavaScript).

I'd add [bug] as well... False negatives in the heuristics for adding an object to the pool lead to slower diff with correct semantics, but false positives are bugs...
vnode.pooled, when non-null, would take precedence on vnode.reuse?
We can skip the vnode.state.pooled transfer, and use a function decorator:
function pooled(status, vnode) {
  vnode.pooled = status
  return vnode
}

And good catch re. custom elements.

I meant vnode.state.pooled instead of vnode.reuse. Also, I'm thinking
just relying on that, and leaving the rest of vnode out of it, since the
property only needs checked after onremove.
On Tue, Feb 28, 2017, 14:39 Pierre-Yves Gérardy<EMAIL_ADDRESS>wrote:

I'd add [bug] as well... False negatives in the heuristics for adding an
object to the pool lead to slower diff with correct semantics, but false
positives are bugs...
vnode.pooled, when non-null, would take precedence on vnode.reuse?
We can skip the vnode.state.pooled transfer, and use a function decorator:
function pooled(status, vnode) {
vnode.pooled = status
return vnode
}
And good catch re. custom elements.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lhorie/mithril.js/issues/1653#issuecomment-283140997,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AERrBBgwGWOt62jq1V6Xd_ntJw-nzGC2ks5rhHgKgaJpZM4MHJZf
.

To clarify, I mean removing vnode.reuse/vnode.pooled altogether.
On Tue, Feb 28, 2017, 15:54 Isiah Meadows<EMAIL_ADDRESS>wrote:

I meant vnode.state.pooled instead of vnode.reuse. Also, I'm thinking
just relying on that, and leaving the rest of vnode out of it, since the
property only needs checked after onremove.
On Tue, Feb 28, 2017, 14:39 Pierre-Yves Gérardy<EMAIL_ADDRESS>wrote:
I'd add [bug] as well... False negatives in the heuristics for adding an
object to the pool lead to slower diff with correct semantics, but false
positives are bugs...
vnode.pooled, when non-null, would take precedence on vnode.reuse?
We can skip the vnode.state.pooled transfer, and use a function decorator:
function pooled(status, vnode) {
vnode.pooled = status
return vnode
}
And good catch re. custom elements.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lhorie/mithril.js/issues/1653#issuecomment-283140997,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AERrBBgwGWOt62jq1V6Xd_ntJw-nzGC2ks5rhHgKgaJpZM4MHJZf
.

:-1: for adding another reserved name on vnode.state (the hooks are sufficient as gotchas)...
I don't understand why you'd want pooled or reuse to be read-only... If we want my original proposal to work, it has to be set to false by the engine when problematic situations are encountered (hooks that allow access to the vnode, etc...). Or do you mean that user code should not mess with it after initialization (in other words, that it is owned by the engine... another :-1: from me if it is on the state, which is the only user-owned object on vnodes)?

My other idea is this:
Make vnode.reuse a nullable boolean (i.e. boolean, null or undefined). You can then set appropriate defaults about vnode.reuse when you're ready to collect. Of course, that'll result in a boxed boolean, but I'm not sure it'll require an object allocation. (Most engines use special constants for null, undefined, true, and false.)

So it would default to null, users could set it to either true or false, and the engine would only mutate it if it were null.
Nodes that have either reuse === null or true would be recycled, whereas nodes with false wouldn't?

@pygy I'd like to clarify:

vnode.reuse is initially set to undefined.
oninit is called.
Things happen.
onremove is called, and the element removed.
If vnode.reuse is true or no oncreate/etc. hooks are defined and the node isn't a custom element, reuse the element
If vnode.reuse is false, any oncreate/etc. hooks are defined, or the node is a custom element, don't reuse it.

So what would happen is a single reuse: undefined field is added to the vnode (minimal memory hit), and at the end, the recycle check would look closer to this:
function shouldReuse(vnode) {
    if (vnode.reuse != null) return !!vnode.reuse
    if (vnode.dom.tagName.indexOf("-") >= 0 && !isSpecialNode(vnode.dom)) return false
    if (typeof vnode.state.oncreate === "function") return false
    if (typeof vnode.state.onupdate === "function") return false
    // etc...
    return true
}

// Helper for brevity
function isSpecialNode(node) {
    if (node.tagName === "color-profile") {
        return node.namespaceURI === "http://www.w3.org/2000/svg"
    } else if (node.tagName === "annotation-xml") {
        return node.namespaceURI === "http://www.w3.org/1998/Math/MathML"
    } else {
        return false
    }
}

You can't rely on the presence of hooks at removal time. The attrs hooks (and now even the state hooks) may be added/removed at any time. You must set reuse to false when the hooks are called/scheduled (this is all detailed in the original post).

Oh, good point. We can set the default after oninit is run, then.

I think we can get away with a single, non-nullable boolean field on vnode.
If the engine sets vnode.reuse to false before calling the hooks, the hooks can set it back to true for the pool to process.
	function initLifecycle(source, vnode, hooks) {
		if (typeof source.oninit === "function") source.oninit.call(vnode.state, vnode)
		if (typeof source.oncreate === "function") hooks.push(source.oncreate.bind(vnode.state, vnode))
	}

Would become
	function initLifecycle(source, vnode, hooks) {
		if (typeof source.oninit === "function") source.oninit.call(vnode.state, vnode)
		if (typeof source.oncreate === "function") {
			vnode.recycle = false
			hooks.push(source.oncreate.bind(vnode.state, vnode))
		}
	}

etc...
This would allow the hooks that only read from vnode.dom to mark themselves as 'safe'.

@pygy Eh...
I still would rather defer resolving the default as late as possible, even if it's via object member. Just in case it's set outside of oncreate and friends.

@lhorie I'm trying to make sense of isRecyclable()
function isRecyclable(old, vnodes) {
	if (old.pool != null && Math.abs(old.pool.length - vnodes.length) <= Math.abs(old.length - vnodes.length)) {
		var oldChildrenLength = old[0] && old[0].children && old[0].children.length || 0
		var poolChildrenLength = old.pool[0] && old.pool[0].children && old.pool[0].children.length || 0
		var vnodesChildrenLength = vnodes[0] && vnodes[0].children && vnodes[0].children.length || 0
		if (Math.abs(poolChildrenLength - vnodesChildrenLength) <= Math.abs(oldChildrenLength - vnodesChildrenLength)) {
			return true
		}
	}
	return false
}

In plain English, it returns true when the new vnode list is closer in length to the pool than to the old vnode list (and if the same goes for the children arrays of the first vnode in each of them).
I don't understand the reason for this. It means that a very large pool will be ignored, for example. I also don't understand why it looks at the children of each first element.
o("element nodes are not recycled when the pool is very large (non-keyed)", function(){
	var initial = [{tag: "div"}, {tag: "div"}, {tag: "div"}, {tag: "div"}]
	var empty = []
	var next = {tag: "div"}

	render(root, initial)
	render(root, empty)

	o(empty.pool.length).equals(4)
	empty.pool.forEach(function(vnode){vnode.dom.wasRecycled = true})

	render(root, [next])

	o(next.dom.wasRecycled).equals(undefined) // passes, I wish it didn't
})

https://github.com/lhorie/mithril.js/pull/1675/commits/75b45a641713e1615d97382a12236b33c7abd1e3#diff-000bdfae56251b0715111d2b18c9de3cL349 implements the last suggestion and seems harmless.


Toast's icon in HD is misaligned

Package name(s)
Core (@salt-ds/core)
Package version(s)
latest
Description
Toast's icon in HD is misaligned

Steps to reproduce
https://storybook.saltdesignsystem.com/?path=/story/core-toast--error&globals=density:high
Expected behavior
No response
Operating system

[ ] macOS
[ ] Windows
[ ] Linux
[ ] iOS
[ ] Android

Browser

[ ] Chrome
[ ] Safari
[ ] Firefox
[ ] Edge

Are you a JPMorgan Chase & Co. employee?

[ ] I am an employee of JPMorgan Chase & Co.

Hi - I could pick up this issue

Hi @liamsms, I think this will need some design input since we should be aligned with what's be spec'd in Figma.


Inventory consolidation

Feature request:
combine inventory so that
letter, stone, tent, herb, herb, bullet, bullet, bullet, feather, bullet, bullet

becomes
letter, stone, tent, 2 herbs, 5 bullets, and a feather.

Time to change arrays to hashes!

This has been added in 0.12.0. However, saves will not work if you upgrade because of another change I made (unless you run a small find-and-replace on them, which I can help with if need be).


Date comparison bug with Postgres dialect

Describe the bug
When I try to compare dates using field.equals(DateTime), I get PostgreSQLSeverity.error 42883: function julianday(text) does not exist Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Other comparison operators work correctly (isSmallerThan, isSmallerOrEqual, isBiggerThan, isBiggerOrEqual).
It's worth mentioning that dates are stored as a string using:
drift_dev:
        options:
          store_date_time_values_as_text: true

With best regards, hope for your help.

Postgres support is drift is currently experimental, and we don't have good support for datetime operations. In fact, the datetime API in drift is pretty much just a slightly more Dart-oriented version of the functions in sqlite3.
I don't currently have plans to make those operations work on postgres (we can probably make them work by generating different SQL based on the runtime dialect), but contributions to improve the situation are certainly welcome.


wlr/types/wlr_xdg_decoration_v1.h does not work with wlr/types/wlr_xdg_shell_v6.h

This is because wlr/types/wlr_xdg_decoration_v1.h includes wlr/types/wlr_xdg_shell.h directly.
I don't know if the protocols xdg_decoration_v1 and xdg_shell_v6 are incompatible, I couldn't find any indication that was the case at least.

xdg-decoration only works with xdg-shell stable. See the protocol file.


Organize commands in to folders

Break out individual commands from src/commands.ts in to individual files that export their main command function.

@swift-server-bot test this please


encode_model throws output file error

Hi,
After setting up environment and blender, when I run encoder_model file, it throws an error:
 [90](file:///C:/Users/../blender/render.py:90)          raise RuntimeError(f"render failed: output file missing")      [91](file:///C:/Users.../blender/render.py:91)           else:      [92](file:///C:/Users/.../Anaconda3/envs/shap/Lib/site-packages/shap_e/rendering/blender/render.py:92)               raise RuntimeError(f"render failed: output file missing. Output: {output}")
Any guide/ help will be appreciated.

Hi,
I get the same error, if you have found any solution then please let me know

try using modified version of blender: https://huggingface.co/datasets/tiange/Cap3D/resolve/main/misc/blender.zip


Issue rendering tables with leading paragraph

Hi,
I noticed this different while rendering some markdown markup into HTML. Is this expected behavior? Shouldn't tables be rendered properly regardless of whether there's text just before it or not?
I'm converting using the following;
var pipeline = new MarkdownPipelineBuilder().UseAdvancedExtensions().Build();
var html = Markdown.ToHtml(markdown, pipeline);

Test case 1: No extra line break added
Input
Hello world
| a | b |
|---|---|
| c | d |

Output

Hello world | a | b | |---|---| | c | d |

Test case 2: Extra line break added
Input
Hello world

| a | b |
|---|---|
| c | d |

Output

Hello world

            a
            b

            c
            d

I agree with your conclusion. This seems to be an edge case that is handled slightly different by different markdown parsers. Most of the parsers seem to align with the behavior of Markdig however, and I haven't been able to find any "spec" for the markdown table syntax that covers this.
Thanks for your reply @xoofx, feel free to close the issue!

Now that github renders the tables with a leading line with text as expected, a table underneath it, is this something worth looking into?
For example
sometext
|col1|col2|
|-|-|
|a|line|

Markdig rendered output:

sometext
|col1|col2|
|-|-|
|a|line|

Github:
sometext

col1
col2

a
line


How to run json-server under HTTPS?

Hi,
I have been working on a project using json-server on localhost. It worked well until I wanted it to serve JSON responses from an actual server. Now, I have hosted my front-end code in Apache (under HTTPS) and ran json-server as a background process using nohup in my server. After this, the front-end code is unable to access any response from json-server.
I see multiple issues here -

Apache is running on port 80 under HTTPS and json-server is running in localhost w.r.to the server on default port 3000
Chrome browser does not allow mixed content.
json-server always runs on localhost. I could not force it to run on HTTPS (I'm not sure if this can be done)

I could not find a solution for this. Any idea how I can run json-server under HTTPS on an actual server? Any help would be much appreciated.
PS. json-server is able to serve responses when the front-end is served from HTTP. However, I am using service workers in my project which need HTTPS to work.

Hi @kranthilakum,
I guess there's many way to solve it, you can:

use --host and --port if you need to make it run on some other port/host
use --static to tell JSON Server to serve your static files, so there shouldn't be an issue with mixed content anymore https://github.com/typicode/json-server#static-file-server

You can also probably configure Apache to proxy requests to JSON Server or use some other tools to do this.
Personally, I would suggest using JSON Server's static file server and hotel if you need https.

@typicode Thank you for the response!

@Gribbs - See if this helps. I had the same issue and the following steps work for me. I am providing steps for windows, and steps on unix based OS should be the same, except variable representation ($ instead of % etc.).

Setup json-server, as provided on the site. Do not start.
a. Create directory "c:\json-server" or any name.
b. Create db.json.
Setup hotel, as provided on the site and start hotel.
Setup localdomain as ".localhost". I setup on firefox with browser method and worked fine with steps provided.
Now add json-server to hotel. This is where it was slightly tricky and wasn't clear at first.
a. Go to directory, where you have created json.db file. e.g. cd c:\json-server
b. Now execute the following command: hotel add "json-server db.json -p %PORT%"
The tricky part is the %PORT%. Your app needs to listen on %PORT% so hotel can dynamically assign it a port.
Now go to hotel https://hotel.localhost and start the app.
a. App will be named "json-server" by default. Once it starts you will see the port it assigns by default.
Now you can visit https enabled site "https://json-server.localhost/"
a. Add exception when the browser suggests an invalid certificate because of the certificate being self-signed.

Hope this helps.
@typicode  - Could these steps be added as a complete guide for setting up json-server and hotel to provide https. It wasn't clear at first and this could save a lot of time. Thanks!

I do use a custom server.js
Create your cert
openssl req -nodes -new -x509 -keyout server.key -out server.cert

// https-json-server.js
import jsonServer from 'json-server';
import https from 'https';
import path from 'path';
import fs from 'fs';

const server = jsonServer.create();

const keyFile = path.join(__dirname, 'server.key');
const certFile = path.join(__dirname, 'server.cert');

https
  .createServer(
    {
      key: fs.readFileSync(keyFile),
      cert: fs.readFileSync(certFile),
    },
    server
  )
  .listen(3000, () => {
    console.log(
      'Go to https://localhost:3000/'
    );
  });

And then start node https-json-server.js

@adrianogalindo Check npm install was executed in the Dockerfile, check Node version and see if uspports ES6 if not use Babel to compile first.

Hi @luishdez, I don't have much js knowledge. I'm actually trying to migrate the https option to a docker container, but I can't make it work even with this js template you show.
Heres the file I created:
// https-json-server.js
import jsonServer from 'json-server';
import https from 'https';
import path from 'path';
import fs from 'fs';

const server = jsonServer.create();

const keyFile = path.join(/usr/src/app, 'server.key');
const certFile = path.join(/usr/src/app, 'server.cert');

https
  .createServer(
    {
      key: fs.readFileSync(keyFile),
      cert: fs.readFileSync(certFile),
    },
    server
  )
  .listen(3000, () => {
    console.log(
      'Go to https://localhost:3000/'
    );
  });

I ran
npm install -g json-server
then
node https-json-server.js
it gives me this error:
MAC:node_modules galiad$ cd json-server/
MAC:json-server galiad$ node https-json-server.js
(node:57757) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
/usr/local/lib/node_modules/json-server/https-json-server.js:2
import jsonServer from 'json-server';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1060:16)
    at Module._compile (internal/modules/cjs/loader.js:1108:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1164:10)
    at Module.load (internal/modules/cjs/loader.js:993:32)
    at Function.Module._load (internal/modules/cjs/loader.js:892:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    at internal/main/run_main_module.js:17:47

Do you have any clues or a more detailed step how to run this?
The '__dirname' is the directory to my docker folder and I copied the certificates before running the commands, but it doesn't even import the jsonServer from json-server.
I'm running node version 13.6.0
Thank you and sorry for my lack ok knowledge on this area.

To fix the module issue, use require instead of import.
Here is the code I ended up using that works for me:
`// https-json-server.js
// Run with: node https-json-server.js
// Generate SSL keys: openssl req -nodes -new -x509 -keyout server.key -out server.cert
const jsonServer = require("json-server");
const https = require("https");
const path = require("path");
const fs = require("fs");
const pause = require('connect-pause'); // Install: npm install connect-pause
const server = jsonServer.create();
const router = jsonServer.router('db.json');
const middlewares = jsonServer.defaults();
server.use(middlewares);
// To handle POST, PUT and PATCH you need to use a body-parser
// You can use the one used by JSON Server
server.use(jsonServer.bodyParser)
server.use((req, res, next) => {
if (req.method === 'POST') {
req.body.createdAt = Date.now()
}
// Continue to JSON Server router
next()
})
server.use(pause(2000));
server.use(router);
// If using custom routes
//var routes = JSON.parse(fs.readFileSync('routes.json'));
//server.use(jsonServer.rewriter(routes));
const keyFile = path.join(__dirname, 'server.key');
const certFile = path.join(__dirname, 'server.cert');
https
.createServer(
{
key: fs.readFileSync(keyFile),
cert: fs.readFileSync(certFile),
},
server
)
.listen(3001, () => {
console.log(
'Go to https://localhost:3001/'
);
});
`

In Chrome, you can use url chrome://flags/#allow-insecure-localhost to allow insecure localhost.

@adrianogalindo To fix the module issue, use require instead of import.
Here is the code I ended up using that works for me:
// https-json-server.js
// Run with: node https-json-server.js

// Generate SSL keys: openssl req -nodes -new -x509 -keyout server.key -out server.cert

const jsonServer = require("json-server");
const https = require("https");
const path = require("path");
const fs = require("fs");
const pause = require('connect-pause'); // Install: npm install connect-pause

const server = jsonServer.create();
const router = jsonServer.router('db.json');
const middlewares = jsonServer.defaults();

server.use(middlewares);

// To handle POST, PUT and PATCH you need to use a body-parser
// You can use the one used by JSON Server
server.use(jsonServer.bodyParser)
server.use((req, res, next) => {
  if (req.method === 'POST') {
    req.body.createdAt = Date.now()
  }
  // Continue to JSON Server router
  next()
})

server.use(pause(2000));
server.use(router);

// If using custom routes
//var routes = JSON.parse(fs.readFileSync('routes.json'));
//server.use(jsonServer.rewriter(routes));

const keyFile = path.join(__dirname, 'server.key');
const certFile = path.join(__dirname, 'server.cert');

https
  .createServer(
    {
      key: fs.readFileSync(keyFile),
      cert: fs.readFileSync(certFile),
    },
    server
  )
  .listen(3001, () => {
    console.log(
      'Go to https://localhost:3001/'
    );
  });

How to expose it to with --host option? I mean external IP?

How to expose it to with --host option? I mean external IP?
. . . .listen(3001, '<IP_ADDRESS>', () => { . . .

I'm actually trying to migrate the https option to a docker container

https://hub.docker.com/r/fabriciomendonca/json-server-https

Alternatively, I have setup a quick Caddy server locally and a .localhost domain. Caddy sources a cery automatically for a domain. Followed the steps mentioned here.


Consider caching oracle announcements on disk

An optimization so we don't need to fetch the values at every start of the app.

This project is unmaintained


How to save and load best model? [Question]

You must follow the issue template and provide as much information as possible. otherwise, this issue will be closed.
请按照 issue 模板要求填写信息。如果没有按照 issue 模板填写，将会忽略并关闭这个 issue
Check List
Thanks for considering to open an issue. Before you submit your issue, please confirm these boxes are checked.
You can post pictures, but if specific text or code is required to reproduce the issue, please provide the text in a plain text format for easy copy/paste.

[ x] I have searched in existing issues but did not find the same one.
[x ] I have read the documents

Environment

OS [e.g. Mac OS, Linux]: Windows
Python Version: 3.7.2
requirements.txt:

[Paste requirements.txt file here]

Question
I think this should be a simple question to answer. When running one of your models - BiLSTM - I want to run a few epochs and save the best model, then use the best model to evaluate and make prediction in the test set. When I try to load the best_model, I get a "NotaDirectory" error.
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras.models import load_model
from kashgari.tasks.classification import BiLSTM_Model
callbacks = [ModelCheckpoint(filepath='best_model.h5',
monitor='val_acc', mode='max',
save_best_only=True)]
model = BiLSTM_Model()
history = model.fit(X_train, y_train, X_val, y_val,
epochs=4,
callbacks=callbacks,
batch_size = 500)
best_model = kashgari.utils.load_model('best_model.h5')
NotADirectoryError: [Errno 20] Not a directory: 'best_model.h5/model_info.json'
However, if I try to save the best_model, it has already been saved, apparently by the callback as I expected.
model.save('best_model.h5')
FileExistsError: [Errno 17] File exists: 'best_model.h5'
So is there a way to save the best model and then load it so that I can make predictions from it?

A follow-up question:
I tried to just use the Keras load_model() method to retrieve the best model:
best_model = load_model('best_model.h5')
But then when I tried to predict using the best model, it returned an error:
y_test_predict = best_model.predict(X_test)
ValueError: Error when checking input: expected input to have shape (15,) but got array with shape (1,)
It looks like the best model is expecting the output of the input layer as input, instead of the X_test vector, as if the the input layer is missing from the best model.
This was the structure of the original model I ran. It seems like the best model does not have the InputLayer at the top and is expecting an input of (15,) instead of (1,), which is what I have,

Layer (type)                 Output Shape              Param #
input (InputLayer)           [(None, 15)]              0

layer_embedding (Embedding)  (None, 15,<PHONE_NUMBER>

bidirectional (Bidirectional (None, 256)               234496

dense (Dense)                (None, 2)                 514

    model_path = os.path.join('model_saved', 'test_classification_checkpoint')
    checkpoint = callbacks.KashgariModelCheckpoint(model_path,
                                                   verbose=1,
                                                   save_best_only=True,
                                                   save_weights_only=False,
                                                   kash_model=model)
    model.fit(train_x, train_y, test_x, test_y, epochs=2, callbacks=[checkpoint])
    model.evaluate(train_x, train_y, 64)

    # load and predict
    from kashgari import utils
    new_model = utils.load_model(model_path)
    print(new_model.predict(train_x[:10]))

this is an example to use the best model


Adding Open Software License 3.0

OSL 3.0 is not popular in terms of wide-spread use, but it is used by Magento and some other commerce-related open source projects, and offers a narrow copyleft similar to LGPL, but with added patent grant and dissuasion against patent litigation.

@derekjones I'd be happy to merge this given two changes:

The change to _config.yml needs to be reimplemented in _data/rules.yml since things have moved
Per https://github.com/github/choosealicense.com/issues/273, please add hidden: true to the YAML front matter.

Thanks @benbalter, done!

Thanks @derekjones!


OPC Publisher log file is growing full, no next log file is created

Hey,
we have the problem that once the OPC Publisher log file reaches 1 GB, no more logs are written. Also currently we need to stop the service to rename the old logfile, to enable the writing of a new logfile.
I think there should be an automatic change to write to a new log file, maybe adding the current timestamp to end of the filename.
Also there should be a retention policy, based on size or time.
Is there maybe an easy way to do this that I have missed?

@JohannesM24 have you checked the section "Option: Adjust log settings for each container module" on Prepare to deploy your IoT Edge solution in production?

Okay, thanks, just had a look, this looks helpful!

glad that it helped!


Add tk, perl, and gettext

These optional dependencies have been disabled. However, we have all of them packaged at this point. So, we should try enabling them and providing support for them to enrich the user experience. 😄

In trying this out, I'm getting warnings from conda-build saying that the Tk was built with the old shorter prefix length of 80 characters instead of 256.
WARNING:/opt/conda/lib/python3.5/site-packages/conda_build/build.py:Build prefix failed with prefix length 255
WARNING:/opt/conda/lib/python3.5/site-packages/conda_build/build.py:Error was:
WARNING:/opt/conda/lib/python3.5/site-packages/conda_build/build.py:ERROR: placeholder '/opt/conda/envs/_build_placehold_placehold_placehold_placehold_placehold_placeho' too short in: conda-forge::tk-8.6.6-3

WARNING:/opt/conda/lib/python3.5/site-packages/conda_build/build.py:One or more of your package dependencies needs to be rebuilt with a longer prefix length.
WARNING:/opt/conda/lib/python3.5/site-packages/conda_build/build.py:Falling back to legacy prefix length of 80 characters.
WARNING:/opt/conda/lib/python3.5/site-packages/conda_build/build.py:Your package will not install into prefixes > 80 characters.

Does that mean I just need to submit a PR to the tcltk-feedstock that bumps the build number so that it gets rebuilt with latest conda-build?

git is unusable for me without Perl, as it's needed for the --interactive commands. Isn't there a perl package on conda-forge?

Agreed. There is already a PR ( https://github.com/conda-forge/git-feedstock/pull/14 ). Wanted to get some buy in from other maintainers. Also unsure whether having multiple builds for different perl versions makes sense.

Unless it builds a Perl library it probably doesn't need to. But I really don't know how Perl works, so don't take my word for it.


Feature Request: support re-deploy k8s job.

Feature request
I hope k8s-deploy action can support k8s job deploy repeatedly.
because in normal situation, github actions will report error because of k8s job is immuteable so that we
have to delete it first.

Can you provide the error that k8s-deploy gives when running repeatedly?

it's just like what
https://stackoverflow.com/questions/56264165/field-is-immutable-k8s
describe.
On Fri, May 12, 2023 at 1:28 AM Oliver King @.***>
wrote:

Can you provide the error that k8s-deploy gives when running repeatedly?
—
Reply to this email directly, view it on GitHub
https://github.com/Azure/k8s-deploy/issues/285#issuecomment-1544402441,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AV7MPKFG5EYQFDGCW6V7A7TXFUOU7ANCNFSM6AAAAAAX4MPMTU
.
You are receiving this because you authored the thread.Message ID:
@.***>

it's just like what https://stackoverflow.com/questions/56264165/field-is-immutable-k8s describe.
…
On Fri, May 12, 2023 at 1:28 AM Oliver King @.> wrote: Can you provide the error that k8s-deploy gives when running repeatedly? — Reply to this email directly, view it on GitHub <#285 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV7MPKFG5EYQFDGCW6V7A7TXFUOU7ANCNFSM6AAAAAAX4MPMTU . You are receiving this because you authored the thread.Message ID: @.>

All k8s tools I'm aware of follow the same approach that k8s-deploy used here. The stackoverflow link's solution is to delete the overlapping deployment. I'm not sure what a good way for k8s-deploy to do that would be. Deleting a resource is an extremely destructive action.


Update renderer.js

Here, instead of reloading when the lives are over, I have made a restart function which sets all the variables to default values. Also, I have a High score variable, which appears in the middle of the game window, and is updated whenever the score is greater than current highscore.
Issue #3

This looks great. I think we don't need localStorage to store HighScore.
One Advice, Please write a descriptive commit message header.
For instance, You could've used "Allow user to save highscore for current session" instead of "update renderer.js"


False values fix

Fix for nuagenetworks/vsdcli#5 and nuagenetworks/vsdcli#6
This might be a bit of a workaround, but it's the only way without delving really deep into bambou. Of course, any suggestion and better/cleaner fix is always preferred :).

@t00f can't we use the type of the attribute to decide its type? I guess we have the same issue for the integer/float, etc

ok so it's not only a work around, this is breaking everything ;)
We have a solution, that we'll push next week.
In the meantime, there is a dirty workaround:
to set the flag to false do
vsd update egressacltemplate -i 095735ae-9f83-46a7-a550-2b5eebd0e1d0 -p defaultInstallACLImplicitRules=""

to set the flag to true do:
vsd update egressacltemplate -i 095735ae-9f83-46a7-a550-2b5eebd0e1d0 -p defaultInstallACLImplicitRules="whatever"

This should work. We'll add a bool parser

I did't see anything breaking, but i can understand why it would (having the value If you use the False value as a string for a non boolean parameter, migth break it). Sorry about that, wanted a quick solution on my end...
Thanks for the workaround! Looking forward to the official fix :)


Request to add a configuration item. The default value for Treemap Sizes is Parsed

This feature request AFAIk isn't a high priority. Currently i'm focus on the bundle size to same with the actual size.


Update travis grid

The travis grid is broken because of astropy updates:
https://travis-ci.org/radio-astro-tools/spectral-cube/builds/563172188

Coverage remained the same at 77.461% when pulling 6c9a828f846ef6caf405c5269c4fb43d2c1979c4 on keflavich:update_travis into 3d071dac6c11e6a8b0bace4ea2898478b5409a01 on radio-astro-tools:master.

docs failures:
https://travis-ci.org/radio-astro-tools/spectral-cube/jobs/563257732

I can't run the tests locally, they freeze somewhere...

none of these messages make sense to me:

WARNING: AstropyDeprecationWarning: astropy.extern.six will be removed in 4.0, use the six module directly if it is still needed [astropy.extern.six]
/home/travis/miniconda/envs/test/lib/python3.6/site-packages/sphinx/util/docutils.py:311: RemovedInSphinx30Warning: function based directive support is now deprecated. Use class based directive instead.
  RemovedInSphinx30Warning)
/home/travis/build/radio-astro-tools/spectral-cube/astropy_helpers/astropy_helpers/sphinx/themes/bootstrap-astropy/layout.html:15: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
  <div class="topbar">
/home/travis/build/radio-astro-tools/spectral-cube/docs/accessing.rst:48: WARNING: py:attr reference target not found: spectral_cube.SpectralCube.world
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/lower_dimensional_structures.py:docstring of spectral_cube.OneDSpectrum.shape:24: WARNING: py:obj reference target not found: ndarray.reshape
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/lower_dimensional_structures.py:docstring of spectral_cube.OneDSpectrum.size:17: WARNING: py:obj reference target not found: a.size
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/__init__.py:docstring of spectral_cube.OneDSpectrum.flatten:32: WARNING: py:obj reference target not found: flat
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/lower_dimensional_structures.py:docstring of spectral_cube.Projection.shape:24: WARNING: py:obj reference target not found: ndarray.reshape
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/lower_dimensional_structures.py:docstring of spectral_cube.Projection.size:17: WARNING: py:obj reference target not found: a.size
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/__init__.py:docstring of spectral_cube.Projection.flatten:32: WARNING: py:obj reference target not found: flat
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/base_class.py:docstring of spectral_cube.Projection.world_spines:8: WARNING: py:meth reference target not found: world
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/lower_dimensional_structures.py:docstring of spectral_cube.Slice.shape:24: WARNING: py:obj reference target not found: ndarray.reshape
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/lower_dimensional_structures.py:docstring of spectral_cube.Slice.size:17: WARNING: py:obj reference target not found: a.size
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/__init__.py:docstring of spectral_cube.Slice.flatten:32: WARNING: py:obj reference target not found: flat
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/base_class.py:docstring of spectral_cube.Slice.world_spines:8: WARNING: py:meth reference target not found: world
/home/travis/build/radio-astro-tools/spectral-cube/docs/api/spectral_cube.SpectralCube.rst:71:<autosummary>:1: WARNING: py:obj reference target not found: fill_value
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/base_class.py:docstring of spectral_cube.SpectralCube.with_fill_value:2: WARNING: py:obj reference target not found: fill_value
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/base_class.py:docstring of spectral_cube.SpectralCube.world_spines:8: WARNING: py:meth reference target not found: world
/home/travis/build/radio-astro-tools/spectral-cube/docs/api/spectral_cube.VaryingResolutionSpectralCube.rst:75:<autosummary>:1: WARNING: py:obj reference target not found: fill_value
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/base_class.py:docstring of spectral_cube.VaryingResolutionSpectralCube.with_fill_value:2: WARNING: py:obj reference target not found: fill_value
/home/travis/build/radio-astro-tools/spectral-cube/build/lib.linux-x86_64-3.6/spectral_cube/base_class.py:docstring of spectral_cube.VaryingResolutionSpectralCube.world_spines:8: WARNING: py:meth reference target not found: world
/home/travis/build/radio-astro-tools/spectral-cube/docs/big_data.rst:45: WARNING: py:meth reference target not found: SpectralCube.filled_data
/home/travis/build/radio-astro-tools/spectral-cube/docs/big_data.rst:45: WARNING: py:meth reference target not found: SpectralCube.unmasked_data
/home/travis/build/radio-astro-tools/spectral-cube/docs/big_data.rst:45: WARNING: py:meth reference target not found: SpectralCube.world
/home/travis/build/radio-astro-tools/spectral-cube/docs/big_data.rst:58: WARNING: py:meth reference target not found: SpectralCube.filled_data
/home/travis/build/radio-astro-tools/spectral-cube/docs/big_data.rst:58: WARNING: py:meth reference target not found: SpectralCube.unmasked_data
/home/travis/build/radio-astro-tools/spectral-cube/docs/manipulating.rst:7: WARNING: py:attr reference target not found: spectral_cube.SpectralCube.spectral_axis
/home/travis/build/radio-astro-tools/spectral-cube/docs/masking.rst:67: WARNING: py:attr reference target not found: spectral_cube.SpectralCube.unmasked_data

astropy.extern.six isn't used anywhere I can see, for starters.  These other attributes... all exist.

@keflavich - you need to use the full namespace, ~SpectralCube.filled_data makes no sense (these warnings came up now as numpydoc and sphinx got more strict of not accepting non existing API links).
It's usually quite useful to look at the generated docs, tons of links are unresolved: https://spectral-cube.readthedocs.io/en/latest/big_data.html
(And the rule of thumb is to use the API docs to see which namespace stuff if documented https://spectral-cube.readthedocs.io/en/latest/api.html (or, of course, one can look into the intersphinx directly))

py:obj reference target not found

That means you are not linking to the correct API link. You can do one of the following:

Find the correct API link.
Use double-backticks to not link it.

Thanks.  Some of those make sense and are easy to fix.  Some I still find utterly boggling; I can't find any references to reshape anywhere... anyway, I'll try to track them down now

If you inherit docstring or doing something funky like that, you need to trace it back to where it is actually defined. A recursive grep has helped me in the past.

I have the feeling that the API docs are searching for methods, finding them, then not finding the references to those same objects?
I'm... also... still not getting it.
/Users/adam/repos/spectral-cube/docs/big_data.rst:45: WARNING: py:meth reference target not found: spectral_cube.SpectralCube.filled_data
/Users/adam/repos/spectral-cube/docs/big_data.rst:45: WARNING: py:meth reference target not found: spectral_cube.SpectralCube.unmasked_data
/Users/adam/repos/spectral-cube/docs/big_data.rst:45: WARNING: py:meth reference target not found: spectral_cube.SpectralCube.world
/Users/adam/repos/spectral-cube/docs/big_data.rst:59: WARNING: py:meth reference target not found: spectral_cube.SpectralCube.filled_data
/Users/adam/repos/spectral-cube/docs/big_data.rst:59: WARNING: py:meth reference target not found: spectral_cube.SpectralCube.unmasked_data
/Users/adam/repos/spectral-cube/docs/spectral_extraction.rst:51: WARNING: py:meth reference target not found: spectral_cube.BaseSpectralCube.subcube_from_ds9region
/Users/adam/repos/spectral-cube/docs/spectral_extraction.rst:58: WARNING: py:meth reference target not found: spectral_cube.BaseSpectralCube.subcube_from_crtfregion
/Users/adam/repos/spectral-cube/docs/spectral_extraction.rst:65: WARNING: py:meth reference target not found: spectral_cube.BaseSpectralCube.subcube_from_regions

these are the ones I'm still tracking down (note to self)

try to drop the :meth from the beginning, they might not listed as methods in the intersphinx mapping

😭

There are very deep problems associated w/automodapi in which properties are not being autodoc'd.  I'm declaring our docs "broken but green" until we come up with a solution; real functionality cannot be held back by these failures any longer.


Traefik and nginx 404 page not found

Do you want to request a feature or report a bug?
Bug
Did you try using a 1.7.x configuration for the version 2.0?

[x ] Yes
[ ] No

What did you do?

What did you expect to see?
What did you see instead?
Output of traefik version: (What version of Traefik are you using?)

Version:      v1.7.14
Codename:     maroilles
Go version:   go1.12.8
Built:        2019-08-14_09:46:58AM
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?
logLevel = "DEBUG"

[traefikLog]
  filePath = "/var/log/traefik/traefik.log"
  format   = "json"

[accessLog]
  filePath = "/var/log/traefik/access.log"
  format = "json"

defaultEntryPoints = ["http", "https"]

[entryPoints]
  [entryPoints.dashboard]
    address = ":8080"
    [entryPoints.dashboard.auth]
      [entryPoints.dashboard.auth.basic]
        users = ["xxxxxx:xxxxxxxxxxxxxxxxxxx"]
  [entryPoints.http]
    address = ":80"
      [entryPoints.http.redirect]
        entryPoint = "https"
  [entryPoints.https]
    address = ":443"
      [entryPoints.https.tls]
[api]
entrypoint="dashboard"

[acme]
email =<EMAIL_ADDRESS>storage = "acme.json"
entryPoint = "https"
onHostRule = true
  [acme.httpChallenge]
  entryPoint = "http"

[docker]
domain = "xxxxxxxxxxx.com"
watch = true
network = "traefik"

services:
  traefik:
    image: traefik:v1.7.6-alpine
    ports:
        - "80:80"
        - "443:443"
        - "8080:8080"
    networks:
        - traefik
    volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - /root/tomas/traefik/traefik.toml:/traefik.toml
        - /root/tomas/traefik/acme.json:/acme.json
        - /var/log/traefik:/var/log/traefik

networks:
   traefik:
        external: true

services:
        nginx:
                image: nginx:alpine
                user: root
                volumes:
                        - /var/run/docker.sock:/var/run/docker.sock
                        - ./html:/usr/share/nginx/html
                        - ./conf.d/nginx.conf:/etc/nginx/nginx.conf
                        - /var/log/nginx:/var/log/nginx
                networks:
                        - traefik
                labels:
                        traefik.enable: 'true'
                        traefik.backend: nginx
                        traefik.port: '80'
                        traefik.frontend.rule: Host:xxxx.xxxx.com
                        traefik.domain: womoapp.eu
                        traefik.frontend.protocol: https
                        traefik.frontend.redirect.entryPoint: https
                        traefik.frontend.redirect.permanent: 'true'
                deploy:
                        mode: replicated
                        replicas: 2

networks:
        traefik:
                external: true

If applicable, please paste the log output in DEBUG level (--log.level=DEBUG switch)
{"level":"debug","msg":"Creating server server-stack-nginx-1-vbbqhfen6up3dx0r3tvkfgtbd-aaeff304a0d8ae08bbe9b1b159d1a653 at http://<IP_ADDRESS>:80 with weight 1","time":"2019-08-31T11:40:07Z"}
{"level":"debug","msg":"Creating server server-stack-nginx-2-vj4nig145c7pdt4yr3r9vsskv-f4b51c4646a9712e087e082650abe316 at http://<IP_ADDRESS>:80 with weight 1","time":"2019-08-31T11:40:07Z"}
{"level":"debug","msg":"Creating route route-frontend-Host-xxxxx-xxxxx-com-1 Host:xxxxx-xxxxx-com","time":"2019-08-31T11:40:07Z"}
{"level":"debug","msg":"Wiring frontend frontend-Host-xxxxx-xxxxx-com-2 to entryPoint http","time":"2019-08-31T11:40:07Z"}
{"level":"debug","msg":"Reusing backend backend-nginx [http - docker - frontend-Host-xxxxx-xxxxx-com-2 -<PHONE_NUMBER>067170285]","time":"2019-08-31T11:40:07Z"}
{"level":"debug","msg":"Creating route route-frontend-Host-xxxxx-xxxxx-com-2 Host:xxxxx-xxxxx-com","time":"2019-08-31T11:40:07Z"}
{"level":"debug","msg":"Add certificate for domains xxxxx-xxxxx-com","time":"2019-08-31T11:40:07Z"}
{"level":"info","msg":"Server configuration reloaded on :8080","time":"2019-08-31T11:40:07Z"}
{"level":"info","msg":"Server configuration reloaded on :80","time":"2019-08-31T11:40:07Z"}
{"level":"info","msg":"Server configuration reloaded on :443","time":"2019-08-31T11:40:07Z"}

{"BackendAddr":"","BackendName":"Traefik","BackendURL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"ClientAddr":"<IP_ADDRESS>:58713","ClientHost":"<IP_ADDRESS>","ClientPort":"58713","ClientUsername":"-","DownstreamContentSize":19,"DownstreamStatus":404,"DownstreamStatusLine":"404 Not Found","Duration":118892,"FrontendName":"backend not found","OriginContentSize":19,"OriginDuration":9924,"OriginStatus":404,"OriginStatusLine":"404 Not Found","Overhead":108968,"RequestAddr":"xxxxx-xxxxx-com","RequestContentSize":0,"RequestCount":9,"RequestHost":"xxxxx-xxxxx-com","RequestLine":"GET / HTTP/2.0","RequestMethod":"GET","RequestPath":"/","RequestPort":"-","RequestProtocol":"HTTP/2.0","RetryAttempts":0,"StartLocal":"2019-08-31T11:42:45.853198974Z","StartUTC":"2019-08-31T11:42:45.853198974Z","downstream_Content-Type":"text/plain; charset=utf-8","downstream_X-Content-Type-Options":"nosniff","level":"info","msg":"","origin_Content-Type":"text/plain; charset=utf-8","origin_X-Content-Type-Options":"nosniff","request_Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3","request_Accept-Encoding":"gzip, deflate, br","request_Accept-Language":"ro-RO,ro;q=0.9,en-US;q=0.8,en;q=0.7,it;q=0.6,pt;q=0.5,fr;q=0.4,ar;q=0.3,es;q=0.2,de;q=0.1","request_Cache-Control":"max-age=0","request_Sec-Fetch-Mode":"navigate","request_Sec-Fetch-Site":"none","request_Sec-Fetch-User":"?1","request_Upgrade-Insecure-Requests":"1","request_User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36","time":"2019-08-31T11:42:45Z"}

Hi! I'm Træfiker :robot: the bot in charge of communication regulation.
Thanks for your interest in Traefik!
We dedicate the issue tracker to bug reports and feature requests only. My advanced AI has spotted that your issue might be a configuration problem or relates to something that doesn't look like a bug.
To confirm this, please join our Community Forum and reach out to us on the Traefik section.
In case I'm wrong (well, that would be embarrassing :sweat_smile:), my developers will re-open the issue and fix me!
In the meantime, you can double check Traefik's documentation.


[fix] k8s - django maintenance resolver

This fix simply adds maintenance host to resolv for <IP_ADDRESS>, avoiding nginx failure when starting k8s pod
Logs when error occurs:
[root]@[mymachine]:[~]# kubectl logs django-b89b5bf97-4qrs2 -c nginx -n squest
2024/01/05 15:44:41 [emerg] 1#1: host not found in upstream "maintenance:80" in /etc/nginx/squest/nginx.conf:48
nginx: [emerg] host not found in upstream "maintenance:80" in /etc/nginx/squest/nginx.conf:48

Hi thx for the PR. Actually the maintenance container is not placed in the django pod. And so not available as <IP_ADDRESS>.
I think here the problem is due to the fact that django is deployed before the maintenance pod.

Hi thx for the PR. Actually the maintenance container is not placed in the django pod. And so not available as <IP_ADDRESS>. I think here the problem is due to the fact that django is deployed before the maintenance pod.

Oh wow! I'm sorry for this confusion, haven't looked into the whole picture.
I could work on that fix too, you reckon I should close this PR and open a new one, or commit changes to this existing PR?

It's up to you. I don't mind if you keep this PR. You can force push once you've done working on it.
But I'm not sure if we really have a bug here. As the maintenance pod finally comes up. I think you only have this log at the bootstraping. Because the django container is not up soon enough for the nginx container. And so the nginx container use the backup config that point to maintenance pod which is not there too at this point 😅.
But this error log should disappear once everything is started.

Closing this one as it seems to not fix the deployment


Add notice in WP admin to tell if an article has been sent to DPS - Enhancement

In working with my clients team, they suggested that it would be helpful to see from the WP post screen columns which articles have been sent to DPS. Those working in teams would benefit from this so they don't duplicate articles.

Hi @TimHaslam,
We're adding this to our roadmap. We'll look into incorporating it into a future release.
Thank you,
Studio Mercury


Feat: signup

Browser metadata
Path:      /signup
Browser:   Chrome <IP_ADDRESS> on Mac OS 10.15.7
Viewport:  1512 x 828 @2x
Language:  en-US
Cookies:   Enabled

Open in BrowserStack

Open Deploy Preview · Mark as Resolved

Browser metadata
Path:      /signup
Browser:   Chrome <IP_ADDRESS> on Mac OS 10.15.7
Viewport:  1512 x 828 @2x
Language:  en-US
Cookies:   Enabled

Open in BrowserStack

Open Deploy Preview · Mark as Resolved


ROX-13915: Move docker.io images used in ImageManagementTest to quayio/rhacs-eng to prevent rate limiting

Description
A detailed explanation of the changes in your PR.
Feel free to remove this section if it is overkill for your PR, and the title of your PR is sufficiently descriptive.
Checklist

[ ] Investigated and inspected CI test results
[ ] Unit test and regression tests added
[ ] Evaluated and added CHANGELOG entry if required
[ ] Determined and documented upgrade steps
[ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.
Testing Performed
TODO(replace-me)
Use this space to explain how you tested your PR, or, if you didn't test it, why
you did not do so. Valid reasons include, for example, "CI is sufficient",
"No testable changes". Feel free to attach JSON snippets, curl commands,
screenshots.
In addition to reviewing your code, reviewers must also review your testing
instructions and make sure they are sufficient.

/retest

/test eks-qa-e2e-tests

Ignore the prow failures. They were from before it was migrated to GHA


Response pages changes

See tasks
### Tasks
- [x] Change “Observing systems (##)” to “Add/Edit observing systems (##)”
- [x] Change “Data products (##)” to  “Add/Edit Data products (##)”
- [x] Change “Applications (##)” to “Add/Edit Applications (##)”
- [x] Change “Societal Benefit Areas (##)” to “Add/Edit Societal Benefit Areas (##)”
- [ ] https://github.com/nsidc/usaon-benefit-tool/issues/217
- [ ] Can we have a different table display for admins vs. respondents/analysts? For admins: keep table fields as is. For respondents/analysts: only display Title, Description, Status, Private, and the edit button.
- [ ] Change the "eye" logo to say simply "edit" (once people are assigned to specific surveys, we'll want to differentiate between "view" and "edit")
- [ ] If easy, timestamp formats throughout should be YYYY-MM-DD HH:MM (remove seconds and milliseconds)

add the ability to sort by different fields

I'm pulling this requirement out; it should be its own ticket! @hazel that's #218 . What milestone do you think? I don't think it will be hard, but also not trivial if that helps :)
There are some larger refactors I want to adopt (specifically HTMX) that would really help with features like sorting and deletion, and it may make sense to, for now, think about sensible defaults for sorting and filtering so we can come back to fully customizable sorting and filtering.

If easy, timestamp formats throughout should be YYYY-MM-DD HH:MM (remove seconds and milliseconds)

Here! Here! :)

If easy, timestamp formats throughout should be YYYY-MM-DD HH:MM (remove seconds and milliseconds)

Setting by default didn't work as I hoped, so I added a convenience function for formatting like this (e.g. {{ survey.created_timestamp | dateformat}} in a page template). We'll have to keep an eye out for wrongly formatted dates and apply this change. Not ideal, but we can come back later to see about setting a default date string format.

add the ability to sort by different fields

I'm pulling this requirement out; it should be its own ticket! @hazel that's #218 . What milestone do you think? I don't think it will be hard, but also not trivial if that helps :)
There are some larger refactors I want to adopt (specifically HTMX) that would really help with features like sorting and deletion, and it may make sense to, for now, think about sensible defaults for sorting and filtering so we can come back to fully customizable sorting and filtering.

Updated #218


Collision cancel ignored

Ever since 1.75, when pushing minecarts was fixed, any property sign/defaulttrainproperties.yml values that cancelled collisions have been ignored. This means that minecarts are ALWAYS able to be pushed, even when they are intended to have cancelled collisions.
Edit 1: I've only tested this with player collisions, not sure if any other collision types act the same way
Edit 2: A staff member on my server informed me that collisions with other trains don't appear to be affected by this

Edit: I've only tested this with player collisions, not sure if any other collision types act the same way

How do I test this? Because I think that I've found the problem.

Ensure that a train's collision property is set to 'cancel,' then have a player push the train. The player will be able to, despite the collision cancellation

Als happend by me. Player collision CANCEL is ignored.

When is this getting fixed? It's quite annoying.

Try the latest version, it is fixed now. Only you cant push trains/carts with playercollision set to cancel.


Add predict.cr

I mean it's pretty sciencey but maybe it should go in the misc category.

Updated.

Thank you!


Bug: Query export fails with high result count.

I wrote a query that yields +- 1100 results, which I want to export to .csv format. However, because the maxClauseCount is set to 1024 the export fails. This also happens when you set the maximum result count to lower than that. The image shows the export result.
A simple solution is to split up the query into chunks, but it would be nice if this issue is resolved in some future.
I used version 2.7.0 of the browser app.

Sorry, but I don't really know what you're referring to. By maxClauseCount, do you mean maxResultCount in the ExportParameters class, or something else? In fact, the only instance of the 1024 integer literal in the project is where I define the SUBJECT VARCHAR(1024) column of the EMAIL table.
Also, you said that "the image shows the export result", but I don't know if you meant to upload an image with this issue or somewhere else?

Oh wait; I see what happened!
Apparently lucenesearch has a cap on the number of clauses inside your query (1024), and throws an error if you exceed this.
The error message in the export panel is just ambiguous; my bad.


Microsoft ASP.NET CoreModule for IIS Express(V1) disappeared in ARP during the installation of VS but related files are not gone

Description:  Microsoft ASP.NET CoreModule for IIS Express(V1) disappeared in ARP during the installation of VS but related files are not gone.
Repro steps:
1.Install  V1 version for both IIS/IISExpress
2.Install V2 for both IIS/IISExpress
3.Install VS2017
Expected:The ANCM for IIS Express(V1) in ARP shows as normal.
Actually: The ANCM for IIS Express(V1) disappeared in ARP during install VS2017

It's considered a bundled component of VS/WTE, correct?

@Jinjin2Ma. When ANCM is installed as a bundled component, we should not show in ARP, which is by design. Please check if the expected files are still remained. The expected files are higher version comparing between the standalone V1 and the ANCM bundled in VS. After verifying that, please test one more thing. When VS is uninstalled, the ANCM should be uninstalled together.


Fix for a crash due to performLocalReleaseMotion and performLocalRetainMotion in LLVMARCOptimizer

• Explanation: Fixes a crash due to low level LLVM ARC optimizer in the Swift Compiler. In the pass during canonicalization we are not replacing all uses of the results of swift_retain and swift_unknownObjectRetain. But later in the pass we can optimize such retains, leaving uses without definition or uses not dominated by definitions.
• Scope of Issue: Compiler crash
• Risk: Low. The fix is low risk because LLVM Compiler has already been doing a similar optimization for @returned attribute. The issue happens only in cases where LLVM Compiler fails to do so.
• Reviewed By:  Andrew Trick
• Testing: Added unit tests

@swift-ci test

@swift-ci Please nominate


Possibly add Touchmote to Known users of ViGEm list

To preface this, Touchmote is a Wiimote mapper utility that can be used to navigate the Windows desktop with a Wiimote and for playing games with the Wiimote by emulating an Xbox 360 controller. Although the official Touchmote project is pretty much dead, I have forked it and I have been updating it for about two months. One of the big updates was to rewrite the XInput output handler to use ViGEmBus rather than ScpVBus.
Official site:
http://touchmote.net/
Official repo:
https://github.com/simphax/Touchmote
Forked version repo:
https://github.com/Ryochan7/Touchmote/tree/ryochan7
I have only one old example of using Xbox 360 controller emulation in Touchmote; my main experiments have been improving the FPS mouse routine. There is a short video showing Star Wars Episode 1: Racer being played with a Wiimote. The Wiimote was placed in a Wheel shell to mimic playing the game with steering wheel. Although updates to Touchmote would make this type of configuration better now, the game was pretty fun playing this way and I played through the whole campaign using the Wiimote.
https://www.youtube.com/watch?v=XgwFgKQwczk
As an aside, HidGuardian was needed to get controller support working properly in the game. The game initially detected the inactive DolphinBar created controllers and would refuse to detect the emulated Xbox 360 controller. Hiding the 4 DolphinBar created controllers was required for the game to detect the virtual Xbox 360 controller.

Thanks for bringing it to my attention, looks very interesting, I haven't forgotten about it, just trying to prioritize 😅

Done

Hi, there is any easy way to compile this fow windows? Or where to find compiled file. There was also a bat file to sign touchmote.
I just want to get rid of touchmote emulation driver and use vigem.


Compare with S3GS?

Thanks for your work. I noticed that you didn't compare your work with G3GS, which has the same goal as your work.
Could you give some information about which is better?
thanks.

Sorry, you are earlier.


1087

Add 1087 - Diablo tutorial

hi , i have resolved the changes you have asked for . Can you recheck and let me know that it is okay , please ? Thanks.

Hi

You haven't followed the title style and format I've asked for.
Your writing has a mix of capital and small letters where it is expected to use capital letter. It looks very informal. Please fix that.
There's still unnecessary blank lines in your code.
Also, please follow the style we are using for headings such as C++ and Java.
Your code is missing C++ header file, main function and some pre-requisites to get the code accepted. I would suggest you to include these missing parts.
You have mentioned about the use of BIT to solve the problem but you haven't explained the algorithm itself. Please provide link to a tutorial which explains this algorithm.

Please resolve the above mentioned issues.

Hello, sorry for the inconvenience. I've rechecked all the points you mentioned. Will you please let me know if there is further changes I need to resolve ? Thanks again.

Hi,
Thank you for your contribution.


refactor(App.Config): remove Utils app config and use System.Configuration

Summary
This PR removes the custom AppConfig  and use the ConfigurationManager that provides access to configuration files for client applications.

This "resource" file is compiled within final executable @afonsocdaniel ?
We don't need any extra file in final product, right?

resource

That's correct 👍


.to_s/.inspect on tuples are not reversable

I'm programmatically generating some crystal code from a set of hardcoded config files in order to embed the data in the program. (Without the need to ship the config files)
Using a tuple as a key for a hash requires a space after the hash's opening { (to prevent the grammar from thinking its a macro) However, when calling .to_s on a hash with a tuple key, the output does not have the space. I always like where .to_s or inspect in other languages creates code you can paste back in if needed.
Would it make since to add a space around the { and } of tuples when calling .to_s?
A quick example of the output:
a = { {1,2} => true }
puts a
# {{1, 2} => true}

Thanks!

I always like where .to_s or inspect in other languages creates code you can paste back in if needed.

Related: https://github.com/crystal-lang/crystal/issues/9966

Yeah, I would expect that you could turn a hash with simple data structures into code like that. Avoiding macro interpolation makes sense.
I don't think we want every tuple or hash wrapped in white space, though. That's probably gonna trigger some OCD.
It should be easy to implement for the specific cases where this applies: a hash or tuple as first element in a hash or tuple. For symmetry, we could also consider last elements, but that's a bit more complex. Without an opening delimiter, the closing delimiter }} should be harmless.

I disagree. I think this is doing exactly what it is supposed to do.


chore(deps): Move pako from dependency to devDependency

As this is (as far as I see) only used at build time to build the worker, we don't actually need to define this as a dependency.
Please let me know if I didn't understand something properly here 😅
Basically, what I think is happening:

With yarn build:worker the source from /worker/worker.ts is built into /src/worker/worker.js
This is then used in the app
pako is only really referenced in /worker/worker.ts, in /src/... it is actually inlined

(For reference is build:worker executed manually whenever something is changed there?)

Please let me know if I didn't understand something properly here 😅
Basically, what I think is happening:

With yarn build:worker the source from /worker/worker.ts is built into /src/worker/worker.js
This is then used in the app
pako is only really referenced in /worker/worker.ts, in /src/... it is actually inlined
This is correct

(For reference is build:worker executed manually whenever something is changed there?)
build:worker is always called when we build the package (see build:all + build:prod)


Update API reference to contain API group and versino info

The updated reference is generated using the https://github.com/kubernetes-sigs/reference-docs/pull/172 change.
Related: #2979
Follow up PRs needed to add reference to the API groups section where appropriate.

Preview link: https://deploy-preview-24635--kubernetes-io-master-staging.netlify.app/docs/reference/generated/kubernetes-api/v1.19/#-strong-api-groups-strong-

/cancel approved

Hi @tengqm .
This pull request updates the v1.19 API reference by adding an API groups table.
The changes look good.
/approve

/lgtm


"Position": "top" doesn't create a banner but a cc-floating

My code looks like this
<script>
 window.addEventListener("load", function () {
	 window.cookieconsent.initialise({
		"palette": {
				"popup": {
					"background": "#000"
				},
				"button": {
					"background": "#cd3141"
				}
		},
		"position": "top",
		"static": true
	})
});
 </script>

and I'm supposed to get the pushdown banner from this, but inspecting the code I get
class 'cc-floating' and not 'cc-banner'?

Sorry, I found out that it was happening because developer tools was open!


Bug Fixes

Bug fixes

Fixed a bug where web service not handling empty web part returns related to #232
Fixed a bug where title bar would show as "Web Part not transformed message" on target page

Thx @pkbullock !


fix: Add missing peer dependencies

tsup looks at dependencies and peerDependencies (not devDependencies) to determine if a package should be considered external. Since react was missing from peerDeps, it must have interpreted react as an internal module, so it was rewriting imports to "react.js".
Fixes #5777

Codecov Report
Patch coverage has no change and project coverage change: +11.42% :tada:

Comparison is base (1f74575) 88.57% compared to head (09fd617) 100.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff              @@
##             beta     #5779       +/-   ##
============================================
+ Coverage   88.57%   100.00%   +11.42%
============================================
  Files          74         1       -73
  Lines        2398        19     -2379
  Branches      623         1      -622
============================================
- Hits         2124        19     -2105
+ Misses        227         0      -227
+ Partials       47         0       -47

see 73 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[sdl2-image] Dependent .dlls are not automatically copied to the build directory

Hi!
I'm looking into integrating vcpkg in an existing CMake-based project which depends on the SDL2 image library, and I'm encountering a problem where dependent DLLs are not copied to the build directory automatically as I had hoped.
On building my project (vcpkg dependencies linked via find_package), the SDL2_image.dll is successfully copied to the build directory. However, the dynamically linked version of SDL2-image also needs libpng16.dll and zlib1.dll to be present to be able to load png files.
Since I'm new to vcpkg, my first question is, is this intended behavior? And if this is the correct behavior, is there a way I ensure that these dlls are copied to the build dir to make development as smooth as possible?
Thanks!

By default, if you run ./vcpkg install sld2-image, vcpkg will install libpng from sdl2-image by default and also install zlib (dependency of libpng), and will put it in the vcpkg\installed\x86(64)-windows\lib directory and search for it with your project.
https://github.com/microsoft/vcpkg/blob/251741475900c9a57549d80f1b5a5e30e63d1887/ports/sdl2-image/vcpkg.json#L8
https://github.com/microsoft/vcpkg/blob/251741475900c9a57549d80f1b5a5e30e63d1887/ports/libpng/vcpkg.json#L17

Same issue, please help.

By default, if you run ./vcpkg install sld2-image, vcpkg will install libpng from sdl2-image by default and also install zlib (dependency of libpng), and will put it in the vcpkg\installed\x86(64)-windows\lib directory and search for it with your project.
https://github.com/microsoft/vcpkg/blob/251741475900c9a57549d80f1b5a5e30e63d1887/ports/sdl2-image/vcpkg.json#L8
https://github.com/microsoft/vcpkg/blob/251741475900c9a57549d80f1b5a5e30e63d1887/ports/libpng/vcpkg.json#L17

How is that supposed to help?

@sam20908

However, the dynamically linked version of SDL2-image also needs libpng16.dll and zlib1.dll to be present to be able to load png files.vcpkg``find_package``SDL2_image.dll

Just need to run the commands ./vcpkg install sdl2-image and ./vcpkg integrate install under vcpkg to successfully integrate the sdl2-image library and link to the required dlls. :)

@bitsauce I am sorry to see the misleading answers from the vcpkg team.
The last update to sdl2-image added SDL2IMAGE_DEPS_SHARED for shared builds. This implies that image DLLs are loaded late as plugins, not early as normal dynamic libraries. And this implies that the information about the required DLLs is not available from normal inspection of the sdl2-image library as done during app-local DLL deployment.
This also affects other platforms. E.g. on x64-linux-dynamic,
ldd packages/sdl2-image_x64-linux-dynamic/lib/libSDL2_image.so shows no trace of png or tiff.
https://github.com/microsoft/vcpkg/pull/27627/files#diff-c81439a8935d25fccef8b46931a98f629278a8e40415de860901380e9230bee4R20-R28
I think SDL2IMAGE_DEPS_SHARED  should be removed.

@sam20908

However, the dynamically linked version of SDL2-image also needs libpng16.dll and zlib1.dll to be present to be able to load png files. vcpkgfind_packageSDL2_image.dll

Just need to run the commands ./vcpkg install sdl2-image and ./vcpkg integrate install under vcpkg to successfully integrate the sdl2-image library and link to the required dlls. :)

The thing is, I'm using this with CMake, and the command doesn't work anyway. All the SDL DLL files appear, but not libpng16.dll and such, which is the same issue with OP

I get the same issue

I get the same issue

so I have the engine to be built as .lib file (basically to a static library), but after installing SDL_image using vcpkg, and when built the lib and a project that references the lib file, is gives me issue that it can’t load libpng16.dll. And I wonder why it needs that one when the debug dll is there….all dlls have named.dll but somewhy when loading the image it needs release one….how can I fix it? People say it is something wrong with way I linked the dlls, when I literally done nothing, and they tell me it’s wrong in linking like I know where exactly in the settings.
And in logs visual studio shows me that it have loaded libpng16d.dll but sdl_image still wants normal one. I’m not 100% sure if that’s SDL_image who needs the release one. But it is for sure something there barking about it.
I installed btw dynamic version so I also got the issue with dlls

@nikeedev The original post didn't distinguish between debug and release. Your description seems quite different and confusing.

@nikeedev I can confirm that for some of my executable targets, it does indeed copy the corresponding libpng16.dll and zlib1.dll for release, and  libpng16d.dll and zlib1d.dll for debug. The debug .dlls are problematic for the reasons you mentioned; that SDL_image seems to be attempting to load the release versions of the .dlls regardless of the build type.
As for why just some of my targets seem to be unable to copy the .dlls I'm not sure. I'm wondering if the targets which have the .dlls have some other dependency which also copies libpng, but I would have to look into that more to verify that

The wrong DLL is probably also related to the activation of plugin-style activation: It uses a hardcoded DLL name in the source code, not the actual name recorded in the import lib. This happened beforein other ports.
Can't one of you try with the mentioned setting removed?

Sorry for the late reply.

@bitsauce I am sorry to see the misleading answers from the vcpkg team.
The last update to sdl2-image added SDL2IMAGE_DEPS_SHARED for shared builds. This implies that image DLLs are loaded late as plugins, not early as normal dynamic libraries. And this implies that the information about the required DLLs is not available from normal inspection of the sdl2-image library as done during app-local DLL deployment.
This also affects other platforms. E.g. on x64-linux-dynamic, ldd packages/sdl2-image_x64-linux-dynamic/lib/libSDL2_image.so shows no trace of png or tiff.
https://github.com/microsoft/vcpkg/pull/27627/files#diff-c81439a8935d25fccef8b46931a98f629278a8e40415de860901380e9230bee4R20-R28
I think SDL2IMAGE_DEPS_SHARED should be removed. CC @FtZPetruska

I tried forcing SDL2IMAGE_DEPS_SHARED to OFF (it's unconditionally set to ON by default) and I was not able to reproduce the issue on my Windows machine. Opened a PR with the fix #28137, let me know if it addresses your issues.

The wrong DLL is probably also related to the activation of plugin-style activation: It uses a hardcoded DLL name in the source code, not the actual name recorded in the import lib. This happened before in other ports.
Can't one of you try with the mentioned setting removed?

Can you navigate on how to do it?

The wrong DLL is probably also related to the activation of plugin-style activation: It uses a hardcoded DLL name in the source code, not the actual name recorded in the import lib. This happened before in other ports.
Can't one of you try with the mentioned setting removed?

Can you navigate on how to do it?

If you have GitHub tools installed, you can checkout my PR which consists in the fix described.
Otherwise you can download the patch, put it in your vcpkg directory and run git apply <patch name>.
In either case, you should then be able to (re)install sdl2-image with the fix.

@FtZPetruska I just tested your patch and it appears to have fixed the problem on my end. Thanks!

@FtZPetruska I just tested your patch and it appears to have fixed the problem on my end. Thanks!

Now we need to wait for the pr to get accepted


Bug: Unsupported custom-type

Describe the bug
It's not possible to provide a custom type as part of the Create method:
type MyThingy struct {
    Name string `json:"name"`
}

// does not work:
// rpc request err Error: Expected a CBOR integer, text, array or map
// unavailable ErrorChannel <nil>
// (See issue #186 )
_, err = surrealdb.Create[struct{}](surrealConn, "table1", MyThingy{
		Name: "Hello World",
	})

// works
_, err = surrealdb.Create[struct{}](surrealConn, "table1", map[string]interface{}{
		"name": "Hello World",
	})

Steps to reproduce

Run the example code.

Expected behaviour
Custom types to be usable as a parameter to Create.
SurrealDB version
surreal 2.1.2 for linux on amd64
Contact Details
No response
Is there an existing issue for this?

[X] I have searched the existing issues

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks for raising this @EtienneBruines. Can you please confirm the code you shared? I tried it successfully


UNet3D MLPerf

Overview
This is the training script for the UNet3D MLPerf model.
How to run training script
Download dataset
Run the following script from the root folder of tinygrad:
./examples/mlperf/scripts/setup_kits19_dataset.sh

Training model
For training on a tinybox green:
time PYTHONPATH=. WANDB=1 TRAIN_BEAM=3 FUSE_CONV_BW=1 GPUS=6 BS=6 MODEL=unet3d python3 examples/mlperf/model_train.py

Bounty locked. We have two more tinyboxes being built today, lmk when you are ready for an attempt. What's your username on discord?

@geohot - thanks for the heads up! The only thing I haven’t tried is training on an AMD GPU, so I might be ready to give it a quick run on the original image size.
Also, my Discord name is flata.

Does this support mulitGPU? Have you tested that? How sure are you it's correct? What's the expected training time?

I have added support for multiGPU for this one
I’ll do some more testing to get an accurate training time duration and will let you know once I’m ready for training on a tinybox.

Recent update:
I ran this training script on my M3 Max using BS=1 on the original size of (128, 128, 128) and it actually trains just fine. Synced with @chaosagent recently and may think there could be something going on with the autograd and/or allocator that’s causing the OOM issue on CUDA.
I’ll do an investigation to narrow it down further + sanity check training correctness before moving to a tinybox for an attempt.

You have tinybox access? Ask on discord if not

You have tinybox access? Ask on discord if not

I recently pasted my SSH public key on Discord for tinybox access here.

Sorry, missed this. Gave you access to tiny4, ssh instructions in discord

After trying my training script out on tiny4, it also receives the OOM error experienced in CUDA. I re-introduced gradient accumulation to my code and running a forward pass multiple times works. It is just when the optim.step() is executed, the gradients are realized, and it is when it runs out of memory.
Whether multi-GPU is used or not, it still runs out of memory (as mentioned in my description before). What I'm thinking now is to implement activation checkpointing to see if I can reduce memory pressure when calling optim.step(). So this is going to be my focus for this one and will report back once I have something for it.

Why is it using so much RAM? Gradient checkpointing is complex, have you looked into the root cause? Do you expect it to use less RAM?

You can try again with the new scheduler and sgd contiguous!

I tried the latest changes to scheduler + SGD contiguous and it didn't work. Also found out that the model isn't actually using a lot of RAM. In addition, I tried @chaosagent 's #3780 and no luck on that as well. So it may be related to this issue: #3572. Will investigate further to confirm if it fixes that issue here on UNet3D MLPerf training.

So it may be related to this issue: https://github.com/tinygrad/tinygrad/issues/3572

just an update on this one: in conv2d implementation, I saw that it bypasses the second expand when I run with WINO=1 and it trains with JIT enabled and BS=1 on the full image size now.
From this point, I'll be running with WINO=1. I will also test out FP16 to make it faster (thanks @chaosagent). I'll also add some time logs, similar to resnet, to keep tabs of the timings of various things during training.

@chenyuxyz - thanks for the heads up with BEAM. I have this run right now with BEAM=2. Beam itself took awhile but I'm getting ~1.5 it/s right now.
I'll keep improving on the iteration speed by scaling batch size by the number of GPUs & the dataloader for the training set.

cool. Feel free to share on discord too. How many steps are there in total?
--
Chen-Yu Yang
@.***
On Sun, Mar 31, 2024, at 7:19 PM, Francis Lata wrote:

@chenyuxyz https://github.com/chenyuxyz - thanks for the heads up with BEAM. I have this run https://wandb.ai/flata/tinygrad_unet3d_mlperf/runs/9gj7qasz/workspace?nw=nwuserflata right now with BEAM=2. Beam itself took awhile but I'm getting ~1.5 it/s right now.
I'll keep improving on the iteration speed by scaling batch size by the number of GPUs & the dataloader for the training set.
—
Reply to this email directly, view it on GitHub https://github.com/tinygrad/tinygrad/pull/3470#issuecomment-2029043417, or unsubscribe https://github.com/notifications/unsubscribe-auth/AATISJQETTPR7JD6SO3T5VDY3C73FAVCNFSM6AAAAABDUE427KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRZGA2DGNBRG4.
You are receiving this because you were mentioned.Message ID: @.***>

not sure exactly how many steps until convergence just yet but per epochs, there are 84 iterations. it will start evaluating at around 500 epochs and as of writing this message, it's at 130 epochs. will provide updates on discord.

@francislata , hey I'm a new contributor, and I would love to get my hands on to help you people to solve the #3572.
AS far as I have searched, this issue is only mentioned in your PR.
Can you please help me get started with this issue?
thanks in advance :D and sorry if its not the right place to ask for this

on a newly setup tinybox, when i run time PYTHONPATH=. WANDB=1 TRAIN_BEAM=3 FUSE_CONV_BW=1 GPUS=6 BS=6 MODEL=unet3d python3 examples/mlperf/model_train.py, it finished immediately without an error. I think it's due to missing data set. Can you add some checking to that to make sure the pipeline reads the correct number of data?
also for completion, add a small README like bert / resnet

@chenyuxyz - Regarding the instant failure without an error when attempting to train, I'll give it a look. I'll simulate it again on  a tinybox green.
About the README, did you want a similar directory path as BERT and ResNet, like as if we're submitting it to MLPerf?

I also tested by following the instructions in the description, I was able to get the training started:

Where there any steps that was different from yours when you attempted to train the model that I missed so I can reproduce the issue?

oh i did not re-run the dataset download because there's one in raid, maybe that's why. running from scratch with your instruction now

By the way, when I click on your W&B link, I get a 404. I think it could be an access control setting for the project level.

finished in 22 hours! fixed wandb permission too https://wandb.ai/chenyuxyz/tinygrad_unet3d_mlperf/runs/kr61rmub?nw=nwuserchenyuxyz
can you add a small README with the steps to run this? it's easier to track in the code base than the pr description


fit_generator get much lower accuracy then the evaluate_generator just one line above?

Hello,
I am following the blog for transfer learning https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html.
##First I compute the saved the bottleneck features and build a new model and train it with the bottleneck features:
input_layer = Input(shape=base_model.output_shape[1:])
x = GlobalAveragePooling2D()(input_layer)
x = Dense(512, activation='relu',name='fc_new_1')(x)
x = Dropout(0.2)(x)
x = Dense(512, activation='relu',name='fc_new_2')(x)
x = Dense(num_classes, activation='softmax',name='logit_new')(x)
Add_layers = Model(inputs=input_layer, outputs=x,name='Add_layers')
##Then I put this new model at the end of pretrained models:
base_model = ResNet50(include_top=False, weights='imagenet', input_shape=(img_shape[0],img_shape[1],3))
x = base_model.output
predictions = Add_layers(x)
model = Model(inputs=base_model.input, outputs=predictions)
model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])
##Then, I evaluate the model :
score = model.evaluate_generator(train_generator, nb_train_samples // batch_size_finetuning)
print('The evaluation of the entire model before fine tuning : ')
print(score)
score = model.evaluate_generator(validation_generator, nb_validation_samples // batch_size_evaluation)
print('The evaluation of the entire model before fine tuning : ')
print(score)
And get training loss and accuracy : [0.015362062912073827, 1.0]
validation loss and accuracy : [0.89740632474422455, 0.75]
##Just one line below it, I trained the new model:
model.fit_generator(train_generator,
steps_per_epoch= nb_train_samples // batch_size_finetuning,
epochs=finetuning_epoch,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size_evaluation,
callbacks=[checkpointer_finetuning,history_finetuning,TB_finetuning,lrate_finetuning,Eartly_Stopping_finetuning]);
Then the output is :
31/31 [==============================] - 35s - loss: 3.4004 - acc: 0.3297 - val_loss: 0.9591 - val_acc: 0.7083
Weird thing is : This problem only happens if I use Resnet50 and Inceptionv3 but not with vgg16. I am pretty sure that changing the pretrained model is the only difference. I understand that the dropout might make it different but it should not be this large and vgg16 has no obvious problem at all.
Another weird thing is: If I change every layer to be .trainable = False and compile, the validation accuracy will still decrease dramatically. I even checked the weights of every layer and if .trainable = False weights will not change and .trainable = True weights will change.
Any help is appreciated!!! THANKS!!!

I tried to use .fit instead of fit_generator and the same problem is still there: here is the new code:
score = model.evaluate(validation_images, validation_labels, batch_size=batch_size_finetuning)
print('(Validation) Evaluate the entire model : ')
print(score)
evaluate the added layers
score = model.evaluate(train_images, train_labels, batch_size=batch_size_finetuning)
print('(Train) Evaluate the entire model : ')
print(score)
print('Start Fine tuning...')
history_finetuning = model.fit(train_images,
train_labels,
epochs=ft_epochs,
batch_size=batch_size_finetuning,
validation_data=(validation_images, validation_labels),
shuffle=False);
I then tried the standard data set cifar10 and the results are as follows:
For the first training of the added layers: the last several epochs are as below:
Epoch 397/400
8000/8000 [==============================] - 0s - loss: 0.0012 - acc: 0.9998 - val_loss: 0.4322 - val_acc: 0.9115
Epoch 398/400
8000/8000 [==============================] - 0s - loss: 7.6729e-04 - acc: 1.0000 - val_loss: 0.4322 - val_acc: 0.9095
Epoch 399/400
8000/8000 [==============================] - 0s - loss: 6.1750e-04 - acc: 1.0000 - val_loss: 0.4314 - val_acc: 0.9085
Epoch 400/400
8000/8000 [==============================] - 0s - loss: 0.0014 - acc: 0.9998 - val_loss: 0.4356 - val_acc: 0.9120
Then I evaluate it
(Validation) Evaluate the entire model :
[0.43556556671857832, 0.9120000194311142]
8000/8000 [==============================] - 34s
(Train) Evaluate the entire model :
[4.2374865099191086e-05, 1.0]
Then I finetune the entire again, it shows:
Epoch 1/300
36/8000 [..............................] - ETA: 1393s - loss: 2.4836 - acc: 0.61112017-06-24 10:40:24.645982: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\pool_allocator.cc:247] PoolAllocator: After 1846990 get requests, put_count=1846697 evicted_count=1000 eviction_rate=0.000541507 and unsatisfied allocation rate=0.0007542
648/8000 [=>............................] - ETA: 149s - loss: 2.1055 - acc: 0.64202017-06-24 10:40:31.522072: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\pool_allocator.cc:247] PoolAllocator: After 2377 get requests, put_count=2300 evicted_count=1000 eviction_rate=0.434783 and unsatisfied allocation rate=0.460244
2017-06-24 10:40:31.522508: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\pool_allocator.cc:259] Raising pool_size_limit_ from 193 to 212
1584/8000 [====>.........................] - ETA: 96s - loss: 1.9212 - acc: 0.65782017-06-24 10:40:42.080824: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\pool_allocator.cc:247] PoolAllocator: After 4773 get requests, put_count=4828 evicted_count=1000 eviction_rate=0.207125 and unsatisfied allocation rate=0.207207
2017-06-24 10:40:42.081293: I c:\tf_jenkins\home\workspace\release-win\m\windows-gpu\py\35\tensorflow\core\common_runtime\gpu\pool_allocator.cc:259] Raising pool_size_limit_ from 493 to 542
8000/8000 [==============================] - 104s - loss: 1.2794 - acc: 0.7563 - val_loss: 0.7371 - val_acc: 0.8465
Epoch 2/300
8000/8000 [==============================] - 99s - loss: 0.6549 - acc: 0.8411 - val_loss: 0.8990 - val_acc: 0.8265
Is it normal because of dropout? But why VGG16 will not be like this but only inception3 and resnet50 and the difference is so large?

@ouzan19 @danielS91 @fchollet Do you think it might because of similar reasons as #6977?

I checked it for many times. I am pretty sure that the code is the same but only the based model is vgg16 then everything is expected, the second training starts with the accuracy as the evaluation and the end of the first training. But Resnet50 and inception3 will all experience a significant drop from the evaluation to the beginning of the second training. The evaluation is the same as the end of first training, so I assume that attaching the new sequential model (Added_layers) to the base models (inception3 or resnet50) is not wrong, otherwise the evaluation should not give expected answer. Really confused about what happened.

This is also happening to me.
Wierd thing: if you load the base_model, put your top layers and evaluate the model, it behaves as expected, but as soon as you train it (even when freezing the layers and using a very small lr) it breaks, and the evaluation will drop by a lot.

@bzhong2 @renato145
This problem happens to me, it confuses me a lot.
Do you find a solution?


Cannot open existing worlds

Legacy4J Version
1.20.4-1.7.4
Mod Loader Version
fabric-api-0.97.2+1.20.4
Bug/Glitch Occurrence Context
When trying to open an existing world the game will be stuck on a continuous loading screen.
Crash Report
No response
What happened wrong?
When trying to open an existing world the game will be stuck on a continuous loading screen.

Remove the Legacy Waters resource pack from your selected resource assort

Also see: #599