text:
Would the people of the 19th Century call our conventional software today artificial intelligence?
It is possible that the view of what is impressive enough in computer behavior to be called intelligence changes with each decade as we adjust to what capabilities are made available in products and services.

They would probably have followed the same sequence we do:

be amazed at the capabilities,
ask how it is done,
wonder whether this is really intelligence and (or) point out our narrow the performance was,
require more next time to be impressed again.

My sense is that they would, based on a high-level take of Babbage and Lovelace's view of the potential capability of the "analytic engine".  If Babbage's Tic-Tac-Toe machine had been built, I am sure that would have been regarded as machine intelligence.  Nimatron (Edward Condon) may have been the first game AI, and the capability seems similar to what Babbage was envisioning.  Certainly the bogus "Turk" chess-playing hoax was considered a machine intelligence.
Conventional software could connote a form of automation, and I think any form of automation, particularly where the operations are "under the covers", would have been considered a form of intelligence.

I think the current idea of only regarding "strong statistical AI" (Machine Learning) as AI is inherently flawed because of the concept of utility.  Intelligence is a spectrum, being a relative measure of problem solving strength, and artificial merely connotes a thing intentionally or skillfully constructed.  The Russell & Norvig definition seems to hew to this viewpoint.


text:
What are examples of applications of the Fourier transform to AI?
The (discrete and continuous) Fourier transform (FT) is used in signal processing in order to convert a signal (or function) in a certain domain (e.g. the time domain) to another domain (e.g., frequency domain). There are several resources on the web that attempt to explain the FT at different levels of complexity. See e.g. this answer or this and this Youtube videos.
What are examples of (real-world) applications of the Fourier transform to AI? I am looking for answers that explain the reason behind the use of the FT in the given application. I suppose that there are several applications of the FT to e.g. ML (data analysis) and robotics. I am looking for specific examples.

To be precise, a discrete Fourier transform can be used to transform a finite set of samples between frequency and time domains. A continuous Fourier transform can be applied in calculus to an expression or a set of equations (through the appropriate techniques) or used to develop algorithms, but digital systems are not continuous, so there is no way to directly integrate in a digital computer except symbolically.
When the features contained in time series or pixels are more easily extracted from spectral information or frequency domain complex numbers than from the time series, an FFT can produce the data in a form that supports easy extraction. For streams containing indefinitely long time series, a windowing scheme may be used.  The common cases are for sound recognition and music recognition. FFTs are also used in the finite math involved in cryptography and especially cryptanalysis (code breaking). In robotics, FFTs would most likely be used in vibration analysis to reduce wear and dampen oscillation.

The filtering (or convolution) process in Convolutional Neural Networks can be implemented using Fourier Transforms. Convolutions in image (or spatial) space are equivalent to multiplications in frequency space, and multiplications can be performed much more efficiently than convolutions.
The algorithm for using Fourier Transforms to calculate a convolution between an image and a filter kernel is as follows:
Assuming a 100x100 input image, and a 3x3 filter kernel (such as a Sobel Edge-Detection filter).

Calculate the 2D Fourier Transform (FT) of the image (C library, Python library)
Pad the filter kernel so that it is the same size as the image, and that the filter kernel is centred at image coordinate (0, 0). This means that some parts of the kernel will be placed in the other corners of the padded filter image. For example, consider a 3x3 kernel where the filter coefficients are [[a, b, c], [d, e, f], [g, h, i]], in this case the coefficients map to pixel positions on the padded kernel as follows:

a -> (99, 99)    //bottom right
b -> (0, 99)     //bottom left
c -> (1, 99)
d -> (99, 0) //top right
e -> (0, 0) //top left
f -> (1, 0)
g -> (99, 1)
h -> (0, 1)
i -> (1, 1)

Set all the other padded filter kernel pixels to zero
Calculate the 2D Fourier Transform of the padded filter kernel.
Perform an element-wise multiplication of the image FT and the padded filter kernel FT. Note that this is a complex multiplication as the FT consists of complex numbers. The result of this multiplication is therefore also an array of complex numbers.
Calculate the Inverse Fourier Transform of the result from step 4. This will return a real valued image which is the original image convolved with the filter kernel.

The advantage for performing the kernel convolutions in this way is that it is much faster than performing an element-wise convolution directly. This speeds up the training process for neural networks with Convolutional layers.

For a general overview of digital signal processing (including image processing), I recommend http://www.dspguide.com/pdfbook.htm. You can find the description of performing a convolution with FFTs in chapter 24: http://www.dspguide.com/ch24/7.htm. The book is quite old, but the principles within are immortal.

Specially on the problems related with PDEs you can find a relatively new article that is using a new approach to solve complex problems and improving the performance of classical approach.
Examples arise in molecular dynamics, micro-mechanics,
and turbulent flows. You can find the paper called: Fourier Neural Operator for Parametric PDEs from a recent research pdf , site.
A more detailed view on this new concept here and find more on the paper's references.

Maybe just to be clearer, you should also say that their approach uses neural networks.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Attempting to solve a optical character recognition task using a feed-forward network
I am doing some experimentation on neural networks, and for that I am trying to program a plain OCR task. I have learned CNNs are the best choice ,but for the time being and due to my inexperience, I wanna go step by step and start with feedforward nets.
So my training data is a set of roughly 400 16*16 images extracted from a script that draws every alphabet char in a tiny image for a small set of fonts registered in my computer.
Then the test data set is extracted from the same procedure, but for all fonts in my computer.
Well, results are quite bad. Get an accuracy of aprox. 45-50%, which is very poor... but that's not my question.
The point is that I can't get the MSE below 0.0049, no matter what hidden layer distribution I apply to the net. I have tried with several architectures and all comes down to this figure. Does that mean the net cannot learn any further given the data?
This MSE value however throws this poor results too.
I am using Tensorflow API directly, no keras or estimators and for a list of 62 recognizable characters these are examples of the architectures I have used: [256,1860,62] [256,130,62] [256,256, 128 ,62] [256,3600,62] ....
But never get the MSE below 0.0049, and still results are not over 50%.
Any hints are greatly appreciated.

actually it's not only about architecture, how about your learning rate value? have you try to tune it? how about the number of your training epoch?

400 training examples is quite a small number, I think, in order to solve this task. You will need thousands of pictures in order to achieve good results, I guess.

No 62. The output of the net is a one-hot array, each position corresponding to a different character to recognize

I would recommend using a larger dataset. The MNIST or EMNIST datasets are great for training OCRs. Also, how many generations are you training your network for.

So the training data is a small number of examples (400) drawn from a small set of fonts, and the test data is a much larger dataset drawn from a much larger set of fonts and is therefore much more variable than the training data. Two issues here are the small training data size and the difference in distributions between the training and test data. I would try the following:

Instead of defining your own architecture, try some of the pretrained architectures available in Keras such as ResNet50 or VGG16. You can start them either with random weights or imagenet weights. Remove the top layer and put your own layers on. You can also selectively unfreeze layers and see if that makes any difference.
To deal with the issues I mention above, use data augmentation to introduce variability into the training set.

You also have not mentioned in the post how you decide to stop training the network. In case the network is overfitting, you could try finishing training earlier.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How do the sine and cosine functions encode position in the transformer?
After going through both the "Illustrated Transformer" and "Annotated Transformer" blog posts, I still don't understand how the sinusoidal encodings are representing the position of elements in the input sequence.
Is it the fact that since each row (input token) in a matrix (entire input sequence) has a unique waveform as its encoding, each of which can be expressed as a linear function of any other element in the input sequence, then the transformer can learn relations between these rows via linear functions?

What is the positional encoding in the transformer model?. The explanation therein may help you.


text:
Is it possible to compute $P( F \mid S )$ given $P(F \mid S,A)$ and $P(F \mid S, \lnot A)$ in Bayesian network?
I have a bayesian network, which has the following data:
$P(S) = 0.07$
$P(A) = 0.01$
$P(F \mid S,A) = 1.0$
$P(F \mid S, \lnot A) = 0.7$
$P(F \mid \lnot S, A) = 0.9$
$P(F \mid \lnot S, \lnot A) = 0.1$
And I'm asked to get $P(F \mid S)$. Is it possible? How can I deduce it?

This is a bit of a puzzle but you can compute a reasonable narrow limit even without knowing whether or not $P(S,A) = P(S) P(A)$.
Start with the contingency table relating $P(S, A)$, $P(S,\neg A)$, $P(\neg S, A)$, $P(\neg S,\neg A)$ to $P(S)$ and $P(A)$ :
$$\begin{array}{cc|c}
P( S,A)& P(\neg S,A) & P(A) \\
P(S,\neg A)& P(\neg S,\neg A) & P(\neg A) \\ \hline
P(S)& P(\neg S) & 1 \\
\end{array} \quad \rightarrow \quad \begin{array}{cc|c}
x& 0.01-x & 0.01 \\
0.07-x& 0.92+x & 0.99 \\ \hline
0.07& 0.93 & 1 \\
\end{array}$$
note that the cells must be between zero and one thus $0 \leq x \leq 0.01$

Then with
\begin{array}{rcrl}P(F|S) &= & & P(F|S,A) P(A|S) + P(F| S,\neg A) P(\neg A|S)\\
&=&& 1.0 \frac{x}{0.07} +  0.7 \frac{0.07-x}{0.07} \\
& = && 0.7 + 4 \frac{2}{7} x  \end{array}
you get
$$0.7 \leq P(F|S) \leq 0.743$$

To solve $P(F|S)$ exactly you need to narrow down $x$ more precisely. Possibilities are:

If you know $P(F)$ then you could use $$\begin{array}{rcrl}P(F) &= & & P(F|S,A) P(S,A) \\ && +& P(F|\neg S,A) P(\neg S,A) \\ &&+& P(F| S,\neg A) P(S, \neg A)\\&&+&P(F|\neg S,\neg A) P(\neg S,\neg A)\\
&=&& 1.0 (x) + 0.7 (0.07 - x) + 0.9 (0.01-x)+0.1(0.92+x) \\
& = && 0.15-0.5 x \end{array}$$
If you know that $S \perp \! \! \! \perp A$ then you can use $x = P(S,A) = P(S)P(A) = 0.0007$

I believe you can deduce it. Using the product rule:
$p(x,y) = p(x\mid y)p(y)$
we have:
$P(F\mid S) = \frac{P(F,S)}{P(S)}$
we have $P(S)$ and we do not have $P(F,S)$, but we can use the addition rule:
$p(x) = \sum_\limits y p(x, y)$
$P(F, S) = P(F,S,A) + P(F,S,\lnot A)$
first term on the right side, using the product rule, is:
$P(F,S,A) = P(F\mid S,A)P(S,A)$
I assume that $S$ and $A$ are not conditionally dependent so we have:
$P(F,S,A) = P(F\mid S, A)P(S)P(A)$
for the second term on the right side, following the same logic, we have:
$P(F,S,\lnot A) = P(F\mid S, \lnot A)P(S)P(\lnot A)$
we have all the needed members so in the end we have:
$P(F\mid S) = \frac{P(F\mid S, A)P(S)P(A) + P(F\mid S, \lnot A)P(S)P(\lnot A)}{P(S)}$

You're making an assumption to calculate it. Your assumption might not be true.

True, maybe the assumption is correct, maybe it isn't, we'll see what op says. If  assumption isn't correct you can't compute it I believe.

I do not think you can compute $P(F \mid S=s)$ only using your given probabilities (and no further independence assumption between your random variables).
First of all, note that $P(F \mid S=s)$ is the probability of $F$ (being equal to one of the values that $F$ can attain), given that the value of $S$ is $s$. Not also that $P(F \mid S)$ is a shorthand for $P(F \mid S=s)$.
In general, by the law of total probability, we have
\begin{align}
P(A)
&= P(A, B) + P(A, B^c) \\
&= P(A|B)P(B) + P(A|B^c)P(B^c) \\
&= P(B|A)P(A) + P(B|A^c)P(A^c)
\end{align}
where $B^c$ is the complement of $B$ (in case $A$ and $B$ are sets or events).
So, in your specific case, we have
\begin{align}
P(F \mid S)
&= P(F \mid S, A)P(A \mid S) + P(F \mid S, \lnot A)P(\lnot A \mid S) \\
&= P(F \mid S, A) \frac{P(A, S)}{P(S)} + P(F \mid S, \lnot A)\frac{P(\lnot A, S)}{P(S)} \\
&= \frac{P(F, S, A)}{P(A, S)} \frac{P(A, S)}{P(S)} + \frac{P(F,S, \lnot A)}{P(\lnot A, S)}\frac{P(\lnot A, S)}{P(S)} \\
&= \frac{P(F, S, A)}{P(S)} + \frac{P(F,S, \lnot A)}{P(S)} \\
&= \frac{1}{P(S)} (P(F, S, A) + P(F,S, \lnot A) ) \\
&= \frac{P(F, S)}{P(S)}
\end{align}
We $P(F \mid S, A), P(S)$ and $P(F \mid S, \lnot A)$, but we do not have $P(A, S)$, $P(F, S, A)$ or $P(F, S)$ (and I think we cannot retrieve them from your given probabilities).
(I will be happy to be corrected, if this conclusion is wrong. Maybe I'm not seeing another way of computing it now.)

authors: Артур Хабибуллин


text:
How would a quantum computer potentially facilitate artificial consciousness, assuming it is possible?
How would a quantum computer potentially facilitate artificial consciousness, assuming it is possible?

Well, this is quite deep question. I would recommend you to have a look on the UK physicist Roger Penrose: https://en.wikipedia.org/wiki/Roger_Penrose
He has some books on the theme of consciousness and quantum mechanics. There are however critics of his ideas. One of them is that the quantum effects are too small to direct any considerable influence on the brain.

Can you be more specific regarding which books?

@nbro I just picked them up myself, and haven't had a chance to read, but I believe the two chief books are "The Emperor's New Mind" and "Shadows of the Mind".


text:
Can (trained) neural networks be combined with symbolic AI to perform operations like AND?
Does anyone work out ways of relating trained neural networks by symbolic AI?
For example, if I train a network on pictures of dogs, and I train a network on pictures of shirts. You could imagine that the simplest way (without going through the process from scratch) of identifying "dog AND shirt" would be to perform an AND operation on the last output of the individual cat & dog neural nets. So, "dog AND shirt" would amount to AND'ing the output of two nets.
But this operation AND could be replaced with a more complicated operation. And, in principle, I could "train" a neural network to act as one of these operations.  For instance, maybe I could figure out the net that describes some changeable output "X" being "on the shirt." This would be sort of like a "functional" in mathematics (in which we are operating are considering the behavior of a network whos input could be any network). If I can figure out this "functional", then I would be able to use it symbolically and determine queries like "dog on the shirt"? - "cat on the shirt"?
It seems like to me there's a lot of sense to turn specific neural networks into more "abstract" objects - and that there would be a lot of power in doing so.

I think you would be interested in Neural-Symbolic Learning and Reasoning, a recent survey on the intersection between connectionist models (e.g. neural nets) and symbolic reasoning. It's a long paper, but it has a lot in it that is relevant to your question, if you're willing to dig. For example, it talks about learning networks that implement boolean logic.

This
  network searches for satisfying solutions for the weighted conjunctive normal form (CNF):$\text{(¬X ∨ ¬Y ∨ Z) ∧ (X ∨ Y )}$.

The closest thing I could find to the "functionals" you mentioned was a discussion on learning "relations" with neural networks. For this, it cited Neural-Symbolic Cognitive Reasoning, a book also about the intersection between neural networks and symbolic reasoning. In Chapter 10, it gives the example of teaching a neural network to learn the relation of "grandparent."

Our goal is to learn a description of grandparent from examples of the father, mother, and grandparent relations such as father(charles, william), mother(elisabeth, charles), and grandparent(elisabeth, william).

I could imagine something like this being done with the example inputs being image features from a CNN. Perhaps then you could start learning functionals like "on the shirt."

Similarly, there have been several attempts to develop theorem provers using neural networks.


text:
Which model should I use to find (only) the object location (in terms of coordinates) in an image?
I am generating images that consist of points, where the object's location is where the most overlap of points occurs.

In this example, the object location is $(25, 51)$.
I am trying to train a model to just finds the location, so I don't care about the classification of the object. Additionally, the shape of the overlapping points where the object is located never changes and will always be that shape.
What is a good model for this objective?
Many of the potential models I've been looking at (CNN, YOLO, and R-CNN) are more concerned with classification than location. Should I search the image for the overlapping dots, create a bound box around them, then retrieve the boxes' coordinates?

Is this related to Numenta's work in any way? I've already seen such picture in that context.

@nbro yes, this is based off their grid cells work

What is a good model for this objective?

I will try to give another perspective: Solve it without machine learning model
Your problem is try to find the most overlapping point. If the image above is image that you used in your case, you can solve it directly by applying some computer vision algorithms.

Try to create some binary image based on the color of the dots. if you are not sure with the available colors on your image, you can list of pixel colors that not black and white uniquely at first. So if there are four colors you need to generate four different binary image. Create a simple condition or a complex one, for example:
if pixel[i,j]=red then
    pixel[i,j]=white
else
    pixel[i,j]=black

Get its location by searching "the white" over your image or use blob detection method (it'll be a little bit tricky if the actual image always have different axis scale). You can save it as a list of coordinate of each color.
What happen if you can't see the dot fully because it's overlapping with another dot? Find the pattern. In your image, dots appear with certain pattern. If you can find two consecutive dots horizontally and vertically, you can predict the position of all your dots.
Find the most overlapping position from your list.

Pros

The result may more accurate than using machine learning model
Faster, you don't need to train it first

Cons

Finding dot's location in image with different axis will be difficult, but it's still solvable
It'll be difficult to predict the pattern if many dots are missing because overlapped by other dots

Neural networks are not only used for classification but also for regression. It seems that a CNN would be a good solution for this problem with 2 output neurons each of them providing a number within the range of your frame.


text:
What are the main benefits of using Bayesian networks?
I have some trouble understanding the benefits of Bayesian networks.
Am I correct that the key benefit of the network is that one does not need to use the chain rule of probability in order to calculate joint distributions?
So, using the chain rule:
$$
P(A_1, \dots, A_n) = \prod_{i=1}^n (A_i \mid \cap_{j=1}^{i-1} A_j)
$$
leads to the same result as the following (assuming the nodes are structured by a Bayesian network)?
$$
P(A_1, \dots, A_n) = \prod_{i=1}^n P(A_i \mid \text{parents}(A_i))
$$

One other thing that comes to mind is markov blankets and other conditional independences, so local information is sufficient and other nodes are conditionally independent. I am not experienced enough to say how this is applied, but you can search for that. Having a Bayesian network feels to me like when I'm happy when I can use a Markov chain as a model, because of the structure and simplified dependencies. Judea Pearl, the man who introduced these networks, can explain this much better I'm sure: https://www.edge.org/conversation/judea_pearl-engines-of-evidence

@Sebastian benefit compared to what?

Yes, you are correct that one of the key benefits of Bayesian networks is that they allow you to calculate joint probability distributions without directly using the chain rule of probability. Bayesian networks provide a more efficient way to represent and compute joint probabilities by exploiting the conditional independence properties of the variables in the network.
In Bayesian networks, you represent the joint probability distribution by decomposing it into a product of conditional probabilities. This representation takes advantage of the conditional independence assumptions implied by the network structure, which often leads to more efficient computations and a more compact representation of the joint distribution.
While this is one of the key benefits, it is not the only benefit. Others include:

Modularity and interpretability: Bayesian networks represent complex relationships among variables in a graphical form, making it easy to visualize and understand the dependencies and independencies between variables. This modularity aids in knowledge representation, making it easy to update and modify the network structure as new information becomes available.

Incorporation of domain knowledge: Bayesian networks can incorporate expert knowledge in the form of prior probabilities and network structure. This allows for the inclusion of valuable information that might not be readily available in the data itself.

Inference and reasoning: Bayesian networks facilitate probabilistic reasoning under uncertainty, allowing you to perform various types of inferences such as causal reasoning, diagnostic reasoning, and predictive reasoning. They can be used to answer queries about the probabilities of events given new evidence or observations.

Handling missing data: Bayesian networks can naturally deal with incomplete or missing data. They can be used to predict the values of missing variables based on the observed data and the underlying probabilistic relationships between variables.


text:
Choosing more than one action in a parameterized policy
I would like to implement a variant of policy iteration that can choose one or more actions in each state. An example would be to heal and move in the game of Doom.
Parameterizing the power set of all single actions would be one idea, but I was wondering if somebody achieved good results on a similar problem, perhaps by simply defining some lower bound on the output layer and taking all actions with values larger than that bound (i.e. with actions and activation values {shoot=0.2, heal=0.51, move=0.6, jump=0.4} I would choose heal and move if the bound was 0.5)
Another idea was to collect these actions iteratively, i.e. choosing an action from a softmax output based on the state $s$ (taking action "healing") and then constructing and using some temporary state $s_t$ to evaluate that state to find another action (e.g. "moving"). This would require some dummy action that is just used to signal the end of that iteration procedure (i.e. choosing action $n+1$ will not add any other action to the set $\{ \text{healing}, \text{moving} \}$, but it will lead to the execution of those two actions and transition to the next state $s'$.


text:
How can it be shown clearly and transparently that the outcomes of data-driven health and care technology are validated?
The National Health Service (NHS) wrote down several principles in a document Code of conduct for data-driven health and care technology (updated 18 July 2019). I am concerned with principle 7.

Show what type of algorithm is being developed or deployed, the ethical examination of how the data is used, how its performance will be validated and how it will be integrated into health and care provision
Demonstrate the learning methodology of the algorithm being built.
Aim to show in a clear and transparent way how outcomes are validated.

But how exactly can outcomes be shown in a clear and transparent way how outcomes are validated?

The current version of the document Code of conduct for data-driven health and care technology provides more details about all principles, including principle number 7, which I will quote here.

Consider how the introduction of AI will change relationships in
health and care provision, and the implications of these changes for
responsibility and liability. Use current best practice on how to
explain algorithms to those taking actions based on their outputs.
When building an algorithm, be it a stand-alone product or integrated
within a system, show it clearly and be transparent of the learning
methodology (if any) that the algorithm is using. Undertake ethical
examination of data use specific to this use-case. Achieving
transparency of algorithms that have a higher potential for harm or
unintended decision-making, can ensure the rights of the data subject
as set out in the Data Protection Act 2018 are met, to build trust in
users and enable better adoption and uptake.
Work collaboratively with partners, specify the context for the
algorithm, specify potential alternative contexts and be transparent
on whether the model is based on active, supervised or unsupervised
learning. Show in a clear and transparent specification:

the functionality of the algorithm
the strengths and limitations of the algorithm (as far as they are known)
its learning methodology
whether it is ready for deployment or still in training
how the decision has been made on the acceptable use of the algorithm in the context it is being used (for example, is there a
committee, evidence or equivalent that has contributed to this
decision?)
the potential resource implications

This specification and transparency in development will build trust in
incorporating machine-led decision-making into clinical care.


text:
Is the definition of machine learning by Mitchell in his book "Machine Learning" valid?
Mitchell's definition of machine learning is as follows:

A computer program is said to learn from experience E with respect to some task T and performance measure P, if its performance at task T, as measured by P, improves with experience E.

Here, we talk about what it means for a program to learn rather than a machine, but a program and a machine aren't equivalent, so this is a definition about "program learning" rather than "machine learning". How is this consistent?

Think of a computer as a Turing Machine--this idea is a model of computation, and all of modern computing is based on the Turing-Church thesis.
Machine and program can be interchangeable—at the end of the day, it's all algorithms, whether hard coded in the form of microchip, or in the form of software.  (Any microchip can be emulated as software.)
Pre-modern computers were mechanical in popular sense. Examples include Babbage's
 difference engine, and mechanical calculators in general.  These led to programmable mechanical calculators and electromechanical computers such as IBM's Harvard Mark I, based on Babbage's notion of an Analytic Engine.
In this context, Machine Learning may also connote software that runs on a machine (hardware) as opposed to human/animal learning, which utilizes a biological medium.

The etymology of "mechanics", and by extension, "machine" is worth looking at.

mechanisma (in the sense of "mechanism")
mechanicus (in the sense of "belonging to mechanics")
mechanema (in the sense of "a trick")

Mechanema is instructive in that math and engineering can be understood as the "trick to doing things". (Machination is not pejorative in this sense, as "noun of action from past participle stem of machinari 'contrive skillfully, to design; to scheme, to plot,'"—one can plot the course of a celestial body.)
Early uses of the mech- root
Aristotle's Μηχανικά ("Mechanical Problems"*)
Euler's Mechanica (Euler was referring to Classical Mechanics, an analytic and predictive science made possible by calculus, which is based on functions.)


text:
Why am I getting spikes in the values of the loss function during training?
I trained a neural network on the UNSW-NB15 dataset, but, during training, I am getting spikes in the loss function. The algorithms see part of this UNSW dataset a single time. The loss function is plotted after every batch.

For other datasets, I don't experience this problem.  I've tried different optimizers and loss functions, but this problem remains with this dataset.
I'm using the fit_generator() function from Keras. Is there anyone experience this problem using Keras with this function?

The spikes could be caused by many reasons: insufficient model capacity, incorrect label, buggy input parsing, ... Finding out the culprit requires some detective work. For instance, you could apply the learned model to the whole train set and manually examine the datapoints which result in the highest loss. Alternatively, you could compare the learning outcomes of different models (both weaker and stronger).

Andrew Ng explains with great details in Deep Learning Course appears in the image below.
He also focuses on some corners can cause this problem :

some mislabeled examples in the dataset.
the size of your mini batch or change it with GD.


text:
Is anyone able to reproduce Hinton's matrix capsule networks?
I've been working on Hinton's matrix capsule networks for several months. I searched each corner of the internet. But I couldn't find anyone that can reproduce Hinton's matrix capsule network. Can anyone get the reported accuracy on SmallNORB and Cifar10 dataset?
PS: I know Hinton's another paper on capsule networks Dynamic Routing Between Capsules is reproducible. Please, do not confuse the two papers.

Have you considered emailing the authors? If you spent months working on this and searching online I doubt anyone here would have better results.

I just did what you suggested. Hope to get a reply.


text:
Can bounding boxes further improve the performance of a CNN classifier?
Suppose I have a standard image classification problem (i.e. CNN is shown a single image and predicts a single classification for it). If I were to use bounding boxes to surround the target image (i.e. convert this into an object detection problem), would this increase classification accuracy purely through the use of the bounding box?
I'm curious if the neural network can be "assisted" by us when we show it bounding boxes as opposed to just showing it the entire image and letting it figure it all out by itself.

Information outside of the bounding box could still be useful as context. So not to  lose it you can but bounding box as pixel mask into additional "pseudo-color" layer. That way you can also have many bounding boxes without changing input architecture. You will give network additional info without losing anything, so result shouldn't be worse at least.

Another way to ask the question is: Does sound get clearer when you remove the background noise?
The obvious answer is yes and in the case of image classification, the answer is also generally yes.
In most cases reducing the noise (irrelevant pixels) will strengthen the signal (activations) the neural network is trying to find.

Thanks for the answer, but I'm still a little confused. If the network is being forced to recognize only the region in my bounding box, thus reducing the noise, how can it then generalize to the noise outside the bounding box in a live/test sample if it's never been trained with that noise? I also read the answer here: https://ai.stackexchange.com/questions/10177/whats-the-role-of-bounding-boxes-in-object-detection which has left me further confused. Does the NN actively learn to ignore the noise outside its bounding box, or is the only thing it's exposed to within the bounding box?

For sure, this may be helpful. You remove excessive information from image and make the classification task a bit more simple. But you need to remember that bounding boxes may work not perfectly and accuracy of the classification algorithm may suffer from corrupted inputs (when bounding boxes are corrupted).

One could imagine using a segmentation network as a first step of processing. Then feeding an area corresponding to a bounding box of each segmented object to the classifier.
Potentially that could yield an increase in performance in classifying objects in an image, but not without a cost of training time, sine suddenly there are two networks to train instead of just one.


text:
What are the domains where SVMs are still state-of-the-art?
It seems that deep neural networks and other neural network based models are dominating many current areas like computer vision, object classification, reinforcement learning, etc.
Are there domains where SVMs (or other models) are still producing state-of-the-art results?

State-of-the-art is a tough bar, because it's not clear how it should be measured. An alternative criteria, which is akin to state-of-the-art, is to ask when you might prefer to try an SVM.
SVMs have several advantages:

Through the kernel trick, the runtime of an SVM does not increase significantly if you want to learn patterns over many non-linear combinations of features, rather than the original feature set. In contrast, a more modern approach like a deep neural network will need to get deeper or wider to model the same patterns, which will increase its training time.
SVMs have an inherent bias towards picking "conservative" hypotheses, that are less likely to overfit the data, because they try to find maximum margin hypotheses. In some sense, they "bake-in" Occam's razor.
SVMs have only two hyperparameters (the choice of kernel and the regularization constant), so they are very easy to tune to specific problems. It is usually sufficient to tune them by performing a simple grid-search through the parameter space, which can be done automatically.

SVMs also have some disadvantages:

SVMs have a runtime that scales cubically in the number of datapoints you want to train on (i.e. $O(n^3)$ runtime)1. This does not compare well with, say, a typical training approach for a deep neural network which runs in $O(w*n*e)$ time, where $n$ is the number of data points, $e$ is the number of training epochs, and $w$ is the number of weights in the network. Generally $w, e << n$.
To make use of the Kernel trick, SVMs cache a value for the kernelized "distance" between any two pairs of points. This means they need $O(n^2)$ memory. This is far, far, more trouble than the cubic runtime on most real-world sets. More than a few thousand datapoints will leave most modern servers thrashing, which increases effective runtime by several orders of magnitude. Together with point 1, this means SVMs will tend to become unworkably slow for sets beyond maybe 5,000-10,000 datapoints, at the upper limit.

All of these factors point to SVMs being relevant for exactly one use case: small datasets where the target pattern is thought, apriori, to be some regular, but highly non-linear, function of a large number of features. This use case actually arises fairly often. A recent example application where I found SVMs to be a natural approach was building predictive models for a target function that was known to be the result of interactions between pairs of features (specifically, communications between pairs of agents). An SVM with a quadratic kernel could therefore efficiently learn conservative, reasonable, guesses.

1 There are approximate algorithms that will solve the SVM faster than this, as noted in the other answers.

Deep Learning and Neural Networks are getting most of the focus because of recent advances in the field and most experts believe it to be the future of solving machine learning problems.
But make no mistake, classical models still produce exceptional results and in certain problems, they can produce better results than deep learning.
Linear Regression is still by far the most used machine learning algorithm in the world.
It’s difficult to identify a specific domain where classical models always perform better as the accuracy is very much determined on the shape and quality of the input data.
So algorithm and model selection is always a trade-off. It’s a somewhat accurate statement to make that classical models still perform better with smaller data sets. However, a lot of research is going into improving deep learning model performance on less data.
Most classical models require less computational resources so if your goal is speed then its much better.
Also, classical models are easier to implement and visualize which can be another indicator for performance, but it depends on your goals.
If you have unlimited resources, a massive observable data set that is properly labeled and you implement it correctly within the problem domain then deep learning is likely going to give you better results in most cases.
But in my experience, the real-world conditions are never this perfect

Totally agree with @John's answer. Will try and complement that with some more points.
Some advantages of SVMs:
a) SVM is defined by a convex optimisation problem for which there are efficient methods to solve, like SMO.
b) Effective in high dimensional spaces and also in cases where number of dimensions is greater than the number of samples.
c) Uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.
d) Different Kernel functions can be specified for the decision function.. In its simplest form, the kernel trick means transforming data into another dimension that has a clear dividing margin between classes of data.
The disadvantages of support vector machines include:
a) If the number of features is much greater than the number of samples, avoiding over-fitting in choosing Kernel functions and regularization term is crucial. Kernel models can be quite sensitive to over-fitting the model selection criterion
b) SVMs do not directly provide probability estimates. In many classification problems you actually want the probability of class membership, so it would be better to use a method like Logistic Regression, rather than post-process the output of the SVM to get probabilities.

For datasets of low-dimensional tabular data. DNN are not efficient on low-dimensional input because of huge overparametrisation. So even if dataset is huge in size, but each sample is low-dimensional SVM would beat DNN.
More generally if data is tabular and the correlation between fields of the sample is weak and noisy, SVM may still beat DNN even for high dimensional data, but that depend on specific of data.
Unfortunately I can't recall any specific papers on subject, so it's mostly common sense reasoning, you don't have to trust it.


text:
Why is dot product attention faster than additive attention?
In section 3.2.1 of Attention Is All You Need the claim is made that:

Dot-product attention is identical to our algorithm, except for the scaling factor of $\frac{1}{\sqrt{d_k}}$. Additive attention computes the compatibility function using a feed-forward network with a single hidden layer. While the two are similar in theoretical complexity, dot-product attention is much faster and more space-efficient in practice, since it can be implemented using highly optimized matrix multiplication code.

It does not make sense why dot product attention would be faster. Additive attention is nearly identical computation wise; the main difference is $Q + K$ instead of $Q K^T$ in dot product attention. $Q K^T$ requires at least as many addition operations as $Q + K$, so how can it possibly be faster?

The additive attention method that the researchers are comparing to corresponds to a neural network with 3 layers (it is not actually straight addition). Computing this will involve one multiplication of the input vector by a matrix, then by another matrix, and then the computation of something like a softmax. Smart implementation of a dot-product will not break out the whole matrix multiplication algorithm for it, and it will basically be a tight, easily parallelized loop.

Do you have a source for the 3 layer structure? If you see the chart comparing attentions here: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html

Lilian says the only difference between additive and dot product is Q + K vs Q K^T. I don't see any mention of multiple layers to compute Q + K

@user3180 I think it's right in the quote in your question. " Additive attention computes the compatibility function using a feed-forward network with
a single hidden layer. " A single hidden layer means it's a 3 layer network (input + hidden + output)

That line still doesn't make sense to me. Looking at a reputable implementation of additive attention, they literally just sum the Q + K tensors. Are you implying the network takes in Q concatenated with K, passes through the MLP, to get Q + K? It does not make much sense to call that concatenation + MLP operation as "addition"

Just think about the parameterizations that lead to Q and K. If they are obtained by some transformations such as Q=f(x; W_Q), K=f(x; W_K), there will be no need to apply additional transformations like Q=f(x; W_Q, A_Q), K=f(x; W_K, A_K).

Two are similar in theoretical complexity.
But matrix multiplication is better optimized. According to this paper, non-matmul ops in transformer are 15x slower than matmul operation.

Making Deep Learning Go Brrrr From First Principles discuses about similar topics.

In additive attention (as described in the paper by Bahdanau et al. (2014)), the alignment model $a$ is represented by a feedforward neural network. If you look in their appendix, they actually implement this as
$$ e_{ij} = V_a^T \tanh \left(W_a s_{i-1} + U_a h_j\right) = V_a^T \tanh \left(Q + K\right).$$
In contrast, in dot-product attention (as described in the paper by Vaswani et al. (2017)), the alignment model is implemented as
$$ e_{ij} = W_Q s_{i-1} \left(W_K h_j\right)^T = Q K^T.$$
The computational advantage is that the dot-product alignment model has only two weight matrices and only needs matrix multiplication, for which highly-optimized code exists.


text:
What does the formula $1-\sum_i(e_i-a_i)^2$ mean in this NEAT Python API?
I have looked at the documentation for the NEAT Python API found here, where it's written

The error for each genome is $1-\sum_i(e_i-a_i)^2$

I have not yet learned calculus, so I can't understand this formula. So, can someone please explain what the calculation means?

$$1-\sum_i(e_i-a_i)^2$$
$\sum$ - there just means sum. It is the greek letter for S.  You can rewrite the above formula as
$$1 -[(e_1 - a_1)^2+(e_2-a_2)^2+(e_3-a_3)^2+\ldots ]$$
$\sum$ just helps us avoid writing dozens of $+$ signs. Read more here.
What they are doing here is taking the difference of expected value $e_1$ and the actual value $a_1$ for the 1st example, and so on. The difference can be positive ($e_1 > a_1$) or negative ($e_1 < a_1$), so usually we square the difference to make it positive number.
The rest is there in the docs. Try putting in concrete imagined values for $a_i$ and $e_i$.

It sums the squared error for the output vs the expected output, this isnt something you need to do for each experiment they are simply telling you the metric they are using as fitness for the genomes in the xor example experiment, in other experiments you could use something else. If you were training it to play video games you set your fitness to be a numerical representation of how well the genome played the game, so you dont always need to have an expected value as long as your fitness function uses a meaningful metric as the fitness value.

The $\sum$ means that they take a sum of the squared difference of each pair of expected/predicted values ($e_i$) and actual values ($a_i$)
That gives them an error metric of how far off they are from their desired result. The goal is generally to optimize the algorithms against such an error function, in this case, to get it as close to one as possible.

authors: Костя Дряхлов


text:
Difficulty understanding Monte Carlo policy evaluation (state-value) for gridworld
I've been trying to read Sutton & Barto book chapter 5.1, but I'm still a bit confused about the procedure of using Monte Carlo policy evaluation (p.92), and now I just cant proceed anymore coding a python solution, because I feel like I don't fully understand how the algorithm works, so that the pseudocode example in the book doesn't seem to make much sense to me anymore. (the orange part)
I've done the chapter 4 examples with the algorithms coded already, so I'm not totally unfamiliar with these, but somehow I must have misunderstood the Monte Carlo prediction algorithm from chapter 5.

My setting is a 4x4 gridworld where reward is always -1.
Policy is currently equiprobable randomwalk. If an action would take the newState (s') into outside the grid, then you simply stay in place, but action will have been taken, and reward will have been rewarded.
Discount rate will be 1.0 (no discounting).
Terminal states should be two of them, (0,0) and (3,3) at the corners.

On page 92 it shows the algorithm pseudocode and I feel as though I coded my episode generating function correctly thusfar. I have it such that, the results are that I always start in the same starting state (1,1) coords in the gridworld.
Currently, I have it so that if you started always in state (1,1), then a possible randomly generate episode could be as follows (in this case also optimal walk). Note that I currently have the episodes in form of list of tuple (s, a, r). Where s will also be a tuple (row,column), but a = string such as "U" for up, and r is reward always -1.
so that a possible episode could be like: [( (1,1), "U", -1 ), ( (0,1), "L", -1 )] So that the terminal state is always excluded, so that the last state in episode will be the state immediately close to terminal state. Just like the pseudocode describes that you should exclude the terminal state S_T.
But, the random episode could have been one where there are repeating states such as [( (1,1), "U", -1), ( (0,1), "U", -1 ), ( (0,1), "U", -1 ), ( (0,1), "L", -1 )]
I made the loop for each step of episode, such as follows: once you have the episodeList of tuples, iterate for each tuple, in reversed order. I think this should give the correct amount of iterations there...
G can be updated as described in pseudocode.
currently the Returns(S_t) datastructure that I have, will be a dictionary where the keys are state tuples (row,col), and the values are empty lists in the beginning.
I have a feeling that I'm calculating the average into V(S_t) incorrectly because I origianlly thought that you could even omit the V(S_t) step totally from the algorithm, and only afterwards compute for a separate 2D array V[r,c] for each state get the sum of the appropriate list elements (accessed from the dict), and divide that sum by the amount of episodes that you ran???

But I don't suddently know how to implement the first visit check in the algorithm. Like, I literally don't understand what it is actually checking for.
And furthermore I don't understand how the empirical mean is now supposed to be calculates in the monte carlo algorithm where there is the V(s_t) = average( Returns(S_t) )
I will also post my python code thusfar.
import numpy as np
import numpy.linalg as LA
import random

# YOUR CODE

rows_count = 4
columns_count = 4
V = np.zeros((rows_count, columns_count))
reward = -1 #probably not needed
directions = ['up', 'right', 'down', 'left'] #probably not needed
maxiters = 10000
eps = 0.0000001
k = 0 # "memory counter" of iterations inside the for loop, note that for loop i-variable is regular loop variable

rows = 4
cols = 4

#stepsMatrix = np.zeros((rows_count, columns_count))

def isTerminal(r,c):      #helper function to check if terminal state or regular state
    global rows_count, columns_count
    if r == 0 and c == 0: #im a bit too lazy to check otherwise the iteration boundaries
        return True       #so that this helper function is a quick way to exclude computations
    if r == rows_count-1 and c == columns_count-1:
        return True
    return False

def getValue(row, col):    #helper func, get state value
    global V
    if row == -1: row =0   #if you bump into wall, you bounce back
    elif row == 4: row = 3
    if col == -1: col = 0
    elif col == 4: col =3

    return V[row,col]

def getState(row,col):
    if row == -1: row =0   #helper func for the exercise:1
    elif row == 4: row = 3
    if col == -1: col = 0
    elif col == 4: col =3
    return row, col

def makeEpisode(r,c):  #helper func for the exercise:1
## return the count of steps ??
#by definition, you should always start from non-terminal state, so
#by minimum, you need at least one action to get to terminal state
    stateWasTerm = False
    stepsTaken = 0
    curR = r
    curC = c
    while not stateWasTerm:

        act = random.randint(0,3)
        if act == 0: ##up
            curR-=1
        elif act == 1: ##right
            curC+=1
        elif act == 2: ## down
            curR+=1
        else:##left
            curC-=1
        stepsTaken +=1
        curR,curC = getState(curR,curC)
        stateWasTerm = isTerminal(curR,curC)
    return stepsTaken

V = np.zeros((rows_count, columns_count))
episodeCount = 100
reward = -1
y = 1.0 #the gamma discount rate

#use dictionary where key is stateTuple,
#and value is stateReturnsList
#after algorithm for monte carlo policy eval is done,
#we can update the dict into good format for printing
#and use numpy matrix
returnsDict={}
for r in range(4):
    for c in range(4):
        returnsDict[(r,c)]=[]

#"""first-visit montecarlo episode generation returns the episodelist"""
def firstMCEpisode(r,c):
    global reward
    stateWasTerm = False
    stepsTaken = 0
    curR = r
    curC = c
    episodeList=[  ]

    while not stateWasTerm:

        act = random.randint(0,3)
        if act == 0: ##up
            r-=1
            act="U"
        elif act == 1: ##right
            c+=1
            act="R"
        elif act == 2: ## down
            r+=1
            act="D"
        else:##left
            c-=1
            act="L"
        stepsTaken +=1

        r,c = getState(r,c)
        stateWasTerm = isTerminal(r,c)
        episodeList.append( ((curR,curC), act, reward) )
        if not stateWasTerm:

            curR = r
            curC = c

    return episodeList

kakka=0 #for debug breakpoints only!
#first-visit Monte Carlo with fixed starting state in the s(1,1) state
for n in range(1, episodeCount+1):

    epList = firstMCEpisode(1,1)
    G = 0
    for t in reversed( range( len(epList) )):
        G = y*G + reward #NOTE! reward is always same -1
        S_t = epList[t][0] #get the state only, from tuple

        willAppend = True
        for j in range(t-1):
            tmp = epList[j][0]
            if( tmp == S_t ):
                willAppend =False
                break
        if(willAppend):
            returnsDict[S_t].append(G)
            t_r = S_t[0] #tempRow from S_t
            t_c =S_t[1] #tempCol from S_t
            V[t_r, t_c] = sum( returnsDict[S_t] ) / n

kakka = 3 #for debug breakpoints only!
print(V)

See: https://ai.stackexchange.com/a/10818/2444. Maybe this is a duplicate of it.

apparently I calculated the value function average in a wrong way... apparently I am required to keep some kind of N(S_t) datastructure, maybe a dict where the key is the each particular state such as (row,col) and the value is the amount of times that state has been visited to overall across all episodes??? Or do I even need to have that N(S_t datastructure, can I just calculate the V(S_t) for each state such that I calculate the sumOfElements from each list of respective state, divided by that listAmountOfElements???

because isnt it so... that if I have dictionary where key is (row,col) for each state, and the value is the list of returns for that state... then the amount of elements in each list, is the amount of times that each state was visited?

also, a new question, are you supposed to start always in the same fixed starting state in this Monte Carlo policy evaluation part? will it distort the algorithm from converging to the actual value function for that policy (equiprobable randomwalk), as long as you have run maybe 100k iterations ( iterations == episodes)

I got the every-visit Monte Carlo done, but I didn't fully understand how to make the checking for first-visit MC, and from what sequience exactly do you check the existence of currently iterated state S_t ??? what is the ending limit that is included in the first-visit check iteration?


text:
Why don't people use projected Bellman error with deep neural networks?
Projected Bellman error has shown to be stable with linear function approximation. The technique is not at all new. I can only wonder why this technique is not adopted to use with non-linear function approximation (e.g. DQN)? Instead, a less theoretical justified target network is used.
I could come up with two possible explanations:

It doesn't readily apply to non-linear function approximation case (some work needed)
It doesn't yield a good solution. This is the case for true Bellman error but I'm not sure about the projected one.

Can you define the "projected Bellman error"?

@nbro It literally means what it says. Projecting a Bellman error onto a representable space of a function approximation. I think Sutton's Gradient TD minimize this. But it is proposed for a linear approximation case.

It might be clear to you, but what does it mean to project something onto a representable space of a function approximation model? Moreover, what is the Bellman error? Do you have a specific source that talks about this topic more in detail?

@nbro I refer to chapter 11.7 in Sutton's 2018 book which he describes Gradient TD methods. The projection you're talking about refers to p. 268 in the same book. Honestly, I don't really understand it that much.

I have found some clues in Maei's thesis (2011): “Gradient Temporal-Difference Learning Algorithms.”
According to the thesis:

GTD2 is a method that minimizes the projected Bellman error (MSPBE).
GTD2 is convergent in non-linear function approximation case (and off-policy).
GTD2 converges to a TD-fixed point (same point as semi-gradient TD).
GTD2 is slower to converge than usual semi-gradient TD.

It doesn't readily apply to non-linear function approximation.

No, it does.

It doesn't yield a good solution.

No, it does. TD-fixed point is the same point for the solution of semi-gradient TD (which is generally used). There is no edge on that.
The only explanation seems to be practical convergence rate.
To quote his words:

Some of our empirical results suggest that gradient-TD method maybe slower than conventional TD methods on problems on which conventional TD methods are sound (that is, on-policy learning problems).

As I understand it above-mentioned projection operator project into linear feature subspace produced from set of feature vectors (or feature functions), that is space of linear combinations of features. Vanilla DQN don't have any feature space, projection into linear subspace doesn't make sense in DQN context. If you attempt to produce feature space for values/Q with some NN it wouldn't be DQN (because Q wouldn't be produced) and it wouldn't work anyway on anything but toy problems because amount of degrees of freedom of output would be too high.

The proof in Maei's thesis on the convergence of GTD2 on non-linear case is based on a "tangent" plane. He argued that there exists a differentiable manifold which is locally linear (given that a learning rate is arbitrarily small). This extends the proof of linear case onto the non-linear case.

Output space of DNN is not even a manifold  strictly speaking because input (and in some treatments even weights) are random variables. Manifold of random variables is hard math problem (unlike random variable on manifold). Tangent hyperplane could be approximated sometimes but it's hard anyway.


text:
Feature Selection using Monte Carlo Tree Search
I'm trying to tackle the problem of feature selection as an RL problem, inspired by the paper Feature Selection as a One-Player Game. I know Monte-Carlo tree search (MCTS) is hardly RL.
So, I used MCTS for this problem, where nodes are subsets of features and edges are ("adding a feature to the subset") and it does converge to the optimal subset slowly.
I have a few questions

Is there a clever way to speed up the convergence of MCTS, besides parallelizing the rollout phase?
Adding nodes to the tree takes time and memory for datasets with a large number of features, for 10000 features, it takes up all my RAM (8GB) from the second iteration, (although it runs for 2000+ iterations for a dataset with 40 features which doesn't make sense to me). Is this expected or is my implementation likely wrong? Are there any workarounds for this?
What are your opinions on using MCTS for this task? Can you think of a better approach? The main problem of this approach is running the SVM as an evaluation function which may make the algorithm impractically slow for large datasets (a large number of training examples).

I was thinking of trying to come up with a heuristic function to evaluate subsets instead of the SVM. But I'm kind of lost and don't know how to do that. Any help would be really appreciated.


text:
What are the key differences between cellular neural network and convolutional neural network?
What are the key differences between cellular neural networks and convolutional neural networks in terms of working principle, implementation, potential performance, and applicability?


text:
How is the number of parameters reduced in the group convolution?
I think I don't understand group convolutions well.
Say you have 2 groups. This means that the number of parameters would be reduced in half. So, assuming you have an image and 100 channels, with a filter size of $3 \times 3$, you would have 900 parameters (ignore the bias for this example). If you separate this into 2 groups, if I understand it well, you would have 2 groups of 50 channels.
This can be made faster, by running the 2 groups in parallel, but how does the number of parameters get halved? Isn't each group having $50*9=450$ parameters, so, in total, you still have 900 parameters? Do they mean that the number of parameters that the backpropagation goes over (in each branch) gets halved?
Because overall, I don't see how it can get reduced. Also, is there a downside in using more groups (even going to 100 groups of 1 channel each)?

Hi. Could you edit your question to include a link to the definition of group convolution that you're using?

The number of parameters is filters*input_channels*output_channels
Groups are formed among input and output channels.
So instead of input_channels*output_channels with two groups you get (input_channels/2)*(output_channels/2) + (input_channels/2)*(output_channels/2)


text:
What can be considered a deep recurrent neural network?
In the paper Deep Recurrent Q-Learning for Partially Observable MDPs, the DRQN is described as DQN with the first post-convolutional fully-connected layer replaced by a recurrent LSTM.
I have DQN implementation with only two dense layers. I want to change this into DRQN with the first layer as an LSTM and leave the second dense layer untouched. If I understood correctly, I would also need to change the input data appropriately.
Are there any other things that need to be modified in order to make DRQN work?

Maybe your network cannot be considered the same as (or equivalent to) the DRQN if you also don't have the convolutional layers, even though the convolutional layers were mainly needed in the DQN paper because the input were images. Yours will be a network with a recurrent layer followed by a dense layer. What is the type of your input? Anyway, the Q part in DQN and DRQN only refers to the Q-learning algorithm. We could have called these networks simply deep networks or CNNs or RNNs. We just use the letter Q in order to contextualise the usage of these networks, which have nothing special.

Thanks for the clarification. I did not realize DRQN is a general term for a deep network with some recurrent layer and uses Q-learning. I thought it referred to the specific implementation.


text:
How can we reach global optimum?
Gradient descent can get stuck into local optimum. Which techniques are there to reach global optimum?

Possible duplicate of How to avoid falling into the "local minima" trap?

In Deep Learning there are several methods to improve "stuck" gradient - decrease learning rate, use cyclic learning rate - cycle it from bigger to smaller value. More radical method is completely reinitialize last or two last (before loss) layers of the network.
In non-Deep Learning ML out of those only decrease learning rate will work, but there is plethora Numerical Optimization methods to help, like second-order methods - variation of Gauss-Newton, or methods specific to the problem which may include projective methods, alternate directions, conjugate gradients etc. There are a lot of methods which are better then gradient descent for non-Deep Learning optimization.


text:
How does bidirectional encoding allow the predicted word to indirectly "see itself"?
Before the release of BERT, we used to say that it is not possible to train bidirectional models by simply conditioning each word on its previous and next words, since this would allow the word that's being predicted to indirectly "see itself" in a multi-layer model. How does this happen?

Check out my answer on Cross Validated: https://stats.stackexchange.com/a/446049/24612


text:
What are stable ways of doing online machine learning?
I am trying to deploy a machine learning solution online into an application for a client.  One thing they requested is that the solution must be able to learn online because the problem may be non-stationary and they want the solution to track the non-stationarity of the problem.  I thought about this problem a lot, would the following work?

Set the learning rate (step-size parameter) for the neural network at a low fixed value so that the most recent training step is weighted more.
Update the model only once per day, in a mini-batch fashion.  The mini-batch will contain data from the day, mixed with data from the original data set to prevent catastrophic interference.  By using a mini batch update, I am not prone to biasing my model to the latest examples, and completely forgetting the training examples from months ago.

Would this set-up be "stable" for online/incremental machine learning?  Also, should I set up the update step so it samples data from all distributions of my predicted variable uniformly so it gets an "even" update (i.e., does not overfit to the most probabilistic predicted value)?

Is your model going to be a neural network?

Ideally, I wanted it to be because the system is quite complex.  Is it easier to do for polynomial / linear models?

Ideally, the model should be trained only with new examples. This might help - https://datascience.stackexchange.com/questions/12761/should-a-model-be-re-trained-if-new-observations-are-available.

Hi Supratim, thanks for the reply.  However, training with only the new examples cause catastrophic interference, especially for time series processes where states at time t and t + 1 are very similar.  Imagine a robot having to learn 3 different tasks, and each task takes 1 week.  Whenever the robot is learning one task, because it does it for a straight week and if it learns all new examples only, the model will completely be overfit to the newest data and forget the other tasks.  I am trying to build an algorithm similar to adaptive resonance theory, except with cheaper maintenance cost.

Hi Rui, that's a good point. It's an interesting question, and I'm watching it too and hope to see an well-explained answer soon.

@Rui Nian what do you mean by catastrophic interference?

And, Also, should I set up the update step so it samples data from all distributions of my predicted variable uniformly so it gets an "even" update...  what do you mean by this?

The gradient descent update is biased towards the most recent experiences because the step size parameter (learning rate) is constant.  Because of this, training will learn more recent experiences well and forget past experiences.  That is catastrophic interference: https://en.wikipedia.org/wiki/Catastrophic_interference

For the second point, when performing gradient descent and if you sample data randomly, most of your data sampled will be close to the mean, given your distribution is Gaussian.  Therefore, experiences that are rare will never get sampled and will be forgotten.  By sampling the whole distribution of your data, you will get the true estimate of your data and not overfit to the most common data points.  The topic is kind of like sample mean and actual mean.  Sample mean will always be wrong, but if you sample selectively, you can more accurate estimate your mean.


text:
Do we have cross-language vector space for word embedding?
Do we have cross-language vector space for word embedding?
When measure similarity for apple/Pomme/mela/Lacus/苹果/りんご, they should be the same
If would be great if there's available internet service of neuron network which already be trained by multiple language

You can try to read about MUSE (Multilingual Unsupervised and Supervised Embeddings) by Facebook. You can read it from its Github or this article. They also provide the FastText dictionary format (.vec file) for some languages.
Their original paper shows how it aligns the vector of words from two different languages:

For cross-language word representation the trend now is:

ELMoForManyLangs: git_repo original_paper_March_2018
MUSE by Facebook: git_repo original_paper_January_2018

Remember that you can also do the task in 2 steps:
Translate the words to a reference language (e.g english), then represent each one of them using any word representation model (in the reference language).
The 2-steps option is also good as specific-language word representation models are more accurate, and there are a bench of easy-to-use libraries for single-language translation (i.e py-translator) and representation (i.e Universal sentence encoder by Google).


text:
How do I create syntactically correct sentences given several words?
Is there an AI application that can produce syntactically (and semantically) correct sentences given a bag of words? For example, suppose I am given the words "cat", "fish", and "lake", then one possible sentence could be "cat eats fish by the lake".

It is possible to construct many syntactically correct sentences here, but most of them would be nonsense. For instance "cat pretends to be a fish that has swallowed a lake" or "purple cat sings about yellow fish flying high above a lake" or "lake eats cat by the fish" (not realisable physically, but semantically correct as it has meaning).  Can you describe how the possible sentences should be constrained? Could you give some more examples and some information about your goal?

In my case any sentence which contains these words are OK, if it makes sense.My goal is to describe a scene to join words together, it would be wonderful if there's provided web service online, I can input words and it returns a sentence to me

There might be a service online, but I doubt it would be anything but a toy or research example. Could you please add "if it makes sense" and define that  carefully in the question? You can use the [edit] link to add clarifying details. For instance, do any of my example sentences "make sense". Please add more examples of inputs/outputs that both do make sense, and that do not make sense.

What a difference a few years make. When this was first asked in April 2019, the parts-of-speech tagging required for grammatical correctness was possible but hard, whilst the constraint of being semantically correct was nigh impossible with any technique at the time, other than in carefully controlled scenarios - e.g. strongly limiting the words allowed for input.
Moving forward 3+ years, now one way to do what you want is via a prompt to a pre-trained large language model (LLM), perhaps via an API, which would be possible on e.g. OpenAI's GPT model.
For instance, I did this in GPT 3.5 playground:

Construct a sentence containing the words "cat", "fish" and "lake".
The cat eagerly watched as the fish swam gracefully in the tranquil lake.

If results are all that matters to you, not learning to make such a model from scratch, then this is the way to go. There are open-source LLMs at various scales that you could use, and the task is simple enough (compared to the high end capabilities of the larger LLMs, such as passing bar exams) that even basic smaller ones should be capable.

One way is by defining a grammar for your sentences manually. But that's going to be a trivial task because after creating a grammar you have to create a parsing tree based on your defined grammar to generate a sentence.
Other way is to use LSTM(Long Short term memory) architecture for this purpose but you have to fine tune your model perfectly (which will not give a very good result but it can give you a above average result) or you will get a gibberish.
In this case you can use BLEU (Bil-lingual evaluation under study) instead by evaluating your results manually for selecting the statements with above 0.6 score to get a above average result
You can also check this research paper : https://arxiv.org/abs/1702.04066
Repo : https://github.com/keisks/jfleg
JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction


text:
Why is a mix of greedy and random usually "best" for stochastic local search?
I read that a mix of "greedy" and "random" are ideal for stochastic local search (SLS), but I'm not sure why. It mentioned that the greedy finds the local minima and the randomness avoids getting trapped by the minima. What is the minima and how can you get trapped? Also, how does randomness avoid this? It seems like if it's truly random there's always a chance of ending up searching solutions that lead to dead ends multiple times (which seems like a waste of processing and avoidable)?

As an example of local/global minima, imagine being on a rugged, mountainous landscape, and you want to find the lowest point within some area. For a greedy search, every step you take will take you downhill. If you go downhill long enough, you'll eventually find a flat spot, which is a minimum - from here, there's no step you can take that will get you any lower. However, there's a nearby ridge, which if you crossed it, you could continue downhill to find an even lower spot, the global minimum (the true lowest point). Using your greedy approach, you'll never go uphill to cross the ridge, so you'll be stuck in the local minimum forever. If you occasionally take random steps (other than directly downhill), you have the opportunity to cross ridges that separate local minima, and you have a better chance of finding the global minimum. You are correct that in many cases, the random step won't help you cross a ridge, and will just take you up a mountain in the wrong direction, which is a waste of time. But unless we allow the algorithm to "explore" a bit, it will be content that the first minimum it finds is the best one, and will never get to the bottom.

The greedy action is the action that maximises some quantity in the present or near future. A stochastic action is an action that is random (it can be the greedy action or any other possible action).
For example, suppose that you are hungry. You can either choose to eat pizza, salad, fruits or fish. Pizza is your favourite food. On the other hand, you don't like much salad, fruits and fish, but you know that they are healthier than pizza. If you choose to eat pizza, then this is a greedy action. What quantiy are you maximising if you choose pizza? Your current happiness. If you randomly pick one of those (either pizza, salad, fruits or fish), then this is a stochastic (or random) action. The stochastic action can also happen to be pizza, but, in the next day, it might not be pizza, and, in general, it will not always be pizza.
Suppose that you always choose pizza. What's going to happen? In the long run, you will get fatter and your health will deteriorate. However, everytime you choose pizza, you will get happier in that moment (local maximum). If you randomly choose the food every time you want to eat, then it is more likely that you will eat also salad, fruits and fish. In the long run, this could be more beneficial for your health, hence this could avoid you getting trapped in the local maximum (being happy in the moment that you eat, but unhappier later in life because of the possible health problems).
In the context of artificial intelligence, the ideas are the same. There are several algorithms that use stochastic actions in order to avoid getting trapped in local extrema. For example, simulated annealing, ant colony optimisation algorithms, $Q$-learning (using $\epsilon$-greedy) or genetic algorithms. An example of local search (or greedy) algorithm is 2-opt (for the TSP problem).


text:
Infinite horizon in Reinforcement Learning
I read this article: "Towards Autonomous Data Ferry Route Design through Reinforcement Learning" by Daniel Henkel and Timothy X Brown. It specifies an infinite horizon problem where they use as a reward function for TD[0] the following:
\begin{equation}
r(s,a) = - \int_{t_0}^{t_1} (Ft +N_0)e^{-\beta t} dt
\end{equation}
where $N_0$ and $F$ are constant, $\beta$ is used to adjust the discount factor and $t_0 , t_1$ are the initial and final time.
Then they proceed to use $e^{-\beta t}$ as the $\gamma$ (discount factor) in the TD[0] update formula and in the policy formula.
Why is the discount factor in the infinite horizon problem $e^{-\beta t}$, and why is it used as $\gamma$ in $V(s)$ update, since it is a variable factor?
Also, in the formula of the TD[0] update they don't subtract $\alpha V(s)$. They do:
\begin{equation}
V_{t+1}(s) = V_t(s) + \alpha( r(s,a) + e^{-\beta t_a} V_t(s'))
\end{equation}
I really think this is a mistake, and the values of $V$ will explode without it, even in an infinite horizon problem. Am I correct ? Is $- V(s)$ missing inside the brackets?
Finally, if someone is willing and has the time to directly read this part of the article and enlighten me on if $t_0$ and $t_1$ represent the initial and final time of an action OR $t_0$ is always 0 and $t_1$ is the duration of the action, I would appreciate it. I ask this because from what is written in the paper $t_0$ seems to be the current time in the simulation, but I'm afraid that would just decay too fast and after some actions the reward would be close to 0. It is really not well explained and I'm a bit confused.
Thank you for your time if you got this far reading. Any guideline answer will be very much appreciated.

You should link the article, or at least give the authors and publication, if you hope for someone to read it or refer to it in their answer. Otherwise in order to help you, someone will need to go search for the article. Use [edit] to add details.

Hi Miguel. Please, see this https://ai.meta.stackexchange.com/q/1527/2444.


text:
How can I implement a GAN network for text (review) generation?
How can I implement a GAN network for text (review) generation?
Please, can someone guide me to resource (code) to help in text generation?

As @Clement mentions, text_gen_description gives a good overview!, but the paper seqGAN paper describes the REINFORCE approach more in depth, as they are the first to do it (i believe). This is probably the approach most take now of days when going the GAN route.
Note that just basic MLE training has shown promise with openAI's GPT2. When i need a text generator, fine tuning one of the provided models is usually my goto.
Also if your looking for seq gans code base (you asked for example code) here is is: git repo
Good Luck!

For the resources, you can refer to this:
https://becominghuman.ai/generative-adversarial-networks-for-text-generation-part-1-2b886c8cab10
If you want to generate text review for specific score, you can input a noise vector and the score to the generator. You could also make a vector filled with the number of score and add noise to that vector instead.


text:
How can auto-encoders compute the reconstruction error for the new data?
Autoencoders are used for unsupervised anomaly detection by first learning the features of the data set with mainly "normal" data points. Then new data can be considered anomalous if the new data has a large reconstruction error, i.e. it was hard to fit the features as in the normal data.
Even if the training is supervised by learning to reconstruct the same data, how is the reconstruction error computed for the new data?

Hi, thanks for the detailed answer, I appreciated it. It guided me towards the right direction. However, it was not directly answering my question. When you say, the AE expects specific features to be of a specific magnitude compared to other features, I feel like that is misunderstanding a bit. The AE itself does not reason about the magnitude. I think the point I was missing is that the recon. error is actually just a value for how different the output is to the (ideally identical) input. If the magnitude of features is out of order as you describe, the recon. error most probably increases.

It is computed just like in training. You take an MSE or something along these lines between the input and the output. You set a threshold for it. If new data's reconstruction error is higher than your threshold, then it is anomalous otherwise it isn't.


text:
Has the spontaneous emergence of replicators been modeled in Artificial Life?
One of the cornerstones of The Selfish Gene (Dawkins) is the spontaneous emergence of replicators, i.e. molecules capable of replicating themselves.
Has this been modeled in silico in open-ended evolutionary/artificial life simulations?
Systems like Avida or Tierra explicitly specify the replication mechanisms; other genetic algorithm/genetic programming systems explicitly search for the replication mechanisms (e.g. to simplify the von Neumann universal constructor)
Links to simulations where replicators emerge from a primordial digital soup are welcome.

How "soupy" does the primordial soup need to be here? There are no simulators working at level of atomic/chemical interaction. But if we started with higher level building blocks, they are likely to include some rules that help build replicators.

The simpler, the better. Not interested in modeling on the chemical level. The soup could be a 2D/3D array of integers with arbitrary meaning (e.g. opcodes).

Look at Conway's game of life. Simple rules beget replicators like gliders. It's not too much of a leap to see how this may occur in biology.

@Ray: Gliders, although an interesting emergent behaviour, are not strictly replicators. If you put a glider into game of life, the structure changes and re-creates the same structure after a few steps in a different position. It is more akin to movement than replication. Replication would be if you placed an object and it became in a certain number of steps two separate objects with the same structure.

@Neil  One could argue that "movement" is actually an illusion or perhaps movement is a by-product of replication. Nevertheless, since GoL is Turing complete, there is a configuration that does the kind of replication you ask for.

@Ray: Yes there is. I mention one that was discovered in my answer. It is far too complex to be created spontaneously in any practical experiment though.

@sihubumi I'd love to see examples of such simulations too. Were you able to find anything?

Systems Approach
Let's set out to replicate a real time system $S: \mathcal{X} \Rightarrow \mathcal{Y} \; | \; I$, where $\mathcal{X}$ is a empirical continuous history of input and $\mathcal{Y}$ empirical continuous history of output, conditioned upon a real initial system state $I$.  Based on some definition, we require $S$ to be alive.
We cannot simulate a replication of a theoretical model of life, with a selfish gene or any other such attribute, simply because no mathematically terse model on which the simulation could be based exists.  As of this writing, only hints to and minutia of such a model are known.
Furthermore, models are mathematical representations that, throughout human history, are found to be approximations of complexities once anomalies are addressed and new models develop to incorporate them into the theory.1
Simulation Roughly Defined
If we examine a general algorithm $\mathcal{A}$ to replicate $S$, replication can be roughly sketched as follows.

Estimate system $S$, essentially forming hypothesis $H$.
Simulate initial state $I$.
Initiate a series of discrete stimuli $\mathcal{X}_t$ approximating the real and continuous $\mathcal{X}$.
Acquire resulting system behavior $\mathcal{Y}_t$ as discrete observations of $\mathcal{Y}$.
Verify the difference between simulated and actual systems to be within allowable error $\epsilon$.

Defining Spontaneous Emergence
By spontaneous emergence is meant that such an astronomically large array of initial states and sequences of stimuli occurred that there is a high probability of one of the permutations being alive, based on some specific and reasonable definition of what is living.
Defining What Life Is
Reviewing several definitions of living organisms, the most reasonable definitions include these:

The organism can be distinguished from its environment.
The organism can acquire and cache potential energy and materials required to operate.
Its operation includes continued acquisition, producing a bidirectional and sustainable relationship with its environment.
The organism can roughly reproduce itself.
The reproduction is similar to but not exactly like the parent(s).
The method of energy and materials acquisition may include the consumption of other organisms or its energy and materials.

Competing for resources, natural selection, and all the other features of evolutionary theory are corollary to the above five requirements.  In addition to these, the current trend toward recognizing symbiogenesis as a common theme in the emergence of species should not be dismissed.

Replication of one organism may be influenced by the composition of another organism through forms of assimilation or symbiosis such that traits are passed across categories of organisms.

Artificial Life as a Simulation
These seven criteria poses a challenge for humans attempting to artificially generate life.  It is easy to create a computer model such that life is simulated in some way.  Consider how.

The environment contains virtual energy and virtual matter.
The model of the organism, distinguished from its environment, can acquire its operational requirements from the environment through a set of operations on it.
Mater and energy are conserved because the temperatures are far below nuclear thresholds.
The model of the organism allows acquisition only if enough of the energy and materials acquisition has occurred to maintain the cache.
Mater and energy acquired by one organism cannot be acquired by another organism except by consumption or absorption of an organism that acquired it or produced it from that which was acquired.
The model of the organism can self-replicate in such a way that stochastic differences in the replication is introduced in small quantities.
Operational information, including replication information, may be acquired through consumption or symbiotic relationship under some conditions.

Magical Genes for Spontaneous Life
Notice that the selfish gene is not mentioned above.  Selfishness, the prerequisite of which is intention, is not a requirement for life.  An amoeba does not think selfishly when it moves or eats.  It operates witlessly.  We should not anthropomorphize every organism we study, or develop theory based on anthropomorphic conceptions.
Similarly, symbiotic relationships form that are neither loving nor altruistic.  They exist because there is a mutual benefit that appeared as an unintended byproduct of normal operations and both symbiotic parents happened to pass that symbiotic connection to their respective offspring.
The mutual benefit, the symbiosis, and the replication are witless and unintended.
There need not be a control mechanism distinct from all other replicated mechanisms to control either symbiotic collaboration or competition.  They too are natural consequences of living things sharing an environment.  Whether an organism dies because it

Lost its symbiont,
Starves because other organisms consumed its necessities,
The organism itself depleted its own resources, or
Those needed resources were otherwise rendered unavailable,

it is still unable to replicate, so its traits die with it.
Note also that there is no known molecule that can replicate itself.  Complex systems of molecules in a variety of chemical states and equilibria are required for reproduction to take place.
Returning to Simulating an Already Existing Organism
Running a time sharing system or distributing these simulated organisms in a parallel processing arrangement may some day simulate a biosphere, but it is not one in that only transistor electro-chemistry is involved.  There is no actual direct relationship between the energy and mater of the system used to assemble the simulation and the energy and mater of the simulated environment in which the simulated systems $S$ reside.
Certainly genetic algorithms, such as Avida and Tierra, have been developed.  Compare those simulations to the modelling scenario described above, and their deficiencies become clear.  Human researchers have not yet found $\mathcal{A}$ to replicate $S$ in a way that aligns with biological reality.
Open-endedness Requires Verification to Have Merit
The most significant limitation on implementations in silico, is that they can never be truly open-ended.
There is no way as of this writing to replicate that which was simulated outside the simulation system.  Until nanotechnology reaches a point where 3D construction and assembly can migrate alive simulations into the unsimulated universe, these simulations are closed-ended in that way and their viability in vito is untested.  The value of open-ended simulations without any way to validate them is essentially zero except for amusement.
Even in the space of digital simulation, as far as that technology has progressed, nothing even close to von Neumann's universal constructor has been accomplished.  Although generic functional copy constructors are available in Scheme, LISP, C++, Java, and later languages, such is a minuscule step toward living objects in computers.
Digital Soup
The simulation of life's origins is considerably more difficult than finding an algorithm $\mathcal{A}$ to replicate $S$, where $S$ is a single life form and a sufficient portion of its environment to be representative of the biosphere on earth with an organism in it.
The issue with primordial digital soup is one of the combinatory explosion.  There are 510 million square Km on the earth's surface, and there are only three categories of life origin time frames possible.

The current estimates are close to correct, that the earth formed 4.54 billion years ago and extremely primitive life emerged 3.5 billion years ago
The organic material found in Canada that is allegedly 3.95 billion years old shortens the gap between planetary formation and life formation on it and older terrestrial life may be found
Vladimir Vernadsky's comment that life may have preexisted earth is more than just a possibility

If we go with the 1.04 billion year gap, then $(4.54 - 3.5) \cdot 10^9 \cdot 510 \cdot 10^6$ Km-years of soup must be simulated, since we cannot assume that life started in the ocean or a puddle or even on the surface.  It could have started underground or in the atmosphere.  The biosphere is currently thought to be 1,800 m above to 8,372 m below thick.
With nanobes being 20 nm in diameter and the possibility that the emergence may have only taken one second we have to simulate in three dimensions over time the following space-time domain in finite elements with at least 50% overlap in all three dimensions.
$$\dfrac {2^3 \cdot (4.54 - 3.5) \cdot 10^9 \cdot 510 \cdot 10^6 \cdot (1,800 - 8,372) \cdot 365.25 \cdot 24 \cdot 60 \cdot 60} {(20 \cdot 10^{-9})^3} \\ = 170,260,472,379 \cdot 10^{9+6+27} = 1.7 \cdot 10^{56}$$
With a quantum computer two stories high the size of Switzerland, the computing time would vastly exceed the duration of the average species on earth.  Humans are likely to be extinct before the computation completes.
As the dating of the oldest found fossils converges on the dating of earth, it may seem that life emerged quickly on earth, but that is not a logical conclusion.  If life formed as soon as the earth cooled sufficiently and no evidence of continuous emergence is found in the remaining billions of years, then Vernadsky's inference that life arrived on earth through one or more of the bodies that struck it becomes more probable.
If that is the case, then one must ask the question, if all assumptions are dropped, whether life had a beginning at all.
Simulating Life Versus Simulating Its Formation
We may simulate what life is, that is, find an algorithm $\mathcal{A}$ to replicate $S$, where $S$ is a single live organism.  It is not realistic to, by brute force, simulate how life began without learning more about what conditions can lead to its formation theoretically to drastically reduce the soup simulation space.  It is that learning that is an ongoing area of research in the genetic algorithm field.
Early musings about the possibility of an algorithm $\mathcal{B}$, which can provide the conditions that allow an arbitrary organism $S$ conforming to the above definition of life to form with out a parent or parents were interesting. Given algorithm $\mathcal{A}$ that simulates a life form and algorithm $\mathcal{B}$ that simulate the formation of life, it may be the later algorithm that proves significantly more difficult.
Conforming physics outside a computer to the simulation may be impossible.  Whether simulated life, when embodied in a robotic system is actually going to be considered life will be left to our descendants, should the species endure sufficiently.
Footnotes
[1] Classic cases include the heliocentric Copernican system giving way to the Law of Gravity, that law being shown an approximation of general relativity as shown by the proper prediction of the orbit of Mercury and light's curvature near the sun, the Four Elements dismissed in light of Lavoisier's discovery of oxygen, and absolute provability of truth within a closed symbolic system disproved by Gödel in his second incompleteness theorem and then recouped partially (in terms of computability) by Turing's completeness theorem.

Although difficult to prove a negative, I don't think that this has been done.
The most advanced simulations of low-level features are not capable of scaling to simulate large enough populations at large enough time scales where scientific consensus claims that this has happened in reality.
Although you say that you are not directly interested in chemistry, but some abstract substrate, I am using chemistry as an example of the challenge. That is because creating a simplified substrate with enough rich emergent behaviour is non-trivial. The chemical elements essentially have rules about how they combine into larger physical structures (via different bonding mechanisms) and only roughly a dozen types of atom are involved. It's actually reasonably simple and tractable at the lowest level. The problems come from the multiple scales of structure - building "unit" molecules (DNA/RNA bases, protein peptides, lipids, sugar bases etc), creating polymers from those units, interactions between polymers, physical structures built and torn down by those interactions, each of which exhibits more complex behaviour. This structural hierarchy is likely required for any self-replicating machinery that is not simply being fed the higher-level units directly. In your question you want to find self-replication that is emergent, not designed . . . so feeding in these higher-level units would probably count as cheating.
We probably don't have the computational power to properly simulate even the Miller-Urey experiment which is far from self-replication - chemical simulations in silico are limited to things like protein folding calculations, and these are far from real-time. Inside just a single bacterial cell getting ready to divide, proteins are produced and fold by the hundreds every second.
One thing that has been done is to create a self-replicating machine in Conway's Game of Life called "Gemini". This was designed, not spontaneously created. However, it would have a very low but non-zero chance of spontaneously being created with random initialisation. It would be a very fragile replicator though, any mutation or collision with other active elements would likely break it. The experiment of attempting to randomly/spontaneously create Gemini is not computationally feasible.
It is likely that any physical system simple enough to be considered "primordial soup" yet rich enough to express replicating units is going to require a few layers of construction before you get to see those units. These layers need to be built up combinatorially, and odds of this happening spontaneously in a small experiment with limited computation appear to be low. You need to bear in mind the extremely large computation that would effectively have been done by an order of $10^{30}$ molecules that can interact at very fast rates in parallel (compared to rate at which these same interactions can be simulated in current CPUs), with processing times of the order $10^8$ years. It is mainly conjecture that this is enough to create an Initial Darwinian Ancestor - it is basically a logical extrapolation to the theory of evolution, following the Occam's Razor principle of looking for simplest compatible explanation.

It's yes in cellular automata space, you have this old one from 1997:
"Emergence of self-replicating structures in a cellular automata space"
http://fab.cba.mit.edu/classes/S62.12/docs/Chou_CA.pdf
I quote you the part of the abstract you'll be interested in:
"This article demonstrates for the first time that it is possible to create cellular automata models in which a self-replicating structure emerges from an initial state having a random density and distribution of individual components."
And those replicators carry some kind of information and are in some minimalistic sense evolvable (the loops can grow).
But, you'll see the emergence is mainly due to the fact that the cellular automata universe designed allow small replicators, and by just being chaotic enough those replicators form by luck. It's an Langton loop system, you have a live demo there (but quite old as well):
https://www.complexcomputation.org/hhchou/research/research_ca_demo1.html
I dont know any other work that come close to that in term of simulating emergence of replicators, it's old and with limitations but at least it's something.
Regards.

Primordial replicators can be simpler than you think. Check out this video:
Self Replication: How molecules can make copies of themselves
[Source: University of Groeningen]
In a noisy environment you get natural mutation. And voila, replication + mutation = evolution.

It's a good link, and relevant to the conversation. However, link-only answers are not considered high quality here. It would be helpful to summarise the video in more detail, and also relate it back to the original question - which is about having this kind of organisation occur spontaneously in a silicon/code environment (whilst the video is a thought experiment which seems feasible for a chemical system).

I actually don't think a summary would help; some things are better shown than explained.  But please feel free to add an answer if you like.  Also, it should be easy to imagine the solution in silico.  Actual implementation details wouldn't add much.

I have already added an answer. If a summary would not help, then please find some other way to make this answer self-contained. It is a goal of the site that question and answer pairs should not rely on links. A good way to assess this: If your link stopped working, would the answer still be complete and useful?