text:
ChatGPT claims that it doesn't get information from anywhere, so can't say why it gave a blatantly false statement. How is that possible?
I asked ChatGPT a question, and its response included this statement:

In "The Ruling Class" film, which was released in 1972, Patrick McGoohan played the role of Bishop Lampton.

Unlike many of its vague or weaselly responses, this is a direct statement of fact.
But it is false and very easy to disprove, so I wondered how it came up with that misinformation?
I asked it how, and it apologized and admitted it was wrong:

Q: You say "Patrick McGoohan played the role of Bishop Lampton", when in fact it was Alastair Sim that played that role.  Where did you get this misinformation?
A: I apologize for the error in my previous response. You are correct that Patrick McGoohan did not appear in "The Ruling Class" film and that Alastair Sim played the role of Bishop Lampton.
I apologize for any confusion or inconvenience that my previous response may have caused. As an AI language model, I strive to provide accurate information, and I will make sure to double-check my sources more carefully in the future. Thank you for bringing this to my attention.

But it avoided answering the question:

Q: But my question was "Where did you get this misinformation?".
A: I apologize for the confusion. As an AI language model, I do not have personal experiences, so I did not get the misinformation from anywhere. I must have made a mistake in my response due to an error in processing or a misunderstanding of the question.

It wouldn't have just made it up in order to provide an amazingly good answer,
so how can it give a false statement of fact and claim that it didn't get the information from anywhere? (I.e. what mechanism does it use to generate facts if they don't originate somewhere else?)
And worse, it actually lied while apologizing:

I will make sure to double-check my sources more carefully in the future.

You should not trust anything that ChatGPT says, its a Stochastic Parrot, and you are asking us about mechanisms, when there is no public information about its inner workings.

The current term for this phenomenon is "hallucination" . It is very easy to induce, once you know how. https://medium.com/@cooper.white_86633/ai-hallucinations-the-ethical-burdens-of-using-chatgpt-5e62dd0f75c7

From what I experimented Chatgpt is not very good with movie.  It seems to check only a few term of the question and return something that is induced from these terms, not the full question.

I think the term parrot is very misleading. It is not small, it is not in a cage and it moves as a lot of us move. I prefer the name myrianthrope.

The reason is that ChatGPT is not a data retrieval system.  In other words, it is not consulting a database, nor is it performing searches of the internet, to find answers to questions posed.
Instead, it is generating novel answers based on the trained parameters within the network.  In a sense, this is far more impressive.
If you think about it, ChatGPT and large language models generally provide a very interesting method of compressing data.  Of course, compression isn't the point of LLMs, but you can view this as one of the outcomes.  How so?
During training, the parameters are learned such that the model is able to reasonably predict the next most likely token (word/bigram/trigram/letter/symbol) based on all of the tokens that have come before it in that session, including any base prompt that is included silently.  When you ask it to recite something like The Lovesong of J. Alfred Prufrock, it can do so very accurately... yet, it does not have a copy of Prufrock "memorized" somewhere.  Instead, it is recreating Prufrock generatively.
This is also why its answers, while incredibly confident, can be wildly incorrect.  It is essentially generating the most likely token that comes next, not reasoning or thinking about what the text means.

How can we assess to what extent it compresses or rather memorizes data?

That's a super question.  Intuitively, we could evaluate the billions of parameters in the model and attempt to recreate data from them directly... However, since they are in an encoded representation (I would think of it as an eigenvector of some high dimensional space), that would be super challenging.  Definitely an interesting thing to study though!

its also funny that it claims it's going to "double-check" its sources in the future, considering it has no sources and it's not performing any sort of review on what it says

Incalculable times i caught it literally making things up it does not know. For example an hour ago i asked it if Arnold Schwarzenegger says the line "i don't want to hurt you" in any of his movies. It first correctly returned no, then i wrote try harder and it apologized and said he says it in Batman and Robin 1997. I asked for exact minute it said 1:03, i looked no such line, then it said it's at 1:26 then i looked and there said there is no line there either. Then it apologize again and said there is no such line. So It lied 3 times, totally making it up. And i have personally caught it lying already countless times.

Even at technical questions, it really wants to please. Coprime Matroid looks like a formal mathematical proof, but some of the smaller details don't actually make sense. It would likely fool a non-mathematician. What's really impressive though is that even though there is actually no such thing, it invented the name "the coprime matroid", which would have been the perfect name for it had such a thing actually existed, making it look like it really had researched the answer and found various references to it using this name.

I know, i been talking to it about physics and electrical engineering, when it does not know it simply makes it up and pretends it's a fact altho it is 100% bs. Like this guy puts it "it creates authorative bullshit" https://youtu.be/1wzPr4cUoMQ?t=479


text:
How does a CNN work in detecting absence of features?
I'm trying to understand how a CNN operates internally. Let's say I'm doing binary classification with 1 output neuron and a sigmoid to classify dog vs no dog. No dog meaning the image does not contain a dog.
So what features does a CNN learn for no dog? Do the convulutional filters just learn the features of dogs and predict no dog when they aren't detected? Or do they learn some arbitrary features present in the no dog images?
Basically, I'm trying to understand how CNNs work to detect "absence of features."

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Combine two feature vectors for a correct input of a neural network
Let's consider this scenario. I have two conceptually different video datasets, for example a dataset A composed of videos about cats and a dataset B composed of videos about houses. Now, I'm able to extract a feature vectors from both the samples of the datasets A and B, and I know that, each sample in the dataset A is related to one and only one sample in the dataset B and they belong to a specific class (there are only 2 classes).
For example:
Sample x1 AND sample y1 ---> Class 1
Sample x2 AND sample y2 ---> Class 2
Sample x3 AND sample y3 ---> Class 1
and so on...

If I extract the feature vectors from samples in both datasets , which is the best way to combine them in order to give a correct input to the classifier (for example a neural network) ?
feature vector v1 extracted from x1 + feature vector v1' extracted from y1 ---> input for classifier
I ask this because I suspect that neural networks only take one vector as input, while I have to combine two vectors

The easiest way can be the concatenation of the feature vectors to create a single feature vector for each sample.
Assume the first sample is made of the pair $X_1$ and $Y_1$. Let the corresponding feature vectors for $X_1$ and $Y_1$ be $\textbf{v}_1$ and $\textbf{v}_2$, respectively.
$$
\textbf{v}_1 = [f_1, f_2, \ldots , f_n],\\
\textbf{v}_2 = [g_1, g_2, \ldots , g_m].
$$
Then, the first sample's feature can be defined as
$$
\textbf{v} = [f_1, f_2, \ldots , f_n, g_1, g_2, \ldots , g_m].
$$
Eventually, when you pass the latter feature vector to a machine learning model, it will try to capture the dependencies among all of these features, to learn a solution for your task of interest (i.e. classification).

Is it a problem if the dimension of v1 and v2 is quite big ? Like 8000 ?

It also depends on the number of samples that you have in your dataset. In other words, you need enough samples to be able to solve your problem in that high dimensional space. If you want to learn more please refer to subsection 5.11.1 The curse of dimensionality on the Deep Learning book (available at https://www.deeplearningbook.org/) [1].
As a quick solution, you may use dimensionality reduction techniques to reduce the number of features that you have.
[1] Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.

$^*$Note - Question is bit unclear, in case the answer doesn't addresses the question, please ask for edit/delete Request.

GENERALIZATION
Suppose there are multiple datasets denoted by $A_i$. Datasets contain a set of Vectors $x_{j} $. Mathematically $A_i = \{ x_j\}_{j=0}^n$. We've to find an estimator function $\hat f$, such that $\hat f( \vec r) = y, \, \vec r \in X$ where $X $ is a special dataset created by combining all $A_i$ which helps in classification into $y \in Y$ which is the set of classes.
.
As @Amir Mentioned out, linearly separable feature can be easily separated by straight combination of vectors i.e. if $x_u \in A_i, w_v \in A_j \dots$, then $r = [x_1 \,x_2 \, \dots  \, x_u \, w_1 \, \dots w_v \dots]$. Where, $r \in X$ which is the required dataset.
There are cases where the features are not linearly separable, We use basis expansion methods[1] to make required shape of hyperplane to separate the features. We create a new dataset combining $A_i \, \forall i \in C \subset \mathbb N$. Suppose that the new dataset is $X$, then $r \in X$ and $r = [r_0, r_1, \dots r_n].$
Then,
$$r_1 = u_1^2v_1^2 \\ r_2 = \sin(u_2)\sin(v_2) \\ r_3 = ae^{u_3 + v_3} \\ r_4 = a v_4 v_4 + a_2 u_4^2 v_4^2 + \dots \\ \dots$$
Here $u_p \in A_i; \, v_q \in A_j$
Here you can use all the creativity to set $r = [r_1, r_2, \dots , r_n]$ and make a new dataset. What equations and what functions you chose fully depends on the kind of hyperplane shape you want to obtain. Basis expansion is just one of the methods for feature extraction is certainly one of the most flexible too.
Now, you feed the newly created vectors into your trained estimator functions (which is Neural Net) which can classify things much easily now.
In case of Regression/Classification without Neural Net needs some extra treatment to train the model[2].

[2]Note: There is also a big role of encoding. For example, if you encode colors by numbers $1, 2, 3$ for RGB or $10,01, 11$ fully changes everything and your features too. In such cases, You may even need different equations to make your required dataset $X$ and vectors $r$.

REFERENCES:

Oleszak, Michal. https://towardsdatascience.com. Non-linear regression: basis expansion, polynomials & splines. Sep 30, 2019. Web. 6 May 2020.
Sangarshanan. https://medium.com. Improve your classification models using Mean /Target Encoding. Jun 23, 2018. Web. 6 May 2020.

I really appreciate your help, your answer is fantastic. How can I know if the two input datasets are linearly separable, so that I can use the @Amir method ?

@AleWolf I'll just put some short points.

Visualize Data to see that the features which you've combined is linearly separable or not.
In case you are not sure or don't have any idea, there's quick and dirty way is to use Deep Networks instead of Neural Networks with number of layers depending on intricacy of Features. Beware - Deep Nets require usually 2-3x more datasets than Probability based classifiers (Usually, but that depends).
The kind of function you need depends on features just like $\sin x , e^x$ are different. Prefer going to links, I've attached to see more.

Thats great, last question. Why intuitively, if the two datasets are linearly separable I can use the combination of vectors method (exposed by @Amir), yet if they are not it's better to use the basis expansion methods ?

@AleWolf Linearly separable means, a straight line can easily divide the classes into 2 equal parts (which in practice, happens less often), Basis expansion gives you method to draw whatever shape of hyperplane, any curve, or even closed shapes when fitted properly (not over fitted/under fitted), they tend to separate any features as you'd like.

But there are drawbacks to it (like you'd need to to have functions that are capable of that kind of curve), but when you use DeepNets, it eases the pain, since, it automatically converges to the required curve without any problem. It works by the principle of approximation and can approximate anything. Hope that answers your question... Please don't forget to read this https://towardsdatascience.com/non-linear-regression-basis-expansion-polynomials-splines-2d7adb2cc226

What i dont understand is the relation between "concatenating two feature vectors" and their "linear separability"

I think you have not properly understood my original question. I'm sorry :( I've probably explained it badly. I'll try to explain it better: I've two datasets A and B. Now, a is a vector that belong to A and b is a vector that belong to B; is correct to use a|b as an input to a deep neural network or there are better ways to do it?

@AleWolf Yes, if you are using deep nets, then you can do that

Thanks so much. Is a problem if the dimension of the combined vector is very high ? Like 16000 ? (Each feature vector has a dimension of about 8000). If yes, is there a way to deal with it ?

@AleWolf I'd suggest using Deep Convolution Neural Networks.. or Deep CNN

My idea was Radial Basis Function neural network because the input dataset (created by concatenating the feature vectors) is quite small. What do you think ?

@AleWolf Sorry, I know heart of RBF ANN, but haven't experimented enough with it to suggest you that. In case you've tried that RBF/ANN before, then try experimenting/tweaking it a bit.

So in your opinion i should try both RBF and Deep CNN and watch which performs better ?

@AleWolf Yep, try :)

I really appreciate your help. Thank you infinitely :)

:) Keep Going..

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Is this the correct gradient for log of softmax?
I am currently implementing the very basic version (REINFORCE) of the Monte Carlo policy gradient algorithm. I was wondering if this is the correct gradient for the log of softmax.
\begin{align}
\nabla_{\theta} \log \pi_{\theta}(s, a)
&=
\varphi(s, a)-\mathbb{E}\left[\varphi(s, a)_{\forall a \in A}\right] \\
&=
\left(\varphi(s)^T \cdot \theta_{a}\right)-\sum_{\forall a \in A}\left(\varphi(s)^T \cdot \theta_{a}\right)
\end{align}
where $\varphi(s)$ is the feature vector at state $s$.
I am not sure if my interpretation of the equation is correct. I ask because, in my implementation, my weights ($\theta$) blow up after a few iterations, and I have a feeling the problem is in this line.


text:
Which hyperparameters in neural network are accesible to users adjustment
I am new to Neural Networks and my questions are still very basic.
I know that most of neural networks allow and even ask user to chose hyper-parameters like:

amount of hidden layers
amount of neurons in each layer
amount of inputs and outputs
batches and epochs steps and some stuff related to back-propagation and gradient descent

But as I keep reading and youtubing, I understand that there are another important "mini-parameters" such as:

activation functions type

activation functions fine-tuning (for example shift and slope of sigmoid)

whether there is an activation funciton in the output

range of weights (are they from zero to one or from -1 to 1  or -100 to +100 or any other range)

are the weights normally distributed or they just random

etc...
Actually the question is:
Part a:
Do I understand right that most of  neural networks do not allow to change those "mini-parameters", as long as you are using "readymade" solutions?
In other words if I want to have an access to those "mini-parameters" I need to program the whole neural network by myself or there are  "semi-finished products"
Part b:(edited)
For someone who uses neural network as an everyday routine tool to solve problems(Like data scientist), How common and how often do those people deal with fine tuning things which I refer to as "mini-parameters"? Or those parameters are usually adjusted by a neural network developers who create the frameworks like pytorch, tensorflow etc?
Thank you very much

You can look at the API of whichever library you're using to see which hyperparameters you can set yourself, e.g. scikit-learn lists a bunch. The default values are decent though.

Often you'd need to adjust the weights yourself so it's in the expected range (which is called "normalisation"). The distribution of values may affect the result and there may hyperparameters to indirectly deal with an improper distribution (like providing weights for different classes), but I wouldn't expect there to be an parameter for specifying the distribution directly. I don't think it would be able to do too much with that information.

NotThatGuy , thanks.
Can you please make me some order in understanding.
Is it right to say: that things like pytorch, tensorflow, keras, scikit learn are alternatives one to another? Thay all allow to build me a desired network with
all hyperparameters ibe mentioned?
Are they all called "frameworks"?

@Igor I'm unfortunately not familiar enough with the others to be able to answer that, but you can certainly create neural networks using scikit-learn (with whichever hyperparameters it supports). Wikipedia calls that a "library". I'll leave it to other people to explain the difference between a framework and a library and judge what is a library and what is a framework.

NotThatGuy , thanks again. You got my question right. BTW It is not that I am trying to learn all frameworks at once, I just understand things better when I have some kind of taxonomy (tree like structure)of relevant terms inside my head. Like which term is a private case of which. Which term is a generalization of some other terms, and which terms are equal in hierarchy (they may be not totally equal but more alternatives)

In general, many of the parameters you mentioned are called hyperparameters. All hyperparameters are user-adjusted (or user-programmed) in training phase. Some hyperparameters are:

learning rate,
batch size,
epochs,
optimizer,
layers,
activation functions etc.

To answer your (a) part of your question, there are obsiously many frameworks and libraries, for example in python; TensorFlow, pytorch and so on. You might never create a net from the very beginning; maybe only in order to understand the forward and backpropagation algorithms. When we call from scatch networks, we mean that these networks are trained from scratch, with learnable weights and chosen hyperparameters; with no transfer learning.
To answer your (b) part of your question, I can understand from it that you mean when a net is good enough. Dependently of your data, of course, a neural network is good enough, when it is trained adequately on them. That is, you should be aware of overfitting, underfitting, and in general of the model you are trying to train with all its parameters and hyperparameters.
Since, you are at the very beginning with Machine Learning, I propose you read some books, in order to get everything needed, in terms of Mathematical and Computer Science aspects.

Hi thanks for answer.
About part a:
Do I get right that "FRAMEWORKS" like those you had mentioned (tensor-flow and pytorch) are NOT ready architectures but kind of a sandbox which allows me to build my own? including the adjusting all hyperparameters like those I've mentioned above? (weights range, weights distribution, option to decide in  layers to use or not use activation functions, and if use which kiind of them)... Do I get right?
About part b: is not what  mean. I'll edit this part of a question.

@Igor as ddaedalus mentioned above, "You might never create a net from the very beginning; maybe only in order to understand the forward and backpropagation algorithms". I am sure with you with the involution of frameworks in this machine learning era, nothing you can't do with them. You can create a new model, tune all parameters, all weights range, change other SOTA model, and they also open the source code of those frameworks, so if you have any problems, feel free to change it, or contribute it to the community.

There are some differences between those frameworks but it doesn't affect you right now, just choose which has syntax you feel comfortable with and try to read and understand as much as possible. Welcome to this world!


text:
Darknet as a part of Yolo v3
I am pretty new to ML and my question may look strange. Especially the last part of it.
1)As far as I understand Darknet53 is an integral part of Yolo just as Resnet50 is a part of R-CNN Am I right?
2)On the other hand I understand that the R-CNN alternatively can be used based on VGG instead of the Resnet, 16 am I right?
3)Can YOLO also be used based on something else but Darknet53 ?
4)If not what was the use in defining Darknet53 as not integral part of YOLO but as some separate section with its own name? Maybe it has some non-Yolo use?
5)I have almost finished an online course in YOLO-V3\4 and I am quite sure that in this course they do not use nor Pytorch nor Tensorflow nor Keras, but when I read  online tutorials they often do mention at least one of them relating to YOLO. So what do they actually do? And what is their advantage comparing to not using them at all as in a course I am doing now?
Thank you very much.

Lets start by listing what is what.

RCNN : Is a type of CNN Model
Resnet50, DarkNet53 and VGG : Are implementations of a CNN Model

Now moving to your questions.

Yes Darknet53 is the backbone of Yolo3. No, Resnet50 is not part of RCNN. Its just one implementation of a CNN model.

Yes

Yes

N/A

Pytorch,Tensorflow and Keras are frameworks for ML. They make it easier to implement and play around with neural networks. You of course do not "Need" them and can write everything from scratch. Using frameworks just makes your life easier.

Thank You sir. I start getting (hopefully) the terminology.
About question  5:
In the the forum of the online course I am taking, teacher answered  as follows about   YOLOv3\4

to RUN pretrained model I can use : CV2 or Darknet  OR TENSORFLOW...
to TRAIN it I can use: Darknet OR TENSORFLOW
Does it make sense? I am confused since I thought That DARKNET is also a type of CNN, how can it be replaces with TENSORFLOW which is framework? I miss something very important' please help me fill the blanks in the taxonomy of terms and subterms .
Thank you very much.

I see you are probably confused between Darknet and Darknet53. Darknet is a framework for neural networks just like Tensorflow. While Darknet53 is one implementation of a CNN model used as Yolo's Backbone. Darknet (the framework) is popular because the original Yolo implementations were in Darknet framework.


text:
Do the True Negative detections in context of Object Detection make the Accuracy metric meaningless?
It seems to me that in context of object detection True Negative "Detections" amount is an infinite number since counting an absence of detected bounding boxes can be described by
an infinite number of not existing boxes of different sizes and locations.
If I am correct, than such metric as Accuracy has no meaning since it includes TN in both - the numerator and the denominator.
Please help me pinpoint my mistake if there is.

All metrics are assessed against test data and/or estimated against a theoretical population. This population is often assumed to be a "target" population, i.e. it contains relative numbers of the values of interest that are likely to occur when a model is used for real.
So, if in real use for a classifier, your positive class detections are a tiny percentage of observations, then yes this may mean accuracy is impractical. However, you should not base this on some theoretical but non-existing population such as calculating the negative class for all possible images. If your model is being used to count parked cars in a car park, then the positive/negative balance in practice doesn't need to take account of what the model would do with random artworks. If the model would not be used overnight, it may not need to be assessed against empty parking lots in the dark.
There is a related issue for using accuracy as a reporting metric, which is a real concern. If a model would perform "well" with zero processing by e.g. always predicting the negative case, then a measure of accuracy is not very useful when reporting results. It's hard to show a useful linear difference between a 99.9% accuracy from a fixed value prediction, and a 99.9% accuracy from a smarter model that actually predicts things from related variables (but that could get a few false positives as well as false negatives). In those cases, you should search for some other metric that helps assess how useful a model would be, perhaps based on some feature that relates to the impact of basing decisions on the prediction.


text:
How is this z-loss implementation in t5x related to this paper's loss X?
I was looking into the loss function in t5x here and see there is a z-loss added to the typical log loss definition.
The only paper I could surface on this was https://arxiv.org/abs/1604.08859, but I don't see the correlation. In the code, it is logsumexp of the logits squared.
But in the paper it is calculating mean/variance for the z-loss you can see the author's implementation here.
I am asking for help either connecting the two or looking for resources on the current implementation if they are in fact unrelated.

Could you please reformulate the title as a question? Like "How is this z-loss implementation in t5x related to this paper's loss X?"

Sure, I changed it.


text:
How do we determine the membership functions and values for this problem?
The goal of our project is to identify the quality of a grain and we have two values, A and B.
A can take the values low  L, medium M and high H. And B can take the values of Low, Medium and High (L,M,H). Specifically, the ranges for A are -> 5-10 (low), 10-15 (medium), 15-20 (high). And the ranges for B are -> 50-75 (low), 75-90 (medium), 90-110 (high)
The output for these are bad B, average A and good G.

How do we determine the membership functions and values for this?
We want to write Python code for the fuzzy system, but we are beginners and have no idea how to go about this. Any help would be appreciated.


text:
How can I approximate a function that determines the priority of objects?
I am facing the following supervised learning problem:
An object is fully characterized by its position in $R^n$. There are $m$ objects. There are fully observable (i.e. their positions are always known).
At each time step $t$, exactly one of these objects is activated. Activation is fully observable, i.e. the index $a_t$ ($a_t \in [1,m]$) of the object activated at time $t$ is known.
We know that, under the hood, activation works this way:  there is a priority function $f$ ($f: R^n \to R$ ), which computes, for each time step, the priority score of each object. The object for which the priority score was the highest is activated.
The goal is to find (the approximation of) one of the possible priority functions that would match a given data-set. A data-set is of size $(m*n+1)*t$ ($m$ positions of dimension $n$, plus the index of the activated object, over $t$ time steps).
As an example, if it turns out there is a hidden fixed beacon, and at each time step $t$ the object the closest to the beacon is activated, then a possible function would be $f(o_{it})=1/d_{it}$, where $d_{it}$ is the distance between the beacon and the object $o_i$ at time $t$.
(If several objects have the same highest priority score, then only one of them is activated, selected randomly).
The function found by the algorithm may be parametric and encoded by a neural network, if this is applicable.
Is there a method for finding one such function ?


text:
How MSE should be appliead with multi target deep network?
I'm having a problem understanding how the MSE should be used when working with a multidimensional target, e.g 3 dimensiones. (My outputs are continuois values, not categorical)
Let us say I have a batch size of 2, to make it simple; I pass my input in the network and my y_pred would be a 2x3 tensor.
The same happens for y_true, 2x3 itself.
Now the thing I'm not sure of: I take first the difference, diff = y_true - y_pred; this maintains the dimension.
Now, for MSE, I square diff, obtaining again a 2x3 tensor, is that right ?
Now the tricky part (for me): I have to Mean. Which one should I consider:
Mean all the (six) values obtaining thus a scalar ? But in this case I do not understand how the backpropagation would make better on specific targets
Mean by rows, i.e, obtaining a 2x1 tensor, so that I have a mean for each example ? Also here I cannot get how the optimization would work
Mean by columns, i.e, obtaining a 1x3 tensors, where I obtain thus an "error" for each target? This seems the more logical to me, but I'm not so sure.
Hope this is clear


text:
How could self-driving cars make ethical decisions about who to kill?
Obviously, self-driving cars aren't perfect, so imagine that the Google car (as an example) got into a difficult situation.
Here are a few examples of unfortunate situations caused by a set of events:

The car is heading toward a crowd of 10 people crossing the road, so it cannot stop in time, but it can avoid killing 10 people by hitting the wall (killing the passengers),
Avoiding killing the rider of the motorcycle considering that the probability of survival is greater for the passenger of the car,
Killing an animal on the street in favour of a human being,
Purposely changing lanes to crash into another car to avoid killing a dog,

And here are a few dilemmas:

Does the algorithm recognize the difference between a human being and an animal?
Does the size of the human being or animal matter?
Does it count how many passengers it has vs. people in the front?
Does it "know" when babies/children are on board?
Does it take into the account the age (e.g. killing the older first)?

How would an algorithm decide what it should do from the technical perspective? Is it being aware of above (counting the probability of kills), or not (killing people just to avoid its own destruction)?
Related articles:

Why Self-Driving Cars Must Be Programmed to Kill
How to Help Self-Driving Cars Make Ethical Decisions

I will not ride in any vehicle that is programmed not to make ME the passenger its top safety priority. I am not alone in that thinking. If anybody has plans to force autonomous vehicle makers to program their vehicle to do anything other than prioritize the safety of its passengers then the entire industry might as well close their doors now. The first couple of times a passenger dies because of some abhorrent ethical decision will be the last time the vast majority of people would ever purchase another autonomous vehicle. Who wants to ride in a death trap?

The so-called "trolley problem" is a troll-y problem. There's a reason it's known as a thought experiment rather than a case study: it's not something that has ever happened. It's not even close to being real, largely because cars are fundamentally not trolleys. The right answer in any remotely applicable case is virtually always "hit the brakes," and if there is ever an exception, it can only possibly be resolved one way: by protecting the inhabitants of the car at all costs and never even considering anything else, for two reasons.

The first, @Dunk covered quite well: pragmatism.  If it's possible for the car to make a different choice, no one in their right mind would want to buy one. The second is that if there's a "kill everyone" function built in to the car's computer, that means it's already there for malicious hackers to find and abuse. Weigh the very real chance of someone figuring out a way to trigger that with adversarial input against the purely fictitious idea of a legitimate trolley-problem situation, and there's only one possible answer that is not morally absurd and abhorrent.

@MasonWheeler says "it's not something that has ever happened". Actually it has: Trolley problem - Wikipedia.
There are also instance of people deliberately crashing, and usually fatally, in order to avoid killing other people, such as Sheriff: Truck driver who died in I-94 crash drove off the road to save lives

Related video: TESLA AUTOPILOT VS IDIOTS IN CARS - 15 CRASHES, FAILS & SAVES | TESLACAM STORIES.

How could self-driving cars make ethical decisions about who to kill?

It shouldn't. Self-driving cars are not moral agents. Cars fail in predictable ways. Horses fail in predictable ways.

the car is heading toward a crowd of 10 people crossing the road, so
  it cannot stop in time, but it can avoid killing 10 people by hitting
  the wall (killing the passengers),

In this case, the car should slam on the brakes. If the 10 people die, that's just unfortunate. We simply cannot trust all of our beliefs about what is taking place outside the car. What if those 10 people are really robots made to look like people? What if they're trying to kill you?

avoiding killing the rider of the motorcycle considering that the
  probability of survival is greater for the passenger of the car,

Again, hard-coding these kinds of sentiments into a vehicle opens the rider of the vehicle up to all kinds of attacks, including "fake" motorcyclists. Humans are barely equipped to make these decisions on their own, if at all. When it doubt, just slam on the brakes.

killing animal on the street in favour of human being,

Again, just hit the brakes. What if it was a baby? What if it was a bomb?

changing lanes to crash into another car to avoid killing a dog,

Nope. The dog was in the wrong place at the wrong time. The other car wasn't. Just slam on the brakes, as safely as possible.

Does the algorithm recognize the difference between a human being and an animal?

Does a human? Not always. What if the human has a gun? What if the animal has large teeth? Is there no context?

Does the size of the human being or animal matter?
Does it count how many passengers it has vs. people in the front?
Does it "know" when babies/children are on board?
Does it take into the account the age (e.g. killing the older first)?

Humans can't agree on these things. If you ask a cop what to do in any of these situations, the answer won't be, "You should have swerved left, weighed all the relevant parties in your head, assessed the relevant ages between all parties, then veered slightly right, and you would have saved 8% more lives." No, the cop will just say, "You should have brought the vehicle to a stop, as quickly and safely as possible." Why? Because cops know people normally aren't equipped to deal with high-speed crash scenarios.
Our target for "self-driving car" should not be 'a moral agent on par with a human.' It should be an agent with the reactive complexity of cockroach, which fails predictably.

In most of those scenarios the people who are potentially killed are more at fault of the damage than the car/occupant of the car -- slamming the brakes seems like the only reasonable response since the occupant couldn't possibly have avoided the accident either. As you point out, it's very possible to use an algorithm that behaves by (potentially) killing the occupant instead of a pedestrian as a way to actually murder the occupant, potentially even putting the blame on the occupant, the car or the manufacturer.

@Clearer I can easily see scenarios where the self-driving sees a completely safe path around an obstacle. But it will still be a case of: safe path => take it. No safe path => slam on brakes. Note this is the same algorithm used in normal driving, not only when something weird happens!

The answer to a lot of those questions depends on how the device is programmed. A computer capable of driving around and recognizing where the road goes is likely to have the ability to visually distinguish a human from an animal, whether that be based on outline, image, or size. With sufficiently sharp image recognition, it might be able to count the number and kind of people in another vehicle. It could even use existing data on the likelihood of injury to people in different kinds of vehicles.
Ultimately, people disagree on the ethical choices involved. Perhaps there could be "ethics settings" for the user/owner to configure, like "consider life count only" vs. "younger lives are more valuable." I personally would think it's not terribly controversial that a machine should damage itself before harming a human, but people disagree on how important pet lives are. If explicit kill-this-first settings make people uneasy, the answers could be determined from a questionnaire given to the user.

Comments are not for extended discussion; this conversation has been moved to chat.

Personally, I think this might be an overhyped issue. Trolley problems only occur when the situation is optimized to prevent "3rd options".
A car has brakes, does it not? "But what if the brakes don't work?" Well, then the car is not allowed to drive at all. Even in regular traffic, human operators are taught that your speed should be limited as such that you can stop within the area you can see. Solutions like these will reduce the possibility of a trolley problem.
As for animals... if there is no explicit effort to deal with humans on the road I think animals will be treated the same. This sounds implausible - roadkill happens often and human "roadkill" is unwanted, but animals are a lot smaller and harder to see than humans, so I think detecting humans will be easier, preventing a lot of the accidents.
In other cases (bugs, faults while driving, multiple failures stacked onto each other), perhaps accidents will occur, they'll be analysed, and vehicles will be updated to avoid causing similar situations.

It is wrong to say that AI will only drive so fast as they can stop from what they see, because the same is true for even defensive driving humans. The reason is that assumptions MUST be made. When cresting a hill, it is assumed that a large obstacle is not right out of view [Source DARPA Grand Challenge]. These assumptions are based on rules of the road to determine which actions are right. Thus, if an accident occurs, as long you followed the rules of the road, you can reduce your liability. So, a car must be able to make these decision, but perhaps in a limited fashion (protect consumer).

@Harrichael Then it will only drive so fast as to stop from what it can see and assume. Also note that a LIDAR mounted on top of the car can see over a hill crest sooner than a human.

In the real world, decisions will be made based on the law, and as noted over on Law.SE, the law generally favors inaction over action.

Ah, that makes a lot of sense for human victims. Would the law's choice be any different if the car could swerve to hit a dog instead of a pedestrian?

This is the well known Trolley Problem. As Ben N said, people disagree on the right course of action for trolley problem scenarios, but it should be noted that with self-driving cars, reliability is so high that these scenarios are really unlikely. So, not much effort will be put into the problems you are describing, at least in the short term.

I wonder how good a fit the Trolley Problem actually is for this? The trolley problem is essentially discrete, describing a `once and for all' choice, whereas in practice the algorithm must make a sequence of choices at (potentially quite small) time increments, possibly with new information becoming available. A useful algorithm is likely to be a 'continuous control' issue embedded in actual 3D space, dealing with velocities and probabilities, not an abstract, discrete moral decision. If nothing else, this would mean the implementation would not have to be defined in such stark moral terms.

@NietzscheanAI The idea of the trolley problem isn't in what decision to make, but rather what to make your decisions on. Should you consider burgulars, number of people, number of animals, who is at fault, driver safety, greater good, least change, etc. You can still apply probabilities to trolley problems, ie. route A has 30 percent chance to kill 10 people. Trolley problems typically involve two routes so that you can consider what factors you base ethical decision on more precisely.

“This moral question of whom to save: 99 percent of our engineering work is to prevent these situations from happening at all.”
—Christoph von Hugo, Mercedes-Benz

This quote is from an article titled Self-Driving Mercedes-Benzes Will Prioritize Occupant Safety over Pedestrians published OCTOBER 7, 2016 BY MICHAEL TAYLOR, retrieved 08 Nov 2016.
Here's an excerpt that outlines what the technological, practical solution to the problem.

The world’s oldest carmaker no longer sees the problem, similar to the question from 1967 known as the Trolley Problem, as unanswerable. Rather than tying itself into moral and ethical knots in a crisis, Mercedes-Benz simply intends to program its self-driving cars to save the people inside the car. Every time.
All of Mercedes-Benz’s future Level 4 and Level 5 autonomous cars will prioritize saving the people they carry, according to Christoph von Hugo, the automaker’s manager of driver assistance systems and active safety.

There article also contains the following fascinating paragraph.

A study released at midyear by Science magazine didn’t clear the air, either. The majority of the 1928 people surveyed thought it would be ethically better for autonomous cars to sacrifice their occupants rather than crash into pedestrians. Yet the majority also said they wouldn’t buy autonomous cars if the car prioritized pedestrian safety over their own.

For a driverless car that is designed by a single entity, the best way for it to make decisions about whom to kill is by estimating and minimizing the probable liability.
It doesn't need to absolutely correctly identify all the potential victims in the area to have a defense for its decision, only to identify them as well as a human could be expected to.
It doesn't even need to know the age and physical condition of everyone in the car, as it can ask for that information and if refused, has the defense that the passengers chose not to provide it, and therefore took responsibility for depriving it of the ability to make a better decision.
It only has to have a viable model for minimizing exposure of the entity to lawsuits, which can then be improved over time to make it more profitable.

How could self-driving cars make ethical decisions about who to kill?
By managing legal liability and consumer safety.
A car that offers the consumer safety is going to be a car that is bought by said consumers. Companies do not want to be liable for killing their customers nor do they want to sell a product that gets the user in legal predicaments. Legal liability and consumer safety are the same issue when looked at from the perspective of "cost to consumer".

And here are few dilemmas:

Does the algorithm recognize the difference between a human being and
an animal?

If an animal/human cannot be legally avoided (and the car is in legal right - if it is not then something else is wrong with the AI's decision making), it likely won't. If the car can safely avoid the obstacle, the AI could reasonably be seen to make this decision, ie. swerve to another lane on an open highway. Notice there is an emphasis on liability and driver safety.

Does the size of the human being or animal matter?

Only the risk factor from hitting the obstacle. Hitting a hippo might be less desirable than hitting the ditch. Hitting a dog is likely more desirable than wrecking the customer's automobile.

Does it count how many passengers it has vs. people in the front?

It counts the people as passengers to see if the car-pooling lane can be taken. It counts the people in front as a risk factor in case of a collision.

Does it "know" when babies/children are on board?

No.

Does it take into the account the age (e.g. killing the older first)?

No. This is simply the wrong abstraction to make a decision, how could this be weighted into choosing the right course of action to reduce risk factor? If Option 1 is hit young guy with 20% chance of significant occupant damage and no legal liability and Option 2 is hit an old guy with 21% chance of significant occupant damage and no legal liability, then what philosopher can convince even just 1 person of the just and equitable weights to make a decision?
Thankfully, the best decision a lot of the time is to hit the breaks to reduce speed (especially when you consider that it is often valuable to act predictably so that pedestrians and motorists can react accordingly). In the meantime, better value improvements can be made in terms of predicting when drivers will make bad decisions and when other actions (such as hitting the reverse) are more beneficial than hitting the breaks. At this point, it is not worth it to even begin collecting the information to make the ethical decisions proposed by philosophers. Thus, this issue is over-hyped by sensational journalists and philosophers.

They shouldn't. People should.
People cannot put the responsibilities of ethical decisions into the hands of computers. It is our responsibility as computer scientists/AI experts to program decisions for computers to make. Will human casualties still exist from this? Of course, they will--- people are not perfect and neither are programs.
There is an excellent in-depth debate on this topic here. I particularly like Yann LeCun's argument regarding the parallel ethical dilemma of testing potentially lethal drugs on patients. Similar to self-driving cars, both can be lethal while having good intentions of saving more people in the long run.

Frankly I think this issue (the Trolley Problem) is inherently overcomplicated, since the real world solution is likely to be pretty straightforward.  Like a human driver, an AI driver will be programmed to act at all times in a generically ethical way, always choosing the course of action that does no harm, or the least harm possible.
If an AI driver encounters danger such as imminent damage to property, obviously the AI will brake hard and aim the car away from breakable objects to avoid or minimize impact.  If the danger is hitting a pedestrian or car or building, it will choose to collide with the least precious or expensive object it can, to do the least harm -- placing a higher value on a human than a building or a dog.
Finally, if the choice of your car's AI driver is to run over a child or hit a wall... it will steer the car, and you, into the wall.  That's what any good human would do.  Why would a good AI act any differently?

'the AI will brake hard', 'it will choose to collide with ...', if these are statements, could you post the some references? Or they're opinion based?

The OP posed the question as, "What damage will the AI choose to inflict?".  I think that's the wrong objective.  Humans don't do that.  Instead the AI will follow a more general strategy of avoiding damage, and doing as little damage as possible.  This optimization objective is more intuitive and harder to argue against when liability concerns arise.

I think that in most cases the car would default to reducing speed as a main option, rather than steering toward or away from a specific choice. As others have mentioned, having settings related to ethics is just a bad idea. What happens if two cars that are programmed with opposite ethical settings and are about to collide? The cars could potentially have a system to override the user settings and pick the most mutually beneficial solution. It's indeed an interesting concept, and one that definitely has to discussed and standardized before widespread implementation. Putting ethical decisions in a machines hands makes the resulting liability sometimes hard to picture.

Yes, braking to avoid a collision would absolutely be the best solution. Unfortunately, in some cases (the cases asked about in the question), hitting something is unavoidable. You seem to agree with some existing answers. Please note that we would like each answer to be able to stand on its own. Once you have sufficient reputation, you'll be able to upvote answers you believe are helpful. The problem of two oppositely-configured cars is indeed interesting; it would be great if you would [edit] your answer to expand on that. Thanks!

Thank you very much for your reply, I actually really appreciate it and will use it going forward on the site. I wasn't aware of the guidelines for answers.

The only sensible choice is to use predictable behaviour. So in the people in front of the car scenario: First the car hits the brakes, at the same time honks the horn, and stays on course.  The people then have a chance to jump out of the way leading to zero people being killed. Also with full brakes (going from 50km per hour to zero is less than 3 car lengths), an impact situation is almost not imaginable. Even if full stop cannot be reached, severe damage to the pedestrians is unlikely.
The other scenario is just crazy. So the distance has to be less than 3 car lengths, at least 1 car length in needed for the steering, then a car crashing is an uncontrollable situation, might lead to spinning and kill all 11 persons.
Apart from saying that I don't believe there is an example in reality where there is a dilemma;  the solution in these unlikely cases if to conform with the expectations of the opposing party to allow the other party to mitigate the situation as well.

I think there would not be a way to edit such ethics settings in a car. But hey, if cell phones can be rooted, why not cars? I imagine there'll be Linux builds in the future for specific models that will let you do whatever you want.
As for who'll make such decisions, it'll be much like privacy issues of today. There'll be a tug-of war on the blanket by the OS providers (who'll try to set it to a minimum amount of people injured, each with its own methods), insurance companies (who'll try to make you pay more for OS's that will be statistically shown to damage your car easier), and car manufacturers (who'll want you to trash your car as soon as you can, so you'll buy a new one; or make cars that require a ridiculous amount of $$$ service).
Then some whistleblower will come out and expose a piece of code that chooses to kill young children over adults - because it will have a harder time distinguishing them from animals, and will take chances to save who it'll more surely recognize as humans. The OS manufacturer will get a head-slap from the public and a new consensus will be found. Whistleblowers will come out from insurance companies and car manufacturers too.
Humanity will grab a hot frying pan and burn itself and then learn to put on gloves beforehand. My advice would just make sure you won't be that hand - stay away from them for a couple of years until all the early mistakes are made.

I think we need to state our own moral before thinking about what the cars moral(or ethical setting) should be. I recommend reading this paper Autonomous Cars: In Favor of a Mandatory Ethics Setting which argues why it is in everyone's best interest that we prioritize the safety of the majority, and not just the driver(yes, in the best interest of the driver too).
You can test your own moral in many different situations, some like your examples, on MITs moral machine. It's rather uncomfortable but very interesting. You can also find some analysis of people's answers on their website.
My answers to your examples:

The car is heading toward a crowd of 10 people crossing the road, so it cannot stop in time, but it can avoid killing 10 people by hitting the wall (killing the passengers)

I assume hitting the brakes is not going to work, or else the dilemma is pointless. I think the car should hit the wall. Pedestrians should not suffer just because someone else is driving a car, especially when there are 10 pedestrians and maximal 5(typically 1 or 2) in the car.

Avoiding killing the rider of the motorcycle considering that the probability of survival is greater for the passenger of the car

I think this one is harder, especially since the motorcycle probably is not autonomous, and (in contrast to the pedestrians in the previous example) riding motorcycles is quite dangerous. Are the motorcyclist accepting the risk when entering the roads? If avoiding the motorcyclist means a probable death for the driver, then no. If not, probably should avoid it.

Killing an animal on the street in favor of a human being
Purposely changing lanes to crash into another car to avoid killing a dog

I think humans are more important than animals.
I don't think there exists a correct answer for this.
One of the really interesting things is that from the data collected by the moral machine is that there are big differences based on where in the world you're from. Typically, the western typically countries prioritize saving children over the elderly, while this is not the case for the whole world. Countries with strong governments like Finland and Japan prioritize people abiding the law, while people from countries with weaker/corrupt government does not care so much about that. Even in this comment section, you can find differences! I, for example, think the pedestrians should be spared in the first example,
while Doxosophoi thinks that it is obvious that the passengers should be protected!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Clonal operator in Immune Clonal Strategy
I was reading about Immune Clonal Strategy, specifically about Monoclonal operator from Immunity clonal strategies, and it goes as follows:

Here $a_i $ is a point and $a_i = \{ x_1, x_2, \cdots, x_m \}$.
I do not understand what $I_i$ really is, It seems like just copy $a_i$ for $q_i$ times or something like that, can someone please explain to me what is really happened here?


text:
learning rate and credit assignment problem in checkers
I have implemented an AI agent to play checkers based on the design written in the first chapter of
Machine Learning, Tom Mitchell, McGraw Hill, 1997.
We train the agent by letting it plays against its self.
I wrote the prediction to get how good a board is for white, so when the white plays he must choose the next board with the maximum value, and when black plays it must choose the next board with the minimum value.
Also, I let the agent explore other states by making it choose a random board among the valid next boards, I let that probability to be equal to $0.1$.
The final boards will have training values:

100 if this final board is win for white.

-100 if this final board is lose for white.

0 if this final board is draw.

The intermediate boards will have a training value equal to the prediction of the next board where it is white turn.
The model is based on a linear combination of some features (see the book for full description).
I start by initializing the parameters of the model to random values.
When I train the agent, he lost against himself always or draw in a stupid way, but the error converges to zero.
I was thinking that maybe we should let the learning rate smaller (like 1e-5), and when I do that the agent learns in a better way.
I think that this happened because of the credit assignment problem, because a good move may appear in a loose game, therefore, considered a loose move, so white will never choose it when he plays, but when we let the learning rate to be very small that existence of a good move in a losing game will change its value by a very little amount, and that good move should appear more in win games so its value converges to the right value.
Is my reasoning correct? and if not so what is happened?

Please, put your main specific question in the title. "AI agent playing checkers" is not a question and it's very general. If you have more than one specific question, split this post into multiple ones (one for each of the questions that you have).

@nbro sorry for that, I edited the question, I hope it is more clear now.

There are a couple of things that might be off here. Number one is, as you mentioned the learning rate. Number two is the exploration policy (i.e. taking a random action only 10% of times), you might want to try and have a higher chance of random move for a while and then progressively lower it. Number 3 is that the agent is playing against itself, but when he's black his aim is to lose. Is the agent fed the information of who he is (black or white) as part of the observation? This would be essential

@FedericoMalerba I tried the second point you mentioned but that does not solve the problem.
I do not understand the third point, the prediction based on the evaluation of the board for white, so black need to minimize the board goodness for white, so he choose the minimum.
For the training, I only take the boards when it's white's turn so no need for the colour.
I take this from the book I mentioned at the top of the question.


text:
Is training on single game each time appropriate for an agent to learn to play checkers
I was facing a problem I mentioned in a previous question but after a while, I realize that maybe the problem in the dataset not in the learning rate.
I build the dataset from white positions only i.e the boards when it's white's turn.
Each data set consists of one game.
First, I tried to let the agent plays a game then learn from it immediately, but that did not work,
and the agent converges to one state of playing (he only lose or win against itself, or draw in a stupid way).
Second, I tried to let the agent plays about 1000 games against its self then train on each game separately, but that also does not work.
Note: the first and second approach describes one iteration of the learning process, I let the agent repeat them so in total it trained on about 50000 games.
Is my approach wrong? or my dataset must be built in another way? maybe train the agent on several games at once?
My project is here if someone needs to take a closer look at it: CheckerBot


text:
Can models get 100% accuracy on solved games?
I had a question today that I feel it must have an answer already, so I'm shopping around.
If we ask a model to learn the binary OR function, we get perfect accuracy with every model (as far as I know).
If we ask a model to learn the XOR function we get perfect accuracy with some models and an approximation with others (e.g. perceptrons).
This is due to the way perceptrons are designed -- it's a surface the algorithm can't learn. But again, with a multi-layered neural network, we can get 100% accuracy.
So can we perfectly learn a solved game as well?
Tic-tac-toe is a solved game; an optimal move exists for both players in every state of the game. So in theory our model could learn tic-tac-toe as well as it could a logic function, right?

It depends on the VC dimension of the model. If the solution space of the game can be shattered by the learning model, theoretically it is possible. You're right.

So can we perfectly learn a solved game as well?

The short answer is yes. If your model has enough complexity it can theoretically learn any behavior you want.

So in theory our model could learn tic-tac-toe

Tic Tac Toe has already been solved. Another popular game that has been solved is Checkers, by the algorithm Chinook.
To be more specific, in Reinforcement Learning we make the assumption that any decision making process can be modeled as an MDP (Markov Decision Process). Once there, there are a host of different methods like Q-Learning and TD that theoretically converge towards the optimal policy - the one the plays perfectly.
Now, just because it is theoretically possible doesn't mean it will always empirically work. In games that are very complex and have a large state space it is extremely difficult to perfectly solve. This is because the only feasible way to go about them is to approximate and getting perfect play even in small edge cases becomes much more difficult as a result.
If you want to learn more about this topic I would highly recommend this series RL Course by David Silver

Indeed, a solved game can be expressed simply as a table that enumerates every possible game state and associates it with the optimal move for that state. A sufficiently complex model can simply memorize the table, as it never needs to generalize to unseen states (which would, by definition, not be valid game states).

"If your model has enough complexity it can theoretically learn anything.", this is not true. Universal approximation theorems say that you can approximate arbitrarily well, they don't say that you can compute exactly. Moreover, they hold for continuous functions. In general, machine learning techniques are naturally approximative methods, so usually find approximative solutions to problems.

@nbro I see your point. Anything is a bit vague. What I was trying to get at was that in RL you can learn any policy you want. In terms of not being able to compute the exact values or being discrete, its not a problem, because approximating values is good enough for your policy to learn optimal behavior. I will edit my answer to make that more clear

authors: هدايه التوبيه


text:
What kind of neural network can be trained to recognise patterns?
Is there a type of neural network that can be fed patterns to train itself on to complete new patterns that it has not seen before?
What I'm trying to do is train a neural network to transform an image into another image. The image may be slightly different each time (denoted with different lines in the shapes) but a human would get the idea of how the new images should look. I'd like to make a network that can learn how to learn what comes next and then predict the rest of the sequence from the first part of a new sequence.
Taking the picture below as an example. The neural network would be fed the patterns in grey and learn how to predict the next ones in the sequence. Then the user would put the blue shapes into the network and hope to get the green ones out.
Is there a neural network that could perform this type of function of completing a pattern based on only a small number of examples to start the pattern based on the other patterns it has seen?

EDIT: Corrected image and added more context

If the sequence of symbols or geometrical objects really has a pattern (e.g. after circles we always have squares, or something like that), then the neural network may be able to capture it, but if the sequence of objects is random, how can you predict anything without an error less than 50%? Do you know if the sequence really has patterns that can be recognized?

Moreover, neural networks typically require a lot of data to learn well, although there are already some approaches that attempt to overcome this issue. Have a look at terms like "one-shot learning", "few-shot learning" or "zero-shot learning".

Thanks for your response. The shape would always be similar (if there is a round shape there will always be a rectangular shape). I've updated the image as I realised I hadn't drawn it completely correctly.

What I'm trying to do is train a neural network to transform an image to another image. The image may be slightly different each time (denoted with different lines in the shapes) but a human would get the idea of how the new images should look. I'd like to make a network that can learn how to learn what comes next and then predict the rest of the sequence from the first part of a new sequence.

The idea is simple, but it requires some time to develop.
Assumption: I am assuming in your problem the final model will have seen all possible shapes.
What your algorithm needs is a convolutional NN to understand each shape by extracting features, but you just need to be very careful with pooling.
Then what you need is a recurrent NN. In the example you showed (the image of shapes) we have bigrams (sequence of 2) which means we have the first shape as an input and the second shape is a target. In this case normal RNN should work.
But if you have sequence of many shapes, for example 10 shapes and let's say the final shape, 10th, is the target; And also if the sequence is in the way that 10th shape could be more depend on initial shapes (e.g. 1st or 2nd shape) then what you also need to consider beside RNN is long-short-term-memory (LSTM).
I cannot think of a simpler solution than this.


text:
Why does our AI play worse at even levels of depth?
We are building an AI to play a board game. Leaving aside the implementation, we noticed that it plays worse when we set an even (2,4,6,...) level of depth. We use a minimax depth-first strategy.
Do you have any ideas why it behaves like that?
Edit: for example if we set a game between an AI with 5 levels of depth and an AI with 6 levels of depth, the first one usually wins (and this is weird).

Have you tried checking for quiescense and going one layer deeper in that case?

When the number of levels is odd, it means the first player can do one more extra movement on the board. As it is an extensive form game, when decide using backward induction, as the last and the first move are for the first player, so the first player can act better than the situation that the second player will make the last move.

But if the AI evaluates up to level x + 1 (suppose x is odd), it means that it has evaluated all moves up to x and it is now evaluating the second player's next move. How can this degrade performance? At the most it should play about the same

@TomDwan Suppose you're playing chess. If you search to depth 1, you only consider your own moves, and don't consider possible responses by the opponent. This will probably often lead to very aggressive play (you can capture pieces and don't search deep enough to see your opponent replying). Increasing to depth 2 would often switch this around completely and lead to much more defensive play, since you no longer consider your own replies to your opponent's reply to your first move. Etc. In some games even may tend to be better, and in some games odd may tend to be better.

@DennisSoemers Ok, thanks for the clarification :)


text:
Are there any board game appropriate to examine the performance of multiple agents that cooperate both inter-group and intra-group?
I want to find out scenarios that useful to examine the performance of intra-group and inter-group cooperation in MARL.
Specifically, I prefer a board game (like sudoku) that is suitable for the cooperation evaluation.
But there are some differences between the requirements and the go like game. Every grid on the board should be treated as an agent. They are designed to form a situation with the local utility and global utility.
Take the sudoku as an example, every grid should choose an appropriate value to reach the sudoku solution.
Due to my non-familiarity with traditional MARL scenarios, it will be a great help for showing me some keywords or lists.

Hi Jarvis and welcome to AI SE! I think it would be better if you defined more precisely what you mean by "matrix game". Are you looking for board games where you can test MARL systems/algorithms?

thanks for your suggestions. I will modify the problem description. Yes! I am looking for some kinds of board games. But there is still some difference between that and the go like games. The detailed description I would add to the problem descriptions.


text:
Owner Search for given Server SNO
I am newbie to NLP.
I have a excel sheet with following columns:
    Server_SNo,
    Owner,
    Hosting Dept,
    Bus owner,
    Applications hosted,
    Functionality,
    comments
a. Except the Server_SNo, other columns may or may not have data.
b. For some records there is no data except Server_SNo which is the first column.
c. One business owner can own more than 1 Server.
So, out of 4000 records, about 50% of data contain direct mapping for a server with owner.  Remaining 50% of data have combination of other columns (Owner, Hosting Dept, Bus owner, Applications hosted, Functionality and comments)
Here is my problem, I need to find the owner for the given Server_Sno for 50% of data which have combination of other columns (Owner, Hosting Dept, Bus owner, Applications hosted, Functionality and comments).
I have just started to build the code using Python and NLTK.
Is this an NLP problem? Am I going in right direction using Python and NLTK for NLP?
Any insights is appreciated.
-Mani

I don't think this classify as an NLP problem, there is almost no semantic analysis needed, it is more like a classification problem using categorical features.
NLTK is surely valuable if you want to perform some text 'cleaning' or preprocessing before encoding the variables. The only NLP application that I think you could apply here is some sentiment analysis on the comments to extract extra features (like a number expressing the negativeness or positiveness of each comment). Nevertheless you might want to do that using some pre-trained models cause your dataset is pretty small.

Hello Eodardo,  Thanks for your reply, I was able to do this in python using dataframe itself.  I am currently adding some more enhancements to it.  I will post the completed code sometime soon.


text:
Why do most deep learning papers not include an implementation?
I'm a novice researcher, and as I started to read papers in the area of deep learning I noticed that the implementation is normally not added and is needed to be searched elsewhere, and my question is how come that's the case? The paper's authors needed to implement their models anyway in order to conduct their experimentations, so why not publish the implementation? Plus, if the implementation is not added and there's no reproducibility, what prevents authors from forging results?

There's somewhat of a movement to change that. Check out paperswithcode.com for a nice example.

A similar question was asked in Academia SE recently. There's an interesting analogy with e.g. CERN - when scientists working there publish results, they do not provide an executable version of the Large Hadron Collider and the original particles. Provided the descriptions of methods are robust, and the conclusions of the research broad enough (not just a specific architecture that scores well on a specific challenge), then reproducability does not have to be perfect to the exact bit-string, and really that's not of great interest anyway, results that apply more generally are the normal goal.

The paper's authors needed to implement their models anyway in order to conduct their experimentations, so why not publish the implementation?

Some papers and authors actually provide a link to their own implementation, but most of the papers (that I have read) don't provide it, although some third-party implementations may already be available on Github (or other code-hosting sites) when you are reading the paper. There may be different reasons why the author(s) of a paper don't provide a reference implementation

They use some closed-source software or maybe it makes use of other resources that cannot be shared
Their implementation is a mess and, so, from a pedagogical point of view, it's quite useless
This may encourage other people to try to reproduce their results with different implementations, so it may indirectly encourage people to do research on the same topic (but maybe not providing a reference implementation could actually have the opposite effect!)

Plus, if the implementation is not added and there's no reproducibility, what prevents authors from forging results?

I had some experience as a researcher, but not enough to answer this question precisely.
Nevertheless, from some reviews of papers I have read (e.g. on OpenReview), in most cases, the reviewers are interested in the consistency of the results, the novelty of the work, the clarity and structure of the paper, etc. I think that, in most cases, they probably trust the provided results, also because, often, for reproducibility, researchers are expected to describe their models and parameters in detail, provide plots, etc., but I don't exclude that there are cases of people that try to fool the reviewers. For example, watch this video where Phil Tabor comments on ridiculous attempts to fool people and plagiarism by Siraj Raval.

The first reason described in nbro's answer can definitely be an important one; authors may have implemented their software using code that they can't share. There's a lot of research coming out of companies (large and small), and they may use all sorts of proprietary libraries that were built in the company and cannot be distributed outside.
As described in this answer, sometimes researchers prefer to keep the code to themselves because it may give them an "advantage" over other researchers fot future work / follow-up research in the same area. I'm not saying that I believe this is a good reason, it definitely doesn't sound like it's good for the overall benefit of science... but it may be understandable in a "publish or perish" world where there's quite a bit of pressure to keep publishing frequently if you want your academic career to survive.
Also described in more detail in the answer I linked above, research code is often messy, and not pretty. Nbro also mentioned this, though I personally don't feel like the rationale is "it's too messy to be useful", and more often it's more along the lines of "it's so messy that I'm too embarassed to share it".
Some researchers, especially in larger teams, do not just work on a single paper at a time. They may have multiple papers they're working on simultaneously, and if they're closely related it can often be convenient to have them all in a single codebase. This is especially the case with longer review times; in the time between submission of a paper -- where it and anything related to it, such as source code, must remain private -- and an acceptance notification, there's plenty of time to start working on a next project. If the code for the previous project is mixed in with the code for the next project, and you can't / don't want to publish the code for the next project either yet... it may be easier to just not release anything.
In some cases, authors may feel it is "dangerous" to release their source code (or trained models). This is probably relatively uncommon, but can happen. Consider the situation surrounding OpenAI's GPT-2 language model, for example.

Not directly a response to your question, but it may also be useful to keep in mind that sometimes not all authors of a paper may agree on whether or not to open-source it. Legally, I suppose that usually all the authors (or all contributors to the source code) would be copyright holders, and it can only be released if they all agree to release it. So if one of them feels (based on any of the reasons listed above, or maybe other reasons) that it shouldn't be released, it won't. In practice, I suppose that it would often primarily be the call of the more senior authors on a paper / principal investigators / supervisors.

Plus, if the implementation is not added and there's no reproducibility, what prevents authors from forging results?

Personally I wouldn't be concerned as much about forging results as just... "accidental" false positives. Yes, it's possible and it will happen. But the pay-off of successfully forging false results and getting a paper published seem REALLY low compared to the risk of your academic career ending if it gets out. If you really have to forge your results just to get your paper accepted, and it has zero other meaningful contributions (no "unforgeable" contributions like theoretical results or really new and useful insights).. it's unlikely to become a really impactful paper, a widely-cited one. The really highly impactful empirical papers only become highly impactful because people will immediately try to re-implement and reproduce it anyway, and if that turns out to be impossible, it will turn into a dead end.
That said, I'm not saying it can't be important to share source code. Especially in deep learning, and especially in deep reinforcement learning, it has indeed been shown that tiny implementation details can be massively important to empirical performance, and these tiny implementation details are rarely all available in papers. There has certainly been a push towards encouraging the publication of source code, and it is important -- but unfortunately it's not always a black-and-white story, and there can sometimes also be good reasons that make it difficult/impossible to do so. If it's good research, I'd personally still rather have it without source code, than not have it at all.

Agree with most the points. I am unsure about point 4. I don't know about all teams but the teams I know usually back-up all version of their code. For example when they run an experiment and produce a plot, that version of code will be backed-up. So even if the code is changed in future, they will still have the old version of the code.

Someone can argue to some human adequate reasons, but there is a bad trend of falsified results in deep learning research papers that propose some nowel solutions or even update state-of-the-art model performance. And that's not just a few papers that lie, it's a large portion of them. And the reason for that is even more sad - most of so-called deep learning research papers just describing some empirical experiments, without any math and proving any theorem, and so it's easy to cheat.
So objectively, if the only thing you propose in your paper is your empirical results - you must confirm them true by sharing source code. Otherwise, your work will be ignored.


text:
Neural Network Results always the same
I have a GRU model which has 12 features as inputs and I'm trying to predict output power. I really do not understand though whether I choose

1 layer or 5 layers
50 neurons or 512 neuron
10 epochs with a small batch size or 100 eopochs with a large batch size
Different optimizers and activation functions
Dropput and L2 regurlarization
Adding more dense layer.
Increasing and Decreasing learning rate

My results are always the same and doesn't make any sense, my loss and val_loss loss is very steep in first 2 epochs and then for the rest it becomes constant with small fluctuations in val_loss.
Here is my code and a figure of losses, and my dataframes if needed:
Dataframe1: https://drive.google.com/file/d/1I6QAU47S5360IyIdH2hpczQeRo9Q1Gcg/view
Dataframe2: https://drive.google.com/file/d/1EzG4TVck_vlh0zO7XovxmqFhp2uDGmSM/view
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from google.colab import files
from tensorboardcolab import TensorBoardColab, TensorBoardColabCallback
tbc=TensorBoardColab() # Tensorboard
from keras.layers.core import Dense
from keras.layers.recurrent import GRU
from keras.models import Sequential
from keras.callbacks import EarlyStopping
from keras import regularizers
from keras.layers import Dropout

df10=pd.read_csv('/content/drive/My Drive/Isolation Forest/IF 10 PERCENT.csv',index_col=None)
df2_10= pd.read_csv('/content/drive/My Drive/2019 Dataframe/2019 10minutes IF 10 PERCENT.csv',index_col=None)

X10_train= df10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_train=X10_train.values

y10_train= df10['Power_kW']
y10_train=y10_train.values

X10_test= df2_10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_test=X10_test.values

y10_test= df2_10['Power_kW']
y10_test=y10_test.values

# scaling values for model

x_scale = MinMaxScaler()
y_scale = MinMaxScaler()

X10_train= x_scale.fit_transform(X10_train)
y10_train= y_scale.fit_transform(y10_train.reshape(-1,1))
X10_test=  x_scale.fit_transform(X10_test)
y10_test=  y_scale.fit_transform(y10_test.reshape(-1,1))

X10_train = X10_train.reshape((-1,1,12))
X10_test = X10_test.reshape((-1,1,12))

Early_Stop=EarlyStopping(monitor='val_loss', patience=3 , mode='min',restore_best_weights=True)

# creating model using Keras
model10 = Sequential()
model10.add(GRU(units=200, return_sequences=True, input_shape=(1,12),activity_regularizer=regularizers.l2(0.0001)))
model10.add(GRU(units=100, return_sequences=True))
model10.add(GRU(units=50))
#model10.add(GRU(units=30))
model10.add(Dense(units=1, activation='linear'))
model10.compile(loss=['mse'], optimizer='adam',metrics=['mse'])
model10.summary()

history10=model10.fit(X10_train, y10_train, batch_size=1500,epochs=100,validation_split=0.1, verbose=1, callbacks=[TensorBoardColabCallback(tbc),Early_Stop])

score = model10.evaluate(X10_test, y10_test)
print('Score: {}'.format(score))

y10_predicted = model10.predict(X10_test)
y10_predicted = y_scale.inverse_transform(y10_predicted)

y10_test = y_scale.inverse_transform(y10_test)

plt.scatter( df2_10['WindSpeed_mps'], y10_test, label='Measurements',s=1)
plt.scatter( df2_10['WindSpeed_mps'], y10_predicted, label='Predicted',s=1)
plt.legend()
plt.savefig('/content/drive/My Drive/Figures/we move on curve6 IF10.png')
plt.show()


text:
Is my GRU model under-fitting given this plot of the training and validation loss?
I was running my gated recurrent unit (GRU) model. I wanted to get an opinion if my loss and validation loss graph is good or not, since I'm new to this and don't really know if that is considered underfitting or not

When ever you are buliding a ML Model don't take accuracy seriously(Mistake done by Netflix that cost them alot), you should try to get the hit scores as they will help you to know how many times your model worked on real world users.However, if your model must have to measure the accuracy try it with the RMSE score as it will penalise you more for being more out of the Line. Here is the link for more information on it RMSE
Its hard to predict if its overfitting or underfitting as your graph is vague(for example what does graph lines representing). However, you can solve underfitting by following steps:
1) Increase the size or number of parameters in the ML model.
2)  Increase the complexity or type of the model.
3)  Increasing the training time until cost function in ML is minimised.
For overfitting you can try Regularization methods like weight decay provide an easy way to control overfitting for large neural network models. A modern recommendation for regularization is to use early stopping with dropout and a weight constraint.

Thank you very much, this is some great info! I my model suffered from overfitting, I did a little bit of tweaking then it fitted well. I added L2 refularization, dropout, decreased number of layers and also reduced learning rate a little.

You should at least crop the plots and add a legend. Maybe also provide some scores (accuracy, auc, whatever you're using).
 Anyway, it doesn't look your model is underfitting, if it was you should have high error at both, training and test phase and the lines would not cross.

My final final MSE loss was 0.001371944 and validation MSE loss was 0.001448105. The Test scores were [0.030501108373429363, 0.00272163194425038]. Does crossing lines indicate that it is a good fit?

It's hard to say from a single plot. The best thing to do is always to perform several training of the same model, save history loss and MSE and then average them. In this way you get rid of some causal outcomes (it's not unlikely for example to have lower validation loss than training one with Keras, due to some batch averaging they defined). Anyway, your MSE is pretty low, I would be more concerned of overfitting but also the test one is low, which is good. Are you using some already made dataset from TensorFlow?


text:
Are the labels updated during training in the algorithm presented in "An algorithm for correcting mislabeled data"?
I am trying to understand an algorithm for correcting mislabeled data in the paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al. The authors are suggesting to update the output class probability vector using the formula in equation 4 and class label in equation 5.
I am wondering:

Are they updating labels while training, starting from very first back-propagation?
It seems like if we train on the same data and then predict labels on the same data, it would be the same as what the authors are suggesting. Does it make sense or I misunderstood?

I think that making some draws might help.
Below I tried to draw the model architecture. We start with classic feed-forward structure: input represented by a vector I with length f (number of features), a hidden layer H which does not have a fixed size, and output O of length c (number of classes). Then we have 3 extra vectors than usual:
a vector U they refer as input (a bit confusing I have to say), and two vectors V and P representing classes probabilities. All these vectors have length c.
P is what we want to learn, new class probabilities for each instance that should correct wrong labels of misclassified instances in the initial dataset.
So in some sense the aim of the work is not to train a model to make predictions but rather train a model to clean a training dataset. I think it's important to stress out this point because we talk about training but there is actually no test after the training, we just end with a modified training dataset with some instances relabelled.
The relabelling depends on the learning of V rather than P (because as I will say also later on, the final arrow that goes from V to P is the identity function). V depends on U that also depends on V, again it sounds a bit confusing, the tricks rely in the initialisation.
In the second picture I just copied some formulas from the paper. They don't specifically say how they get the vector P in the first place, but I treat it simply as given because we need it to initialise U by applying the inverse sigmoid function, element-wise. We need also to define the hyper parameter D, the initial probability value for the true class V$_y$ in the initial dataset. In the paper they set it to .95. Using D we can initialise the vector V.
Once we initialise U and V we have all the elements to iterate over the dataset.
A first thing to notice is that they don't use back propagation to update U,V and P, they just define some updating rules (which require to define some other initial parameters, $\textit{u$_0$}$ and L$_p$). I want to stress out again that the mapping between V and P is just the identity function, I guess that they define it just to avoid confusion, because in the paper P appear as an input element, but nevertheless is also the output we want to learn. Back propagation is used only to update the weights in H.
ANSWERS TO THE QUESTIONS:
So we can finally say that, regarding your first question, the answer is yes, they do start updating the classes probability vectors P from the very first iteration, even though is not clear how they initialise them (or from where they got them).
Regarding the second question instead I would say that no, this is definitely not as training a model with fixed label and then making predictions over the training dataset again. The whole point is that similar training instances will be initialised with the same P, and they will also produce similar U and V vectors. For misclassified instances, U will be updated with larger changes, because of the different produced O. For example an image of a 2 will generate a different output than an image of a 5, this would be reflected first on U and then on V and P . If training with fixed label instead you would just force the weight in the hidden layer to learn a function that treat the representation of a 2 and a 5 in a similar way, leading to low accuracy cause you would be telling the model "Hey, the straight line in the 5 is important also to recognise a 2".
I have to say that I never read about this dataset cleaning approach but it is interesting, and their results show that the cleaned version of the dataset lead to better performances, which is interesting because machine learning analysis usually gives for granted the correctness of the labels.
Hope this is of help in some ways!

Many many thanks. This is really helpful. I think vector P is coming from D i.e. initial probability values with higher values to actual class: "In our experiment we chose D = 0.95. If C = 3 and y = 1, for example, then p1(O) = 0.95, and p2(O) = p3(O) = 0.05/2 = 0.025" Is it correct? 2. May be I missed it, where do they mention initialising U by inverse sigmoid. In equation 3, its V by sigmoid? 3. Do they only update class probabilities during training or also label (equation 5)? I thought labels should be updated in the end other wise whole input labels will be messed up after 1st Epoch.

D is used to initialise V, which in principle is the same as P after the first iteration, but you might be right and they could have use the same initialisation for P as well. U initialisation is explained at page 495 "The input Ui is then determined from the corresponding output using the inverse sigmoid function". The arctan used to calculate V is another mystery, they say sigmoid and then they write tanh in the formula.  The label in question 5 are calculated to perform the back propagation update on the hidden layer weight, so in my opinion they update it every step, not every epoch.

Waao, this is also what I missed "The input Ui is then determined from the corresponding output using the inverse sigmoid function".

Sorry, I meant every iteration not epoch. By step you mean one iteration?

Yes I mean one iteration, sorry I'm not always consistent in my terminology :)

Thanks. So it means that we first update labels on forward-propagation and then calculate the error with updated labels and then back-propagation?

You may be right again, I followed the updates order as describe in the paper. But thinking about it I would say that the order is probably not so determinant. I could update the label straight away and then back propagate using the new label, or I could back propagate with the actual label, then update the label and at that point the weights will be refined at the next epoch with the updated label (slowing down convergence probably). This is a classic situation in which the paper code would be really useful.


text:
Recent algorithms for correcting mislabeled data using multilayer perceptrons
I am doing literature research on algorithms for correcting mislabeled data using multilayer perceptrons. Found an "old" paper An algorithm for correcting mislabeled data (2001) by Xinchuan Zeng et al. Please share if you are aware of recent/current updates with a brief thoughts. Thanks in advance.

Are you just looking for techniques that specifically use MLPs or any other technique for correcting mislabelled data?

I have numerical data with "unrelated" columns of numbers. Therefore, I think MLPs make more sense rather than, for example, CNNs.

I am also not sure if https://arxiv.org/abs/1801.08019 and https://ieeexplore.ieee.org/abstract/document/6413805 will be more powerful as compared to MLPs.

The most general solution today for the problem of finding label errors in datasets is called "confident learning" which works for all datasets and models, can be run time-efficiently in one line of code using cleanlab, and has substantial theory to prove that it works in realistic conditions on real-world datasets. This "confident learning" paper was a culminating result during my PhD at MIT and I am an author on the paper.

The Confident Learning Paper: https://arxiv.org/abs/1911.00068
Code: https://github.com/cleanlab/cleanlab
Examples of label errors found with cleanlab: https://labelerrors.com

Find label issues in your dataset in 1 line of code
from cleanlab.classification import CleanLearning
from cleanlab.filter import find_label_issues

# Option 1 - works with sklearn-compatible models - just input the data and labels ツ
label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)

# Option 2 - works with ANY ML model - just input the model's predicted probabilities
ordered_label_issues = find_label_issues(
    labels=labels,
    pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
    return_indices_ranked_by='self_confidence',
)

Train a model as if the dataset did not have errors -- 3 lines of code
from sklearn.linear_model import LogisticRegression
from cleanlab.classification import CleanLearning

cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
cl.fit(train_data, labels)

# Estimate the predictions you would have gotten if you trained without mislabeled data.
predictions = cl.predict(test_data)

Documentation and runnable tutorials for cleanlab: https://docs.cleanlab.ai/
The above approaches find issues. To correct label issues in your dataset:
The above approaches, just find issues and train without them. To correct the labels or train a model on a corrected dataset (that still includes all the errors, but now corrected with the right label), there is a no-code tool for that called Cleanlab Studio (https://cleanlab.ai/studio) for which I am also an author.
Background if you're interested:
I spent half a decade working with Isaac Chuang (inventor of the quantum computer) to solve this problem in a way that works for every dataset and every model (and every future dataset and future model) during my PhD at MIT. I originally decided to solve this problem after discovering (while building MIT and Harvard's cheating detection system in 2013) that most real-world datasets have significant label errors and this is one of the biggest problems that companies and universities struggle with when training ML models.


text:
Which one is better: multivariate regression with basis expansion or neural networks?
Assume we are given a training dataset $D = \{ (x_i, y_i)\}_{i=1}^{N}$.
My question is: which is better?

A multivariate regression with basis expansion with independent matrix $X$ and dependent matrix $Y$, such that $X \in K; K \subset \mathbb R^n$ and $Y \in \mathbb R^m$ with training data $D$.

Or

A neural network which takes $n$ input variables and returns $m$ output with training data $D$

Without a doubt, the multivariate regression option is better with its basis polynomials because it can adapt any curve required in any dimension and doesn't need a large number of datasets than neural networks. Then, why neural networks are used more than multivariate regression?

Note: Prefer explaining the mechanism of neural network used as regression in your answers. To help us know the degree of flexibility of both.
Edit: You may prefer choosing your own loss function in case you need.

This question is impossible to answer. There is no objective 'better' solution unless you know more about the shape of the data. It's like asking "what's better, a fork or a spoon?" -- depends on whether you want to eat soup or steak.

@OliverMason You may explain in which case which is better.


text:
Policy Gradient Reward Oscillation in MATLAB
I'm trying to train a Policy Gradient Agent with Baseline for my RL research. I'm using the in-built RL toolbox from MATLAB (https://www.mathworks.com/help/reinforcement-learning/ug/pg-agents.html) and have created my own Environment. The goal is to train the system to sample an underlying time-series ($x$) given battery constrains ($\epsilon$ is battery cost).
The general setup is as follows:

My Environment is a "sensor" system with exogenous input time-series and battery level as my States/Observations (size is 13x1).
Actions $A_t$ are binary: 0 = keep a model prediction $(\hat x)$; 1 = sampling time series $(x)$
Reward function is

$$
R = -[err(\tilde x, x) + A_t\cdot \epsilon ] + (-100)\cdot T_1 + (100) \cdot T_2
$$
where $err(\tilde x, x)$ is the RMSE error between the sampled time series $(\tilde x)$, and true time series x.

The Terminal State Rewards are -100 if sensor runs out of battery $T_1$ or 100 if reached the end of the episode with RMSE < threshold and remaining battery level $(T_2)$. The goal is to always end in $T_2$.
Each training Episode consists of a time-series of random length, and random initial battery level.

My current setup is using mostly default RL setups from MALTAB with learning rate of $10^{-4}$ and ADAM optimizer. The training is slow, and shows a lot of Reward oscillation between the two terminal states. MATLAB RL toolbox also outputs a $Q_0$ value which the state is:

Episode Q0 is the estimate of the discounted long-term reward at the
  start of each episode, given the initial observation of the
  environment.  As training progresses, Episode Q0 should approach the
  true discounted  long-term reward if the critic is well-designed,

Questions

Is my training and episodes too random? i.e., time-series of different lengths and random initial sensor setup.
Should I simplify my reward function to be just $T_2$?
Why doesn't $Q_0$ change at all?


text:
Can the addition of low-quality images to the training dataset increase the network performance?
I already trained a deep neural network called YOLO (You Only Look Once) with high-quality images (1920 by 1080 pixels) for a detection task. The result for mAP and IOU were 93% and 89% respectively.
I wanted to decrease the quality of my training data set using some available filters, then I use those low-quality images along with high-quality images to train the network again.
Does this method increase the accuracy (or, in general, performance) of the deep neural network (for a detection task)? Like mAP and IOU?
My data set is vehicle images.
mAP: mean average precision
IOU: intersection over union ( or overlap)

Hi and welcome to AI SE! I don't know if it will improve performance, but data augmentation (e.g. the addition of Gaussian noise to the images) can potentially make your model more robust. Maybe someone will provide later a more formal answer with more helpful content.