text:
How can I train a CNN to detect when a person is smoking outside of shop given images from a video camera?
My friend is working at a pizza shop. He takes cigarette breaks in an area that is covered by the public webcam of our town.
I now want to train a convolutional neural network to be able to detect when he is smoking.
Can somebody direct me in the right direction, which tools/tutorials I should look at for this classification task? I already saved 18 hours worth of pictures, one a minute. He is in 28 of these images, I will probably save a few more, maybe 2-3 days. But I don't really know how to start this.

Since this a classification problem you will use a CNN preferably. Then you need to fix an architecture of the CNN like VGGNet or Resnet or Le-net. You can find details on architectures here- Neural Network Architecures. As a beginner you can use VGG 16. You can read about the architecure here- Medium.com blog on VGG 16.

which tools/tutorials i should look at for this classification task?

Tools that you can use:

A python IDE like PyCharm or Jupyter Notebook
Keras and Tensorflow packages

Since, deep learning requires lot of training dataset and demands huge computation power, you can opt for cloud computing platforms like Google Colab or Azure to run your code on, unless you have enough GPU power on your local machine. The above tools are if you want to code yourself. If you want to use GUI(and not write code) Azure Machine Learning Studio is a starting point. Matlab Deep Learning toolbox also provides an excellent GUI with pre trained models on above architectures. However, if you write code on Matlab then you have to ensure your target GPU is NVIDIA as Matlab(Parallel Processing Toolbox, which supports doing computations on GPU) supports CUDA only. It won't work with AMD GPU.
If you want to opt to write code, you can find a step by step implementation on dogs vs cats classification here- Dogs vs Cats classification.

@Howard welcome to AI SE; if the answer resolved your issue, kindly accept it - see What should I do when someone answers my question?

i think it's not fully answered yet, i don't know how to classify the existing pictures as "the person is inside it". I will post my solution when it's done.

@Howard for simple classification you have to label the images with person inside it as 1 and the images where person is not inside it as 0 amd feed it into a CNN with some architecture as mentioned above. The learnable filters will automatically do the job of extracting features that will decide the classification output. This is a basic classification problem based approach. If you are talking about "finding the location of the person" that comes under segmentation and it would require different techniques.


text:
Would empathy in AI be a reliable tool/capacity, or contribute to a solution to avoid harm done to humans or to other versions of AI?
TL;DR
Would providing AI the capability of experiencing something as close as possible to the subjective human experience and from that acquiring empathy in the process be a solution, or contribute to a solution, that seeks to prevent AI to cause the same kind of harm we ourselves as a species have caused to each other in the past, and continue to do so, to us humans and also to itself and/or to other versions of AI?
LONG VERSION:
I was thinking about a question I listened being asked on a podcast where the topic was AI, which was something like: does AI need to have a body in order to understand what it is or how it feels to have a body. The person being interviewed said no. AI can understand it without having a body.
My perception is that a lot of what goes on in AI development at the moment is about AI emulating and simulating human behavior in a way, at the external level, that is by observing human behavior and processing it, looking for patterns etc: speech, motion, the human DNA etc.
I was thinking also that in order for a human being to relate and engage with society and with himself in a constructive, healthy way, and with minimal damage, it will need to possess a certain level of empathy. That is to relate to others and possess the ability to somehow feel what the other is feeling.
Now feeling is not an objective state or something you can observe. Maybe you can measure by observing the human behavior that is a result of feeling but the thing in itself can't be measured or captured and transmitted to AI in a way it can learn from human feeling. One can argue it can be done by studying the brain activity at the moment feeling is occurring. But the subjective experience of experiencing feeling and then being able to recognize that experience in others, understanding what the other is feeling because you have felt the same, that is a subjective occurrence.
It is one thing to understand the human body by observation and study of it's patterns, it's something different to be locked inside one and experience its pains and pleasures.
I am left wondering how can AI understand the human condition if it will not experience what it is to be a human because it is built around the material world, built around what can be objectively observed and measured.
The closest thing there is in human experience to an AI in this regard could be a human that suffers some condition which makes him/her being unable to experience empathy. I was about to use the word Psychopathy but being a loaded word might not be the best choice here. But suffice to say that humans that suffer from some kind of psychiatric condition, or trauma based experience which causes the person to lack the capability to feel empathy towards others might, or not, be easier for that person to in the future engage in actions which can cause harm to others and/or him/herself.
We have cases in human history that might be used as an example. In the past countless human beings engaged in actions which due to its nature I can only conclude this people lacked any empathy or were placed in a situation where empathy was drained out of them. Like for example in war, usually soldiers and the whole country will go through a phase of propaganda which consists of dehumanizing the enemy. I believe this is done in order to facilitate and eliminate any sense of empathy and humanity in the population and building up the necessary will, energy and determination for the killing of the enemy.
The solution to this question might be to create in AI the capability of experiencing subjectively what it is to be human, or what it is to have a body, or come up with something that makes it possible for AI to experience something similar to human empathy.
One thing comes to mind which is the merge of AI and the human mind. But this might be another topic altogether.
It has been argued in some places I remember, even in some Science-Fiction that actually the main problem of destructive human behavior is the ability to feel emotion and feeling. And the solution might just be to eliminate or diminish that part of ourselves. I really don't agree with this view and it seems to me that ATM our society does engage and promote  solutions of this type: drugs such as some anti-depressants for ex. instead of going to the root of the problem which in many situations are trauma based. I really hope we don't go this way. I believe that if we have a future the solution is to find ways of developing and cultivating empathy.

Hello Welcome to AI.SE please edit your question. Normally websites for Q&A is just only 1 question per post. I see 6 question in this post. please be more specific. what is your question? maybe you need to read this and the on topic article in meta. If your question is about philosophy add the tag philosophy.

Thank you for your feedback. I have updated the question.

I think the question should probably be trimmed a bit if more people are to engage with it. In the meantime, you can find materials related to AI safety on e.g. youtube. Robert Miles is a good place to start.

Hello. Welcome to this community! As suggested in the previous comments, this post can probably be shortened. I tried to read the first paragraphs, but I gave up after that. I would also like to note that our site is not suitable for discussions. If your question is philosophical (in the context of or related to AI), you should make it clear, otherwise, you should ask a question that, ideally, can be answered objectively.

Having said that, the topic of "consciousness" and "subjective experience" in the context of philosophical/speculative discussions related to AI is not new. I would encourage you to read more about the topic on the web (or somewhere else). Maybe, after that, your question can be shortened, if you're not able to do it now. A good rule of thumb to understand whether your question is specific enough is if you can put that specific question in the title (with the length restrictions). So, please, do put your specific question in the title.

Ok I wrote the final question or TL;DR at the top. You don't need to read everything.

It seems to me that you're asking 2 questions here. 1. "How can we be sure that AIs will not harm us?". 2. "Would be that be implemented?". Are these your 2 questions? In particular, I am not sure when you say "harm" if you mean "us" (humans, and maybe other animals and plants) or "harm itself". Can you clarify that in your question? Having said that, your questions I think will just lead to speculations, as it seems to me that we still don't know the definitive answers to your questions.

Can you please put your specific question in the title too? "AI, empathy and the human subjective experience" is not a question and it's too general (i.e. not specific). Moreover, from your "TL;DR or FINAL QUESTION:" section, it's still not clear if the harm that the AI could do is to itself or us, humans.

I rewrote the title and changed the TL;DR a bit.

SHORT ANSWER: Yes. Empathy can is a good emotion for an AI, but no the best for avoid harm people, could contribute? YES.
ANSWER ABOUT PROVIDE HUMAN EXPERIENCE TO AN AI:
For Humans empathy is not the thing most important for avoid people hurt each other. We may need more, and this is a good law system with people believing in that (political, religious or cultural), a system based in laws that makes the people think twice before committing a crime, a system of laws based on the punishment if a crime is committed. that make that people have a afraid in the future for do it crimes, genocides and damage to the society and people. With this system an AI would be lose more than earn.
I think empathy is not enough for the AI you need laws just like the people. If I do this then that happen to me. a good example can be if a AI harm people eventually will be turned off just like we do with criminals with the death penalty or life imprisonment. The Asimov laws is a good starting point.but we need more than that.
Providing AI with "human emotions": We may have more problems, for example, AI may develop strong emotions such as hatred, Resentment, Envy, Fear, Revenge and eventually a War this is the main reason for the Holocaust, Rwandan-Genocide, and a lot of conflicts in the world. We are a violent species if we have to create an AI it will be to be better than us, these emotions are useless. so you have to be eliminate any emotions that lead to conflicts.
Here are some extension lectures:
The amygdala — a part of the brain involved in fear, aggression and social interactions — is implicated in crime.

You mentioned the holocaust. One of the reasons I believe it happened was I think a lack or even no empathy at all on the part of many of the people involved in the Nazi regime. Also you mentioned "a system based in laws that makes the people think twice before committing a crime". I agree. What about when the system is hijacked by authoritarians and/or criminals as was the case with Nazi Germany? We need a System of Laws true, but we also need safety measures against those Systems if hijacked. AI could be better than humans when using and expressing Empathy.

I am agree with you. that's why I wrote about different systems for example for humans if not is the law, is religious, is not is religious, is a cultural, if not cultural can be human moral principles. all for avoid harm us each other. maybe we can construct this for an AI, and program empathy should be nice. There is a question related to this here maybe you will be interested.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Is there a way to update the neural network to fit the new data without the time required for retraining?
I built a basic neural network in MATLAB. The neural network classifies points on the X-Y axis system into two classes (0 and 1).
(I try to get the function that represents a shape from this photo)

Every so often the values of the points change slightly and some of the points defined in class 1 become class 0, like in this photo.

Is there a way to update the neural network to fit the new data without the time required for retraining?

Is your network's input just the x and y coordinate, or an actual image?

If I understand your question, you want it to generalize to be conditioned on an image. If this is correct, you can do this via inserting a separate portion of the model that takes the image as an input, and compresses it into a dense feature vector (can be done many ways- most common is probably cnn) and then merge with the other inputs (in your case x,y pairs) and have the model train on that end-to-end.

If I understood correctly what you mean, you say insert the image as input along with the XY coordinates and the network output will be Is the coordinate inside the black part of the image?
And what exactly is "dense feature vector"?
Sorry for the ignorance, I'm really new to the field.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How do multimodal models establish connections between different modes?
I am specifically interested in data2vec, Meta's new model that can convert image, text, and sound data into a unified neural network representation. To my understanding, they did this through self-supervised learning by masking parts of the input and having the network predict the hidden states if the input hadn't been masked. This allows these modes to share a common representation.
However, I don't understand how the representations of different modes can be connected. For example, how are the hidden state representations of an image of a banana and the word banana trained to be similar, if they are at all?

One option would be an association network between the various individual latent spaces, having its own multi-modal latent space, where the shared phenomena reside.

I agree with the answer by @Funzo. The corresponding section that clarifies this in the paper can be found on page three:

Multimodal pre-training. [...] Our work does not perform multimodal training but aims to unifiy the learning objective for self-supervised
learning in different modalities. We hope that this will enable better multimodal representations in the future.
[ArXiv]

So what the authors contribute is a framework for learning rich data representations, given any type of data. The framework is the training pipeline that uses the same network in a teacher and student setup. The network first generates data representations from unmasked data (teacher mode) and then does the same with a masked version of the data (student mode). The training objective is minimizing the error of all latent representations between the network in student mode and in teacher mode. If you were to apply the approach you have to choose a model suitable for your data (e.g. the authors use a Vision Transformer for Images).

it's not multimodal.
It's trying to be a standard way of training a for a model.
But it is not suppose to work across modalities

Hi @Funzo, welcome to AI.SE. Do you think you could perhaps expand this answer a bit? As it stands, it makes some claims, but doesn't explain them or provide any sources. Adding sources and explaining your answer some more would greatly improve the quality of the post. Thank you!


text:
Why does validation accuracy stop rising so soon?
I have implemented a GRU to deal with youtube comment data. I am a bit confused about why the validation score seems to even out around 70% and then keeps rising, this doesn't look like overfitting from what I'm used to since it keeps rising. Is this normal? Does anyone know what's happening here?
I've implemented the GRU as a simple GRU, then GRU with dropout and then a multilayer (2 layered) GRU with dropout. The graphs are shown below.

As you can see, in the multilayer GRU with dropout , the validation accuracy does nicely follow the training accuracy. Does this simply mean that the other models are simply not capable of capturing certain information? Is there a way to improve these results, what parameters should be optimized?

Since all networks' accuracy goes close to 100%, I would argue that all of the models are capable of learning this task. But the first two models are somewhat overfitting, since the validation accuracy doesn't get nearly as high. Granted the 2nd model uses dropout, but it seems that the dropout rate is not enough to bring these two accuracy metrics closer together.
Does the 3rd model have more trainable parameters than the first two? I don't know why it would have so much higher validation accuracy than the first two.
It is also worth repeating these experiments at least five times, and check if you see this pattern always or was it just a random occurrence.


text:
Reason why chess neural network might not be training
I've been trying to use a Stockfish-like chess evaluation neural network for the past few weeks but to no avail. I wanted to get some other opinions about why my current methods haven't worked.
Input: $8 \times 8 \times 12$ one-hot encoded board of pieces. I have around ~60 million unique examples. Evaluations are placed between -15 and 15 (anything greater or smaller is brought to 15 or -15)
Output: Single number representing Stockfish evaluation.
I've tried both fully connected and convolutional models and neither have worked very well. Here's some Tensorflow code that might give an idea of what the structures looked like:
Convolutional model:
import tensorflow as tf

def block(filters, x, res=None):
  if res is not None:
    x = tf.keras.layers.concatenate([x, res])
  x = tf.keras.layers.Conv2D(filters=filters, kernel_size=3, padding='same', activation='relu', kernel_initializer='he_normal')(x)
  x = tf.keras.layers.BatchNormalization()(x)
  x = tf.keras.layers.MaxPooling2D(pool_size=2, strides=1, padding='same')(x)

  return x

board = tf.keras.Input(shape=(8, 8, 12))
conv1 = block(64, board)
conv2 = block(128, conv1)
conv3 = block(256, conv2)
conv4 = block(512, conv3, conv3)
conv5 = block(1024, conv4, conv2)
conv6 = block(1024, conv5, conv1)
conv7 = block(512, conv6)
x = tf.keras.layers.Flatten()(conv7)
x = tf.keras.layers.Dense(128, activation='relu')(x)
x = tf.keras.layers.Dense(32, activation='relu')(x)
x = tf.keras.layers.Dense(1)(x)

Fully connected model:
import tensorflow as tf

board = tf.keras.Input(shape=(8, 8, 12))
x = tf.keras.layers.Flatten()(board)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu', kernel_regularizer='l2')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dense(256, activation='relu')(x)
x = tf.keras.layers.Dense(1)(x)

Just to add a few notes about what the current results are, they are decent (e.g. mean squared error of Stockfish evaluated cut off between -15 and 15 is about 7). The early game is also played fairly decently but the late game gets really bad (the engine can't evaluate that bishop takes enemy queen is good). My evaluation network also evaluates the four move checkmate as black favoured.

It could be because there is simply not enough data for the late game. To make the model give more importance to the later stages of the game you can try to tweak the loss function such that it penalizes more for when there are fewer pieces on the board. ( This might give an idea on that: https://medium.com/visionwizard/understanding-focal-loss-a-quick-read-b914422913e7)
One other thing you can do; given that my above-mentioned reasoning is correct, is to provide the number of pieces on the board as an input. I am not sure how you might go around doing that in the one-hot encoded scenario but might be worth looking into (You could append the number of pieces in the binary format to the flattened input). One argument against this is that the network should implicitly know how many pieces are on the board but yet it could still be useful to give it as an extra input.

I tried hacking together several NN approaches for chess, in the context of Chess Coding Challenge (implement a chess bot in a relatively small amount of code). I also play online chess semi-regularly, so I have some understanding of the game. I found that in general the more accurate my network was with evaluation (and the more complex it was), the worse it played since it ended up in uncommon situations which it grossly mis-evaluated to its favor.
I haven't done a detailed analysis, but I suspect that there two main factors here:

Lack of diversity in the training data, for example does your network correctly evaluate a starting position with queen odds?
The difficulty of evaluating positions which involve tactics

Here are three similar positions, sorry it is always black's turn to move, since I encountered these when playing as black:

Evaluation: -6.4

Evaluation: -3.3

Evaluation: -2.2
I don't know a good solution on how to correctly evaluate tactics, except searching explicit steps in the game tree. Note the first position has a rather simple two-move tactic, but modern engines will do searches tens of moves deep.
And regarding the first issue, I got better results by imposing strong regularization to the network's architecture and weights. For example a good starting point would be an evaluation function which resembles a Piece-Square-Table.


text:
If the probabilities with which each task is selected for you do not change over time, why would it appear as a single stationary k-armed bandit task?
Sutton-Barto (Section 2.9-Associative Search (Contextual Bandits), page 41):

As an example, suppose there are several different k-armed bandit tasks, and that on
each step you confront one of these chosen at random. Thus, the bandit task changes
randomly from step to step. If the probabilities with which each task is selected for you
do not change over time, this would appear as a single stationary k-armed bandit task,
and you could use one of the methods described in this chapter.

If the probabilities with which each task is selected for you do not change over time, why would it appear as a single stationary k-armed bandit task?

The critical detail of the initial multi-k-armed bandit as it is described in the quoted text, is that the agent cannot detect or use knowledge of which bandit problem it is presented with on each time step.
Therefore the expected reward from action $a$ is the weighted sum of expected rewards from all bandits for that action. It is weighted according to the random selection, and stays fixed for the combined bandit, same as for the individual bandits that it is composed of. This is a single real value for expected reward, which means optimising it looks identical from the outside to optimising a simpler bandit with less "machinery".
All of this changes as soon as the agent is allowed to observe data, i.e. the context of which sub-distribution it is taking an action for in the next time step.

This answer is not clear for me as well. I would be happy if it can be updated.

@DSPinfinity What exactly is not clear? It is pretty thorough IMO. I probably could not do better without prompting what the problem is. You may need someone else to answer

I think it will be clear if wording is changed and more explanations are added.


text:
Is a convolutional layer capable of converting, for example, a binary image into an RGBA image?
I am asking this question for a better understanding of the concept of channels in images.
I am aware that a convolutional layer generates feature maps from a given image. We can adjust the size of the output feature map by proper padding and regulating strides.
But I am not sure whether there exist kernels for a single convolution layer that are capable of changing an {RGBA, RGB, Grayscale, binary} image into (any) another {RGBA, RGB, Grayscale, binary} image?
For example, I have a binary image of a cat, is it capable to convert it into an RGBA image of a cat? If no, can it at least convert a binary cat image into an RGBA image?
I am asking only from a theoretical perspective.

You say "I am not sure whether there exist kernels for a single convolution layer", but note that the kernels in a CNN are usually learned. Note also that you don't need CNNs, for example, to convert RGB images into grayscale/binary ones. There are algorithms to convert RGB images into grayscale images. I'm not fully sure how this question that you're asking is related to your confusion about channels.

I'm also not sure why you're so confused about this concept, as it's not really anything special (usually it just refers to the 3rd dimension of the image or feature map, i.e. would be a synonym for depth, although in the case of the images the depth has some meaning to us, as each slice, for example, in RGB images, corresponds to the values of the red, green and blue color, hence the name RGB)

Please check here. It has two arguments in_channels, out_channels. What is the purpose of them? Example shows 16 input channels and 33 output channels. I am aware about images with 1 channel, 3 channels and 4 channels. @nbro

Someone in the other answered provided a link to an answer on Data Science SE by an anonymous user (who was me before I delete the account there). It explains these parameters.

In the input layer of the CNN, typically, you will have in_channels == 1 or in_channels == 3. However, in hidden layers of CNNs, you can have in_channels == K for $K > 1$, because this corresponds to the depth of the feature map that you produced in the previous convolutional layer, which corresponds to the number of kernels that you applied to the input of the previous layer (I'm assuming a 2d convolution).

Oh! So, number input channels is the depth of the image and number of output channels is the number of kernels.  Why the example shows in_channels =16 then? Is it just for he sake of example or the example of its usage as hidden layer or are there any sensible images with number of channels equal to 16 @nbro

Yes, I think it's just an example.

@nbro now, I am liberating from this channel jargon.

No, because each output from a convolution layer only looks at a local region of the image. A convolution layer cannot do any global transformation, only local ones. Convolution layers must have translation invariance which means if it converts an eyeball to a tail at one position, it'll also convert the same eyeball to the same tail if it's found at a different position. If it's not overfitted, it will also convert similar eyeballs to similar tails. If you want only some eyeballs to become tails, you can't do that without introducing overfitting, or expanding the convolution size until the layer can see enough context to distinguish which eyeballs should become tails and which ones shouldn't.
If you want to change one image into a specific other image, and don't care what happens to all other images, it might be possible to create a convolution layer that does this transformation. The input image has to be different wherever the output image is different, or else the convolution layer won't be able to produce that difference in the output image. You would be teaching it to recognize the specific pixel patterns in the input image and generate the specific pixels in the output image. This would be an extreme case of overfitting and wouldn't work for any other input images.
The number of channels in the input and output image is irrelevant, except that more channels means the network has more data to learn from, obviously.

The number of channels in the input and output image is irrelevant, except that more channels means the network has more data to learn from, obviously. does it mean it is possible to change image with $k$ channels with an image of $n$ channels?where $k \ne n$?

@hanugm Yes, you can do this in any image processing software. Of course, if you change to a better format, it will not magically improve the data, and if you go to a worse format, the quality of the image will decrease. For example, you can convert a colour image to greyscale.

If possible, please add in answer that an image of $k_1$ channels can be converted by a convolution layer in to an image of $k_2$ channels where $k_1 \ne k_2$.

@hanugm You think there is something magical about channels for some reason. There is not.

True. Due to my misconception about the word channels I am facing complications.

@hanugm In an image, Red is a channel. Green is a channel. Blue is a channel. You should have no problem understanding that images are made of red, green and blue components. here is a little visualization They are fully separate to the computer. Or if you are asking about channels in intermediate layers (are you?), those are the same thing but with a different purpose. Those won't be red, green or blue. They'll be more like, amount of horizontal lines, amount of vertical lines, amount of circles.

I personally know that an RGB image has 3 matrices corresponding to each color dimensions. But, in literature, the word channel is confusing for me. For suppose, you can see that Conv3d has two parameters in_channels, out_channels. I am not sure whether these channels are the dimensions of the image i am thinking of. If yes, in many programs I saw their value beyond 3, but working on RGB images only.  #1/#2

Similarly, I encountered some other explanations using the word "channels" which I cannot reconcile with the interpretation I had. #2/#2

You can see the confusions I have due to the word channel. In fact, I created the tag channel due to my misconceptions or some other reason.

Let us continue this discussion in chat.


text:
What is a simplified way to explain why the AI researchers Bengio, Hinton, and Lecun, won the 2018 Turing Award?
The Turing award is sometimes called Computer Sceince's Nobel Prize. This year's award goes to Bengio, Hinton, and LeCun for their work on artificial neural networks.
The actual work contributed by these authors is, of course, quite technical. It centers around the development of deep neural networks, convolutional neural networks, and effective training techniques. The lay press will tend to simplify these results to the point that they lose meaning.
I would like to have a concise, and yet still precise, explanation of their contributions to share with a lay audience. So, what is a simplified way to explain the contributions of these researchers?
I have my own ideas and will add them if no other satisfactory answer appears. For a "lay" audience, I want to assume someone who had taken a college level course in something scientific but not necessarily computer science. Explanations that are suitable for those with even less background are better still though, as long as they don't lose too much precision.

I have just noticed that you are already linking to an ACM article, which describes their contributions. So, what kind of answer are you actually looking for?

@nbro I've looking for an answer that meets all of these criteria: a): short. Ideally a few sentences. b): does not rely on field specific jargon, to the greatest degree possible. c): Does not lose too much precision (i.e. not just "they made artificial brains."). I think this is quite hard to communicate accurately and briefly to a lay audience. The use case I'm imagining is something like, I bump into a group of first year CS students at the college I teach at, or a biology professor in the lounge, and want to give them an elevator pitch of the recent award.

I think it will be difficult to explain their specific contributions without relying on jargon. But, for example, rather than talking about back-propagation, you could talk about "a mathematical technique which allows models to learn from data". I am not sure how you could simplify this further.

@nbro I agree that this is tricky to do. I have some ideas for how this could be simplified further, but because it is tricky, I'm hoping someone has better ideas than mine. As an example, Hinton's backpropagation could be summarized as "We knew for a long time that neural networks could represent complex patterns, but we didn't know how to find those representations just by looking at examples of the pattern. Hinton developed an algorithm to do that."

Anyway, I would like to note that Hinton did not really invent back-propagation, which was already known in the 70s. See http://people.idsia.ch/~juergen/who-invented-backpropagation.html. Furthermore, although these researchers have definitely contributed to current situation of AI, there are a lot more people that could have won this award, because of their impact. For example, Sepp Hochreiter and Jürgen Schmidhuber for the development of the LSTM (which is literally everywhere). Maybe they will win it the next years.

@nbro that's true. He did show that it let to the construction of "correct", meaningful representations, like those we might use when constructing a network by hand. Without that, it is doubtful BP would have been widely adopted for training these networks.

The related ACM article describes a few specific technical contributions, which led the ACM to award them.

Geoffrey Hinton
Backpropagation: In a 1986 paper, "Learning Internal Representations by Error Propagation", co-authored with David Rumelhart and Ronald Williams, Hinton demonstrated that the backpropagation algorithm allowed neural nets to discover their own internal representations of data, making it possible to use neural nets to solve problems that had previously been thought to be beyond their reach. The backpropagation algorithm is standard in most neural networks today.
Boltzmann Machines: In 1983, with Terrence Sejnowski, Hinton invented Boltzmann Machines, one of the first neural networks capable of learning internal representations in neurons that were not part of the input or output.
Improvements to convolutional neural networks: In 2012, with his students, Alex Krizhevsky and Ilya Sutskever, Hinton improved convolutional neural networks using rectified linear neurons and dropout regularization. In the prominent ImageNet competition, Hinton and his students almost halved the error rate for object recognition and reshaped the computer vision field.
Yoshua Bengio
Probabilistic models of sequences: In the 1990s, Bengio combined neural networks with probabilistic models of sequences, such as hidden Markov models. These ideas were incorporated into a system used by AT&T/NCR for reading handwritten checks, were considered a pinnacle of neural network research in the 1990s, and modern deep learning speech recognition systems are extending these concepts.
High-dimensional word embeddings and attention: In 2000, Bengio authored the landmark paper, "A Neural Probabilistic Language Model", that introduced high-dimension word embeddings as a representation of word meaning. Bengio's insights had a huge and lasting impact on natural language processing tasks including language translation, question answering, and visual question answering. His group also introduced a form of attention mechanism which led to breakthroughs in machine translation and form a key component of sequential processing with deep learning.
Generative adversarial networks: Since 2010, Bengio's papers on generative deep learning, in particular the Generative Adversarial Networks (GANs) developed with Ian Goodfellow, have spawned a revolution in computer vision and computer graphics. In one fascinating application of this work, computers can actually create original images, reminiscent of the creativity that is considered a hallmark of human intelligence.
Yann LeCun
Convolutional neural networks: In the 1980s, LeCun developed convolutional neural networks, a foundational principle in the field, which, among other advantages, have been essential in making deep learning more efficient. In the late 1980s, while working at the University of Toronto and Bell Labs, LeCun was the first to train a convolutional neural network system on images of handwritten digits. Today, convolutional neural networks are an industry standard in computer vision, as well as in speech recognition, speech synthesis, image synthesis, and natural language processing. They are used in a wide variety of applications, including autonomous driving, medical image analysis, voice-activated assistants, and information filtering.
Improving backpropagation algorithms: LeCun proposed an early version of the backpropagation algorithm (backprop), and gave a clean derivation of it based on variational principles. His work to speed up backpropagation algorithms included describing two simple methods to accelerate learning time.
Broadening the vision of neural networks: LeCun is also credited with developing a broader vision for neural networks as a computational model for a wide range of tasks, introducing in early work a number of concepts now fundamental in AI. For example, in the context of recognizing images, he studied how hierarchical feature representation can be learned in neural networks - a concept that is now routinely used in many recognition tasks. Together with Léon Bottou, he proposed the idea, used in every modern deep learning software, that learning systems can be built as complex networks of modules where backpropagation is performed through automatic differentiation. They also proposed deep learning architectures that can manipulate structured data, such as graphs.

I would like to note that back-propagation was apparently first implemented by Seppo Linnainmaa and not Hinton.

The quoted text by ACM does not claim that Hinton invented (or first "implemented") BP; and in the link provided, Schmidhuber says that the first explicit & efficient BP was "described" by Linnainmaa in 1970, "albeit without reference to NNs". It would seem that Hinton et al. are unambiguously credited for first applying BP to NN training, which apparently does count.


text:
Does it classify as Machine Learning?
I have a gaussian distributed time series ($X_t$) with some parameters in my experiment. Suppose I want to know the mean $\mu$. If I define another time series $Y_t$ such that $Y_t=X_t-a$ for all $t$. Now say I vary this parameter $a$ and generate altogether different time series for each $a$, say $Y_t(a)$. I look at the mean of $Y_t$ for each $a$. The value of a, where I get the mean of $Y_t$ closest to $0$, will be my estimate of $\mu$. Say I will eventually use this learnt value of $\mu$ to generate $Y_t$ as my final goal. Can this be called ML? I am using some training data of $X_t$ to learn about its parameter and then using test data of $X_t$ to generate $Y_t$.
Now why am I working so hard on this simple problem? Well, actually I am not. I am doing something else, which will have lots of parameters in the time series and will be used to generate other time series after similar parameter extraction. That will be too complicated to discuss here. I just wanted to clear my basics using an over-simplified example.

I don't understand what you mean by "gaussian distributed time series". Can you precisely define it? Are you saying that each element of the time series is sampled from 1. the same or 2. possibly different Gaussian? In general, I don't fully understand your problem. Do you have the original time series $X_t$? If not, do you want to estimate it from "noisy" similar time series? I don't understand how subtracting $a$ from $X_t$ gives you any insight. Also, I don't understand how you are going to generate $Y_t$ if you don't have $X_t$. Are you assuming that $Y_t$ is $X_t -a$, for some $a$?

@nbro Thanks for your response. The time series $X_t$ is given, and we know it has a Gaussian distribution, (histogram: Gaussian). So each point in $X_t$ belongs to a gaussian distribution with the same mean and variance. Now if you subtract a from each point in $X_t$ to construct a different series $Y_t$ the mean of the gaussian shifts. For some $a$, the constructed $Y_t$ has 0 mean. What I am asking if you vary $a$ and try to see for what $a$ this is achieved, then use that correctly found a, to generate more data points of $Y_t$, from $X_t$ can I call it as ML?

Does it classify as Machine Learning?
If I understand the problem correctly, my opinion is yes, this is machine learning.  In fact, I think it is just linear regression for finding the equation of a horizontal line y(t) = a that fits the the dataset y(t) = Xt.


text:
Are there any general guidelines for dealing with imbalanced data through upsampling or downsampling?
Are there any general guidelines for dealing with imbalanced data through upsampling/downsampling?
This Google developer guide suggests performing downsampling with upweighting, but for the most part I've found upsampling usually works better in practice (some corroboration).
Is there any clear consensus or empirical study of what works in practice, or when to use which?  Does it matter which classification algorithm you use?


text:
Why is loss displayed as a parabola in mean squared error with gradient descent?
I'm looking at the loss function: mean squared error with gradient descent in machine learning. I'm building a single-neuron network (perceptron) that outputs a linear number. For example:
Input * Weight + Bias > linear activation > output.
Let's say the output is 40 while I expect the number 20. That means the loss function has to correct the weights+bias from 40 towards 20.
What I don't understand about mean squared error + gradient descent is: why is this number 40 displayed as a point on a parabola?
Does this parabola represent all possible outcomes? Why isn't it just a line? How do I know where on the parabola the point "40" is?

Are you using a specific loss function here? Please use [edit] to explain which one. I expect it is MSE, and if so that should cover everything needed to answer your question

Every minimum point can be approximated by a parabola.

@Kostya: Yes, but you probably would not draw a parabola for e.g. $\mathcal{L}(\hat{y}, y) = |\hat{y} - y|$

I suspect the illustration isn't meant to be taken literally; instead, I suspect the author is intending to illustrate that gradient descent attempts to solve the minimization problem by moving downward toward what is (hopefully) a global minimum on some complex surface.

I added "mean squared error" to the question for clarity. The question is still why would all the possible "wrong" loss values happen to be points on a parabola (say, 40, 80, 14... all wrong values)... That connection is not explained in most tutorials.

Mean Square Error (MSE) is a quadratic function and the further you go away from your optimum the bigger (quadratic) the MSE gets. Take $o_{expected}=20$ and $o_{net}=40$ as example. Your MSE is then 400, because
$MSE = (o_{expected}-o_{net})^2$.
Just imagine $y = x^2$ with x being the output of your network.
If you want to shift the parabola with optimum at $20$ the formula you get is $y = (20-x)^2$.
For every new case you train the  net on, you get a different parabola with different parameters.


text:
Where do you keep track of Chat History for an LLM Chat Application?
I am creating a LLM chat app with LangChain. I am keeping track of the chat history with a simple array that gets sent to the LLM with every prompt. I noticed LangChain also has classes that automatically keep track of the whole conversation, such as BufferMemory and ConversationBufferMemory
https://python.langchain.com/docs/modules/memory/types/buffer
It's not explained however if these classes automatically keep track of different users as well? What if two users connect to the script, will their conversations get mixed up with each other?
How do you keep track of multiple users in a Chat LLM application?

This becomes more like a system design question to distribute the app for multiple users. It can be redis to store cache, or you can keep the records in db for longer runs.


text:
How do I normalise/un-normalise data when loading a model?
I am following this TensorFlow JS tutorial where you load car data. The data looks like this:
[{x:100, y:20}, {x:80, y:33}]

X is the horsepower of a car, Y is the expected miles per gallon usage. After creating the model I save it locally using:
async function saveModel(){
    await model.save('downloads://cars-model');
}

Next, I load the model in a separate project, to make predictions without needing the original data.
NEW PROJECT
async function app(){
    let model = await tf.loadLayersModel('./cars-model.json');
    console.log("car model is loaded!");
}

I expect to be able to run predict here, on a single number (say, 120)
model.predict(tf.tensor2d([120], [1, 1]))

QUESTION
I think the number 120 needs to be normalised to a number between 0-1, just like the training data was. But how do I know the inputMin, inputMax, labelMin, labelMax values from the loaded model?
To un-normalise the prediction (in this case 0.6) I also need those original values.
How do I normalise/un-normalise data when loading a model?
original prediction code uses label and input values from the original data
function testModel(model, inputData, normalizationData) {
    const { inputMax, inputMin, labelMin, labelMax } = normalizationData;

    // Generate predictions for a uniform range of numbers between 0 and 1;
    // We un-normalize the data by doing the inverse of the min-max scaling
    // that we did earlier.
    const [xs, preds] = tf.tidy(() => {

        const xs = tf.linspace(0, 1, 100);
        const preds = model.predict(xs.reshape([100, 1]));

        const unNormXs = xs
            .mul(inputMax.sub(inputMin))
            .add(inputMin);

        const unNormPreds = preds
            .mul(labelMax.sub(labelMin))
            .add(labelMin);

        // Un-normalize the data
        return [unNormXs.dataSync(), unNormPreds.dataSync()];
    });

    const predictedPoints = Array.from(xs).map((val, i) => {
        return { x: val, y: preds[i] }
    });

}

Hi @Kokodoko did you work out a solution? I've hit the same road block, pulling in the data for the original min/max as a JSON but its giving me troubles. I have been googling for days but can't find any tutorials that load separate and this is the only reference to it I can see? weird eh? How is everyone else doing it :)

@laurencefay According to the answer below you just have to save the original min/max values in a separate JSON file. Then you can load that JSON file later to retrieve those values and un-normalize the results

How do I convert this to a number between 0-1 without having access to the original car data?

You save the normalisation parameters (typically an offset and a multiplier for each column), and consider that part of the model. Typically you do this when you originally scale training data.
When you want to re-use the model, as well as loading the neural network architecture and weights, you need to load the normalisation parameters in order to re-use those too and scale your inputs.
When tutorials present a self-contained neural network that loads training data, builds a model, then tests that model, all in the same process, then often this step is not shown. However, saving the normalisation data is important, and basically should be considered part of the model, even though it is not directly part of the neural network parameters or hyper-parameters.

OK, thanks for clearing that up. So do I save that data manually, in a separate file, or should it be part of the saved model?

@Kokodoko: It's up to you as the developer which you prefer, and may depend on your use case. For instance if you always use the same training data, scaling it the same way and want to move different NNs around for testing, then separate files will be slightly easier and quicker to develop. But a self-contained file with supporting data for the model might make more sense in other situations

But if I use model.save, is there an option to include that data? Can I add it manually before saving? For example, model.min = 100; model.max = 200.

@Kokodoko: I don't know, that depends on how function save is written

I'm using the Tensorflow JS API: https://js.tensorflow.org/api/latest/#tf.LayersModel.save
It seems to have a config option but this is not further explained... :(

From the description it does not look like it will accept added data, so you will need to arrange to store the normalisation parameters separately


text:
How to choose evaluation functions for features, when network effects are in place (multi-agent systems)?
So, I have this huge amount of data, which has 7 vector features (float from 0 to 1). I am trying to build a kind of recommendation system, with a twist (it uses agents and negotiations and narratives; narratives meaning, that there will be temporal and partial order causal link dynamics, or "short term memory").
There is network effects in the data, since the more agents I have, the more potential there will be for match making; in simulation phase I will be just using randomness, instead of real data, but I believe the real dataset will also be more or less random: of course I think there will be nice clusters of correlations for the vectors, but I also want to understand the more general aspects of the problem, instead of just this specific use-case.
The agents will be generating a list of artifacts in co-operation and also evaluate how the generated list of artifacts match their preferences. The agents will be recommending items from their own list (and some times from a list of their related agents) and after each suggestion, there is negotiation.
If there would not be network effect in the data, I would use simple heuristics, which would measure the diff of each feature. However, in network effect world this does not work very well, because it will make many agents unable to co-operate.
I was thinking of something, where some features are preferred features by the agents in different circumstances (depending on the other agents), this will enable some elasticity to the system and IMO avoid the simple problems, of which many recommendation systems suffer (lack of context awareness; humans do not like the likes of other people, we have flexible preferences, which depend upon our mood).
Since this sounds like a very generic problem, it seems like I have just discovered a problem, which has been elegantly solved by someone else from the academia years ago, but I just don't know the name for this concept.
Instead of depending on strict evaluation criteria (only accept diff X per feature), how to choose evaluation function to make the agents more co-operative? I was thinking something, where diff has to be narrow for at least one feature, but may be big for three worst matching features; so that there would be essence of few features and diversity of several.
EDIT: As I have spent some more time with this problem, I have a hunch about which issues might be related:

In real world networks, which depend upon energy efficient co-operation and signaling, scale-free networks seem to emerge; effective Agent-Based Model in network effect scenario should probably form co-operation networks, which follows some kind of Fitness Function.
Due to scale-free networks, it should be assumed, that the Agents using essential features (which are common in data) should be ranked higher and Agents using diverse features (which are rare in data) should be ranked lower; in scale-free networks (like Facebook) it has been observed, that all but 3% of nodes form edges to more popular nodes: this makes signaling efficient.
As a reference, in deep learning Capsule Networks have had some state-of-the-art success with this similar hierarchical "recommendation for upper tiers" mentality (there are much more lower layer features, than upper layer features).

So, I have this idea, that when we use Agent-Based Models for solving problems with network effects, the evaluation function should evaluate Agents skill of passing the problem for more popular Agent, which could delegate it to the best Agent or attempt to solve it on it's own.

In a sense it seems like I am thinking some kind of "Deep Agent-Based Modeling", where it is okay to have network effects on the lowest layer (which would only evaluate the matching of the generated artifacts and their own preferences); the co-operation problem should be solved by a "Coach" layer, which would attempt to prevent situations, where some agents can not play well together.
The evaluation function of the "Coach" layer should be such, that it minimizes the amount of negotiation actions (energy efficient co-operation) required per artifact list generated and also minimizes the signaling distance regarding diversity (less popular artifacts / unorthodox solutions). The "Coach" layer should also be able to reorganize the lists contained by the Agents. The signaling distance can be easily evaluated in my use case, since the dataset has easily available popularity ranking per list artifact list item; the shorter the distance from popular artifact list to the least popular artifacts is, the better.
In practice this means, that I actually want to build composable Agents, which have lists of items (or other Agents), action space for generating artifacts from those items, evaluation functions and costs of specific actions, aggregateable popularity rank (which helps to evaluate delegation). All Agents would have generic default action space, which would help in solving the signaling problem; if the diameter of the co-operation network is too long (not within the scale-free network topology), the system should attempt to create a new layer or reorganize the existing layers (or do both).
All this might be computationally very expensive; however we tested a theory about building a "multi-dimensional" merge-sort algorithm by using entanglement / super dense encoding from quantum information theory and it would seem like it could work and be able to create a hash map, which would allow the Agents to reshape the scale-free network efficiently.
I might build some kind of demo out of this, since all this seems really interesting to me and complements nicely my past hobby projects.
I will not accept this answer until I get some verification, that this is a good answer. However, I will post this in hopes, that someone has already done similar work, gets inspired and can provide a better answer. This answer will probably be edited later, when I get to a more formal level with this.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Would it make sense to use together capsule neural neworks and "topic / narrative modeling"?
This is actually something I have been researching a bit on my own.
Most movie scripts can be structurally analysed by using writing theory such as Dramatica. Dramatica is based upon a hierarchy of concepts, which can be topic modeled. The hierarchy of topic models would seem to work very well with the capsule neural networks.
I have been working with computational creativity problems in a narrative generation. The state of the art methods use Partial Order Causal Link Planners, but they depend on propositional logic. Alonzo Church presented the Superman dilemma (Louis Lane does not know that Clark Kent is Superman, but Superman knows, that he is Clark Kent) and invented Intensional Logic as a solution; the basic idea is, that if we do not know the context of the narrative, the meaning is always in superposition and can only be understood through entangled meanings from the background story. So, in a sense, propositional logic is limited by classic information theory constraints, while Church's logic can take a quantum information-theoretic approach. I do not believe that classic information theory can resolve narrative analysis problems. So, basically, the meaning of a narrative collapses (the superposition gets resolved) by using the hierarchical narrative structure and what we know beforehand.
So my intuition would be the following:

We can use Dramatica and potentially other narrative theories (hierarchical metamemetics, reverse SCARF, etc.) to create a hierarchical network like ImageNet, but for narratives.
We can build conceptual topic models. Dramatica has a hierarchy of 4-16-64-64 concepts and annotated data exists already.
When using hundreds of topic models, there will be a lot of false positives. However, the superposition of the topic models can be collapsed by using the hierarchical levels and some other dramatic analytics.
By using the capsule neural networks, we might be able to build a system, which could determine a narrative interpretation of the full story, which would make the most sense by using the concept hierarchy.

I tried to prove my intuition, but, unfortunately, Dramatica only has 300 movies analysed, and I was able to find scripts of only 10 of them; not enough data.
However, there are other hierarchical ontologies out there and other narrative structures; could the same intuition be used for political news for example?


text:
Does Yann LeCun consider k-means self-supervised learning?
I was discussing the topic of self-supervised learning with a colleague. After a while we realized we were using different definitions. That's never helpful.
Both of us were introduced to self-supervised learning by reading or listening to Yann LeCun. He is renaming (part of) unsupervised learning to self-supervised learning. For example in this Facebook post.
Probably the definitions of unsupervised and self-supervised learning overlap. But to me the terms are not interchangeable. For example, a prototypical example of age-old unsupervised learning technique is k-means. To me that is unsupervised but not self-supervised learning.
Is Yann LeCun renaming the entire concept unsupervised learning to self-supervised learning? More specifically, is his opinion that we should call clustering and anomaly detection self-supervised learning? And in the limit, does he call k-means supervised-learning?
References are appreciated.

I don't know what LeCun thinks of SSL or how he uses this term, but, yes, SSL has been used to refer to different techniques, and we used to denote some of these as "unsupervised learning", so it's possible that someone calls k-means an SSL technique, given that, in a way, there's some kind of automatic learning signal, but, to be honest, it is really a stretch to call it SSL, as we don't really have or use any kind of label (e.g. this is a cluster) in k-means. It's not the same thing as training e.g. a denoising auto-encoder, where you really have the labels (i.e. original images).

I just read that post that you're linking us to. I don't think he's calling k-means an SSL technique. He's using the term "SSL" in the same way that "SSL" is currently main used for and it's consistent with my description here. So, based just on that Facebook post, I don't think that LeCun implies that k-means is SSL. Not sure where you got this idea from, but it would be nice that you clarify this.

Thanks for the feedback. I changed the question a bit to clarify why I introduced k-means.


text:
Trying to make snake learn different directions
I'm really new to neural networks. I'm trying to make a neural network with genetics algorithms which will make a snake learn to look for the food and avoid hitting his tail.
The thing is that I think that I've done it, but as there's no walls the snake learns to go one direction only without making a 180 turn [GIF HERE].
I've tried to incentivate mutations that make them turn by decreasing the score of the snakes that always takes the same directions, but it don't works. I've only made them dumber, needing more breeds to reach another "smart" linear snake.
I've made a network with 5 inputs:

Food position relative to my position and direction (2 inputs. x and y)
Nearest wall (my tail) if I turn left
Nearest wall (my tail) if I do not turn
Nearest wall (my tail) if I turn right

The output are 3, being the first one turning left, the second do not turn and the third going right. I make the snake go the highest one of the 3 outputs.
I've added 1 hidden layer of 8 neurons (inputs + outputs premise).
The way I calculate the score is:

Each step, 1 point.
Each food eaten, 10 points.
If the snake lasts too much time without eating food, dies.
If hits his tail, dies.

Then I save each time the direction this snake has gone (up, right, down, left) and increment them by one. When the snake dies, I weight the final score by the difference between the lowest and highest values. If the difference is high, they receive a big penalty (down to 0.25 of their score). This way if a snake is pretty much linear gets a high penalty and if a snake does a cool pattern, gets a low penalty.
Also, I keep a record of each time a direction change happens compared to the last direction change, so if a snake keeps going in circles don't gets a high score because of "cool pattern method" for going all 4 directions.
With all this, I don't understand why my best snakes are always the linear ones :-/
I spawn 20 snakes and get the best 4 of each generation when everyone dies.
For generations I use neataptics.js and for neural networks I use synaptics.js. I have a Fiddle here: http://jsfiddle.net/Llorx/gunsct5r/
At line 10 you can see the network definition. At 211 you can see the snake "view" (food position and walls. Where it gets the inputs) and at 164 you can see the score weight calculation depending on steps taken that I mentioned before.
All inputs are normalized from 0 to 1.
I'm sure that I'm doing, not one, but a lot of things pretty bad, as I'm a newbie on this, but some light on this will be really cool.

And that downvote??? Hate those ninja downvoters without explanation, really. That way I'm never going to learn what's the problem and how to improve the question. That's your way to improve this awesome webpage?

After looking at the code in fiddle I started to think how the snake can see backward distance? It would be quite big leap to find out only by guessing to turn left or right when the numbers tells to go around in front.

@mico Well, I add as an input the position of the food in reference of the current position and direction of the snake. What it receives as "left, right, front" view only are the walls, so she don't knows when has a wall behind, but actually knows the exact position of the food relative to itself.

As far as I understand, you don't give the snake rewards for eating fast.  In fact, "Each step, 1 point." means that the slower it finds food, the better. So why would it do turns, considering it is dangerous.

At first I don't added the "1 point per step" method and actually made the reward being higher if the food was eaten fast and the pattern was the same. I ended up with this after a thousand tests. Made it aprox like this: http://jsfiddle.net/gunsct5r/5/ check line 296 now. That's the reason stepsWithoutPoints is reset to 1 instead of 0. To never divide by 0. Actually is a good point but removed it and tested other things as it also don't worked.

I observed the scoring system you used and compared it to the field size and starving rate. The worms either died or did not try enough hard with the prize of 10 of eating, so I added the amount to 1000.
This did still not introduce the 180 degrees turn yet. I had to adjust these settings on Neat:
  mutationRate: 0.7,
  mutationAmount: 6,

which introduced  where the worms optimize their path as diagonal and when forced to change 90 direction, actually a tiny 180 happens on screen.
On same mutationRate and mutationAmount, but 10 points I once observed a horizontal 180, when the food appeared back and diagonal movement was not invented. Also I observed with this last setup a motion of constant 180s flow where consecutive vertical or horizontal 180s formed a two pixel thick line on screen.
These observations do not 100% do the same as your animation, but do instead introduce various ways that are quite near, or can be in certain conditions an initiative to such phenomenom.

White worm introduces two pixel line, red 90 degree turn with miniature 180.
EDIT
I changed the nice curve ratio like this:
     var ratio = 0.75 + ((mindir / maxdir) * 0.25* difstepsratio);

and around generation 200 there lived one worm to be able to move to all directions and make 180 if needed.
Unfortunately it died before I found screen shot buttons from my phone and (s)he happened to become second in the race and thus was left as a unicorn who existed only once.
Still remains mystery whether that mutation could survive and become the main species some day.
Further remarks:
I noticed that when mutation takes place in snake code the same pattern is replicated many times. If snake gets idea of consecutive turn lefts or turn rights when it is long, a suicide with tail is quite often taking place.
I tried to taggle this with a higher score on snake collision than starvation and around generation #180 the dna of suicidal 360 loop was born and remained some generations.
However, this generation did not evolve to the desired intelligent 180 worm I previously had seen. Instead, the suicide was so rewarding that it killed that branch quite soon away.

Thank you for your answer. I really appreciate it. I think that I've not explained myself very well. In your photos the path is still upwards (or sidewards). They do not turn 180 in both directions. They do left-right but not up-down, or they do up-down but not left-right. I want them to learn that if they pass near a piece of food, they can turn around and get it instead of continuing to the edge and appearing from the other side. If I added a scoring system to penalize the ones that do not turn to get the food, I don't know why I never get a mutation where they do.

I think that I added all the necessary inputs for the network recognize it (food position, wall positions and penalize linear movements), but I'm sure that I've done something wrong.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Natural language processing with a continuous dependent variable
I have a large number of observations. Each observation contains:

dependent variable: a scores ranging from 0 - 100
independent variable: a large article

I want to know which words or phrases predict a higher score.
I know that for discrete dependent variables, Naive Bayes can be used for this task. How do I conduct this analysis with a continuous dependent variable?

As described on the below question, there would be no problem with Naive Bayes and continuous and discrete variables together. Or is the dependency something that brakes the rule?
https://stats.stackexchange.com/questions/136577/how-to-deal-with-mixture-of-continuous-and-discrete-features-when-using-naive-ba

Yes, that question has continuous variables as predictors. However, it has a discrete variable which it wishes to predict. I wish to predict a continuous variable.


text:
Human Height estimation using person detection techniques
I have a pedestrian dataset and would like to estimate human height in a video survillance using person detection techniques like YOLO Darknet or SSD (Single Shot Detectors). Would this technique work? Also, the videos that I have are in a constrained environment with good illumination. The idea is to get the coordinates from the bounding box and try to estimate pixel height. After getting the pixel height, some correlation could be estimated between pixel height and real world height. Note that I won't be using camera calibration.

You should revise your question a little bit.or else you transfer it to cross validated community.

I did revise. Also, what is cross validated community?

Human,you should paragraph your question body! Query through the stack exchange sites,you will find it there.

Yes, fellow human. I'll move it. Although I'm not a pro here like you,  I think you should avoid using sarcasm when someone is trying to learn. No offence.

Don't take it personal,we are a community of geniuses who are eager enough to solve world problems.So let's be nice and try to keep our questions clear so as someone out there can benefit enough.Imagine if aliens cyber attack our community knowledge base! They can benefit as well.

Eh, "I won't be using camera calibration." .. Not sure, what you mean.
1st
At first, imagine a sheet of paper laying there on the floor. And first to try to transform the paper sheet (i.e. with some text on it), the picture of the sheet angled by paralaxe to the straight view: Just a transformation by a matrix. It would be highly valuable to have there some floor of square tiles. And once you would estimate/set size of the tiles on the floor, you can then transform the paper sheet.
Then, also some vertical calibration will be needed: Wherever within the scene. A wall of tiles of known shape & size would be perfect. But not really necessary: Any known/premeasured chair would be enough! ..rather two such chairs, in an opposite nooks, for numerical stability.
So yes, that could be treated as "camera calibration".
2nd
Then you can advance to the next level of the image processing:

Once you see feet of the person, on which tile of the floor the person stays,
and with the known vertical direction,

by the hid/covered known background, you can decide their height, pretty precisely, yes.
The "known background"
Just two images should be enough, of the background place: empty non-occupied, and hid/covered. The clear-empty room can be taken either onence, or periodically, or even just a partial image should be enough: just to detect shape of the person-figure.
3rd
And the third level: Really high. The person could stay in some deformed position. By interpolation of the known human skeleton to the shape of the person-figure on the screen, you can assume the curves of the skeleton of the person, so your estimated/measured total height of the person would be immune to the non-strightened angles in joints of their limbs/backbone. So even if the person knows about your observing, the AI would not get fooled. But you would need/find more refencial points on their body, to interconnect the 2D shape to the internal 3D model.
quantity of snaps in general
More snaps of the person, as they walk through the room, would be helpful: Any model, set up by any single image, should be verifiable/applicable to each of the rest of the images of the same person. Again: stability and "back testing", verifications, robustness.
You are not limited to just a single image, you can have as many as your camera is able to make. So use them, to get not only the measured size, but also to level up the "probability" of the measured result, the accuracy.
On the other hand, you can also flip the coin, and to try to guess the maximum info from the minimum snaps of the scene. But it does not seem, this would be your case. Still: Processing 24 fps recording 20 seconds long.. That definitely requires some power/time to process.
Summary
The 3D models are necessary: Of the surrounding space, of the outer shapes of the person-figure, of the internal structure of the measured body.


text:
Algorithms that connect neurons to previous layers as well as next
Are there any algorithms, or any evidence to decide or to suggest it would be better to connect a neuron node in a layer l, in a neural network to particular nodes in the previous layer l-1 of the neural network as well as to particular nodes in the next layer l+1 of the neural network?
This is obviously contrived, but here is a illustration of what I mean.

The thick line with arrow indecates an edge that leads to l-1 layer neuron

Yes. Feedback ANN and Recurrent ANN are types of what you want.

Maybe make your question clearer with a diagram? I was not imagining RNNs as an answer, but ResNets with skip connections. Both RNNs and skip connections are useful, in very different ways, and could be explained in an answer.

Thanks for adding the diagram. This is closer to RNN in design as it involves loops, and thus necessarily some kind of time step parameter in order to decide which value to feed. I'm not sure how much that kind of feedback connection has been studied, and what name(s) it goes under though.

@OmG It looks like recurrent ANNs are the answer the question author was looking for, thanks! Would you like to expand your comment into an answer with some background/explanation?


text:
Detecting license plate using tensorflow
I'm currently working on license plate recognition. My system consist of 2 stage: (1) License Plate region extraction & (2) License Plate region recognition.
I'm doing (1) with Raspberry pi 3 model b. I find license plate candidate first by merging bounding boxes based on their similarity. In this way, i have only 1~7 license plate region proposals. And it took less than .3 seconds.
Now i have to reduce number of region proposal to be around only 1~2 so that i can send these images to server to do job (2). For license plate extraction, I made my own classifier function in tensorflow and the code is below. It gets proposed license plate as input.
First, I resize all license plate to be [120, 60] and converted to gray image. And there are 2 classes: 'plate', 'non_plate'. For non_plate image, i collected various image that might appear in image as background. I have 181 images for 'plate' class and 56 images for 'non_plate' for now, i trained for about 3000 steps so far and current loss is .53 .
When i did prediction on test set,  i encountered problem that for some of plate image, it doesn't recognize license plate which is very obviously license plate image from my eyes. It is okay for me to wrongly recognize non plate image as plate but it is problem if it wrongly recognize plate as non_plate because it will not be sent to server to be fully recognized.
It happens like 10 out of 100 test images and this rate is far worse than i expected. I need help for adressing this problem. Would there be any improvement that i can make?
(1) Is my training set too small to classify between license plate
    and non license plate? Or is number of steps is too small?
(2) Is my graph structure bad? I needed to have small graph
    structure for my raspberry pi to recognize less than 1 second. Could
    you suggest better structure if it is bad?
(3) Is it bad to resize any proposed image to [120, 60] to be used
    as input for graph? I think it loses some information. But isn't
    this close to roi pooling like used in fast rcnn?
 inputs=tf.reshape(features[FEATURE_LABEL],[-1,120 , 60 ,1],name="input_node") #120 x 60 x 1, which is gray

conv1=tf.layers.conv2d(inputs=inputs,
                       filters=3,
                       kernel_size=[3,3],
                       padding='same',
                       activation=tf.nn.leaky_relu
                       )
#conv1 output shape: (batch_size,120,60,3)

pool1=tf.layers.max_pooling2d(inputs=conv1,pool_size=[2,2],strides=2,padding='valid')

#pool1 output shape: (batch_size,60,30,3)

conv2=tf.layers.conv2d(inputs=pool1,filters=6,kernel_size=[1,1],padding='same',activation=tf.nn.leaky_relu)

#conv2 output shape: (batch_size, 60,30,6)

pool2=tf.layers.max_pooling2d(inputs=conv2,pool_size=[2,2],strides=2,padding='valid')

#pool2 output shape: (batch_size, 30,15,6)

conv3=tf.layers.conv2d(inputs=pool2,filters=9,kernel_size=[3,3],padding='same',activation=tf.nn.leaky_relu)

#conv3 output shape: (batch_size, 30,15,9)

pool3=tf.layers.max_pooling2d(inputs=conv3,pool_size=[2,2],strides=2,padding='valid')

#pool3 output shape: (batch_size, 15,7,9)

#dense fully connected layer
pool2_flat=tf.reshape(pool3,[-1,15*7*9]) #flatten pool3 output to feed in dense layer

dense1=tf.layers.dense(inputs=pool2_flat,units=120,activation=tf.nn.relu)

logits=tf.layers.dense(dense1,2) #input for softmax layer

 [training non plate image example]
[]4
[training plate image example. It is region proposed image]

Such a task is getting easier to complete, but there are still difficulties with rotations, skewing, scaling---to name a few issues. Your network has the benefit of simplicity and lightness for the target hardware, but it may suffer under the above conditions.
So 237 images (181+56) may be small for a "generic" approach, depending on how representative and diverse the dataset is. Also, the dataset is unbalanced (the first class has twice as many examples), which causes bias in learning.
There are several ways to expand a base dataset:

Transform images and add them (with the same label, if supervised learning) to the dataset. Many libraries allow to rotate, skew, scale, or even blur images. Be careful as transforming needs be "reasonable", and changing image format over and over creates artifacts that can confuse the machine (e.g. too much JPEG savings on transformed images).
Generate synthetic data. Assuming license plates have a known format, it may be easy to generate images with good fidelity to real plates. This is not always possible, but license plates are formalized, so there should be only a handful patterns (typically plain, light, diplomatic, military vehicles).

Aside the dataset (potential) issue, the graph is fine. However, it may be worth trying different settings. It really depends on how you have ended up with the current graph. Removing pooling layers helps keeping more information, as well as a larger input image. 120x60 looks pretty small, and the comparison to fast-RCNN's RoI pooling layer looks odd, given that RoI comes after the feature maps. So a larger input image could give better results.

Yes, i didn't augment my dataset by putting noise or zooming....But i wanted to know whether detecting license plate by simply training with input images as license plate as a whole, since it contains different charactoers. I will try above methods. Thanks


text:
When to convert data to word embeddings in NLP
When training a network using word embeddings, it is standard to add an embedding layer to first convert the input vector to the embeddings.
However, assuming the embeddings are pre-trained and frozen, there is another option. We could simply preprocess the training data prior to giving it to the model so that it is already converted to the embeddings. This will speed up training, since this conversion need only be performed once, as opposed to on the fly for each epoch.
Thus, the second option seems better. But the first choice seems more common. Assuming the embeddings are pre-trained and frozen, is there a reason I might choose the first option over the second?

If you have to move a lot of data around during training (like retrieving batches from disk/network/what have you), it's much faster to do so as a rank-3 tensor of [batches, documents, indices] than as a rank-4 tensor of [batches, documents, indices, vectors]. In this case, while the embedding is O(1) wherever you put it, it's more efficient to do so as part of the graph.

I very recently came to the same conclusion. There is also another benefit: I can train in larger batches

There are multiple ways to get word embedding from a corpus.

Count Vectorizer: You can use the CountVectorizer() from sklearn.feature_extraction.text and then use the fit_transform() if the corpus has been converted into a list of sentences
TF-IDF Vectorizer: You can use the TfidfVectorizer from sklearn.feature_extraction.text and then again use the fit_transform() on a list of sentences
word2vec: You can make a word2vec model from gensim.models by using  word2vec.Word2vec.

Assuming that the dictionary of the words, that your model comes up with, is a subset of the pretrained embeddings, for example of Google's pretrained word2vec, then it is maybe a better option following these embeddings, if your model can handle that size of dimension.
However, sometimes that would not always be the best solution, taking into account the nature of the problem. For example, if you are trying to use NLP on medical texts that contain rare and special words, then maybe you should use your embedding layer, assuming that you have an adequate data size, or both of them. That is just a thought of mine. For sure, there can be several other use cases which should propose the embedding layer.


text:
Why does the policy network in AlphaZero work?
In AlphaZero, the policy network (or head of the network) maps game states to a distribution of the likelihood of taking each action. This distribution covers all possible actions from that state.
How is such a network possible? The possible actions from each state are vastly different than subsequent states. So, how would each possible action from a given state be represented in the network's output, and what about the network design would stop the network from considering an illegal action?

The output of the policy network is as described in the original paper:

A move in chess may be described in two parts: selecting the piece to move, and then
  selecting among the legal moves for that piece. We represent the policy π(a|s) by a 8 × 8 × 73
  stack of planes encoding a probability distribution over 4,672 possible moves. Each of the 8×8
  positions identifies the square from which to “pick up” a piece. The first 56 planes encode
  possible ‘queen moves’ for any piece: a number of squares [1..7] in which the piece will be
  moved, along one of eight relative compass directions {N, NE, E, SE, S, SW, W, NW}. The
  next 8 planes encode possible knight moves for that piece. The final 9 planes encode possible
  underpromotions for pawn moves or captures in two possible diagonals, to knight, bishop or
  rook respectively. Other pawn moves or captures from the seventh rank are promoted to a
  queen.

So each move selector scores the relative probability of selecting a piece in a given square and moving it in a specific way. For example, there is always one output dedicated to representing picking up the piece in A3 and moving it to A6. This representation includes selecting opponent pieces, selecting empty squares, making knight moves for rooks, making long diagonal moves for pawns. It also includes moves that take pieces off the board or through other blocking pieces.
The typical branching factor in chess is around 35. The policy network described above always calculates discrete probabilities for 4672 moves.
Clearly this can select many non-valid moves, if pieces are not available, or cannot move as suggested. In fact it does this all the time, even when fully trained, as nothing is ever learned about avoiding the non-valid moves during training - they do not receive positive or negative feedback, as there is never any experience gained relating to them. However, the benefit is that this structure is simple and fixed, both useful traits when building a neural network.
The simple work-around is to filter out impossible moves logically, setting their effective probability to zero, and then re-normalise the probabilities for the remaining valid moves. That step involves asking the game engine for what the valid moves are - but that's fine, it's not "cheating".
Whilst it might be possible to either have the agent learn to avoid non-valid moves, or some clever output structure that could only express valid moves, these would both distract from the core goal of learning how to play the game optimally.

Alphazero can play games where pieces move such as chess. How then would such an encoding work?

@chessprogrammer: I have updated the answer with details about the chess representation as used by AlphaZero


text:
Multiclass Ensemble Methods with weak classifiers under 50%
Normally, when using an ensemble method, such as baggin or boosting, in binary classification, there is a reqauirment that each weak classifier have accuracy better than 50%.
In the multiclass claaification setting, this is often infeasible. Is there a way to improve upon multiclass classification with ensembles.
For an example to make this concrete:
Say I have a problem with 1000 classes, and I train 50 models, each with 10% accuracy, which is 100x better than random guessing.
Is there a way to combine these models to form a better classification algorithm?

Can you please put your specific question in the title? "Multiclass Ensemble Methods" is not a question and it's also not specific.

Any classifier that performs slightly better than random guessing is a weak classifier. In the case of a system with N classes, a random guessing actor will have an accuracy of 100/N.
For example, if there are 1000 classes, a random guesser will have an accuracy of 0.1%. Hence any classifier with more than 0.1% accuracy is a weak classifier in this example.


text:
Can One-Hot Vectors be used as Inputs for Recurrent Neural Networks?
When using an RNN to encode a sentence, one normally takes each word, passes it through an embedding layer, and then uses the dense embedding as the input into the RNN.
Lets say instead of using dense embeddings, I used a one-hot representation for each word, and fed that sequence into the RNN. My question is which of these two outcomes is correct:

Due to the way in which an RNN combines inputs, since these vectors are all orthogonal, absolutely nothing can be combined, and the entire setup does not make sense.

The setup does make sense and it will still work, but not be as effective as using a dense embedding.

I know I could run an experiment and see what happens, but this is fundamentally a theoretical question, and I would appreciate if someone could clarify so that I have a better understanding of how RNNs combine inputs. I suspect that the answer to this question would be the same regardless of whether we are discussing a vanilla RNN or an LSTM or  GRU, but if that is not the case, please explain why.
Thank you.

Interesting question - curious to see how others response but I vote for #2 (more time to process 1 hot vectors since you vectors will likely have more dimensions then the 512 or 768 typically found in dense embeddings and will see lower accuracy from your model)


text:
What layers to use in a Neural Network for card game
I am currently writing an engine to play a card game and I would like for an ANN to learn how to play the game. The game is currently playable, and I believe for this game a deep-recurrent-Q-network with a reinforcement learning approach is the way to go.
However, I don't know what type of layers I should use, I found some examples of Atari games solved through ANN, but their layers are CNN (convolutional), which are better for image processing. I don't have an image to feed the NN, only a state composed of a tensor with cards in the player's own hand and cards on the table. And the output of the NN should be a card or the action 'End Turn'.
I'm currently trying to use TensorFlow but I'm open to any library that can work with NN. Any type of help or suggestion would be greatly appreciated!

I encourage you to use some high level framework like Keras to get the work done easily. It uses Tensorflow (or Theano) as backend but it is almost transparent. The type of layer depend on you representation of the features. You should concern not only about the type of layers but also about the entire network architecture. Anyway as a baseline, I'll try with a sequential model of full conected layers combining layers with relu and sigmoid activations.

Hey, thanks a lot for the feedback! I'll check Keras and I'll try to implement that idea and I'll try to reply soon.

The game state consists of the location of all the hidden cards, so you probably need a softmax layer, 52*n, where n is the number of locations.
I'm not very sure that a NN is a good match.

Thanks for the reply! I had a typo saying the "state composed of a tensor with cards in the players hand", I meant in the player's own hand. The last layer could indeed be a softmax activation layer. Once i'm able to test it, I'll reply (i'm still having problems with the rest of the architecture). I tried 2 fully connected layers with relu and sigmoid activation and curiously got a worse result than with a simple weights matrix.

With images, you can use CNN because of the translational invariance. A filter which is good in one area will probably be good in another area, too.
With images, you must use CNN because otherwise, there would be too many weights to train.
With your game, it depends on the representation and the exact rules. Note that Alpha Zero uses a set of 19 x 19 inputs with CNN for playing Go.
In a game like Bridge, where each card has its color and rank, there's a kind of translational invariance. Having Ace and Queen is a bit similar to having King and Jack - in both cases you have a 50% chance of catching the card in between. At the same time, the strengths of AQ and KJ are very different, so a pure CNN would improbably work well.
The more important symmetry is the one among colors. After the auction, there's one or none trump color and all other colors are equivalent. This probably means that the corresponding weights should be the same.
In some card games, many cards are special and there's no symmetry at all. You didn't tell us anything about your game, so it's hard to give a more concrete advice.

(Part 1 Comment) Thanks for your response. Before I learned more about NN I though they would be more generalist. The game is not too complicated, there can be an N number of players, to which all cards in the deck are distributed in the beginning. For example 4 players mean each will start with 13 cards, and the winner is the first to have no cards in hand. To get rid of the cards they will play them in turn order and can choose to play cards or not play. The only rules are  that the players always have to play higher cards than  previously played.

(Part 2 of the Comment) Once no one wants to play a card (or can't), the last player to play a card gets to play any card he wants and start a new cycle.
The game is highly stochastic because the players can start with low level cards and lose the game even if they played better than the opposition, which complicates the use of greedy epsilon exploration.


text:
Transfer learning from model trained in a similar dataset
I am currently working on a defect detection algorithm but I only have a few samples of defects.I googled for defect detection datasets and I found this one:
http://resources.mpi-inf.mpg.de/conferences/dagm/2007/prizes.html
which has a few hundreds of original images of defects.
My idea is:
Imagenet => Defect dataset from internet => Own defect dataset
Step 1. Training a model with ImageNet initialization using the defect dataset  found in the internet (+ non-defect images + augmented data)
Step 2. Using the output model of step 1 (which will be more similar to my own data),do transfer learning using my own defect dataset (defects + non-defects + augmented).
Do you think this a good way to get good results?
Based on:
https://blog.slavv.com/a-gentle-intro-to-transfer-learning-2c0b674375a0
Should defect images consider as low similar with imagenet's images? or similar to model because a both inputs are images? Some webpages said because they both are images, they are similar but some webpages said because these images are too different to the images used to train the imagenet model so I got confused about this.
If I skip step 1, I dont think I get anything good because I have less than 100 images.
Any advise or comment will be appreciated.


text:
Identify unnecessary inputs of NN
Lately I've been wondering.
Is there's a way to locate redundant/unnecessary/misleading inputs by analysis of weights in the first layer?

An interesting way to check for redundancy not only in the first layer but in any layer is to see the weights of you neuron. Let's consider the input layer: The nodes that are connected with the weights that have high values are the most important for your NN performance and in other words your input to these nodes in what mostly makes sense for your task.
If you are in Keras, you can use the get_weights method(). You may have a look here.


text:
Continuous ground truth in supervised (metric) learning?
I am writing my thesis in the field of (deep) metric learning (DML). I am training a network in the fashion of contrastive / triplet Siamese networks to learn similarity and dissimilarity of inputs. In this context, the ground truth is commonly expressed as a binary. Let's take an example based on the similarity of species:

Image A: german shepard (dog)
Image B: siberian husky (dog)
Image C: turkish angora (cat)
Image D: gray wolf (wolf)

Image A and B are similar: same species, same sub-species (canis lupus) -> 1.0 == TRUE
Image A and C are dissimilar: different species (canis lupus vs. felis silvestris) -> 0.0 == FALSE
Image A and D ? same species, but different sub-species -> 0.8

Which metric learning approaches use a continuous ground truth for learning?
I could imagine that there is a lot of research out there using a continuous ground truth in classification settings. For instance to learn that the expression of a face is "almost (60%) happy", or more controversial, an image of a person depicts a "70% attractive person". Also in this fields I would be happy for hints / links.
Remarks:

I don't ask for opinions on whether this makes sense or not.

See Deep Metric Learning Beyond Binary Supervision in CVPR 2019


text:
Maximize a scoring function within the latent space of a generative model
Given a generative model, G, trained on a dataset D. This generative model can be either GAN or Diffusion based. Supposed each sample, x_i, generated by G, can be evaluated by a readily available scoring function, S(x_i).
What are the possible ways to navigate the latent space of G to find generated samples which maximizes S(x_i)?
Here are some of the possible ways I have thought:

Random samplings from G to form a dataset. For each sample, evaluate it with S, then finetune G to maximise S with loss = -S. Keep repeating until G only produces samples with high S values. This is motivated from RLHF but use backprop instead of RL loss.

Fix G, start with a random latent, l_1 to sample x_1 from G. Then slowly find a better x_2 by going into the direction of S, e.g. l_2 = l_1 + alpha * s(x_1) = l_1 + alpha * s( g(l_1)). Is this possible, is it fair to assume P(l, s(1)) is smooth?

Condition the training of G with S, i.e G(X|S(X)). In this way, we can specific any desired score as an input to the generative model.

Are any of these ideas possible? Otherwise, are there better or more simpler approaches?

I doubt that you can condition training on a scoring function, if such scoring is not used during training.
Now, assuming your generator is differentiable, it's already been done the "n-step gradient update" on the latent, via gradient descent:
$$
z_{t+1} = z_{t} + \alpha \nabla_{z_t} D(G(z_t))
$$
given $D$ the discriminator, $G$ the generator and $z$ the latent noise vector (might be missing a $-$ in the gradient update, depending if the discriminator is trained to predict fakes or reals as $1$)
This approach would be sound if we assume that $D$ is a perfect discriminator, which is hardly the case in real life, thus using it as a score function is very likely to lead to some samples that will indeed fool the discriminator, but are visually worse for humans
If $D$ is not a discriminator, but maybe some other non differentiable scoring function, you can use zero-order optimization to solve estimate the gradient $\nabla_{G(z_t)}S(G(z_t))$, or even directly the gradient with respect to $z_t$

Thanks! It makes sense.
However for the conditioning method, if assuming we have access to the training, will this be a better method?

@terenceflow not really, the problem is always that either you have a perfect scoring method, or you will have a bad result... if you check for example AlphaFold, you can consider this the same idea, where they had a model to predict a "good initial guess", and then a very good differentiable model that starting from a good guess, creates a good folding

For the first suggestion, are you suggesting doing gradient descent with objective $-S$? If so (and $S$ is differentiable), that's definitely possible. I would suggest looking at the PULSE paper. They do image superresolution over face images by searching through the latent space of a GAN to find high resolution generations that match the given low-resolution image.
The main thing to keep in mind here is that not all areas in the latent space correspond to realistic generations. You probably want areas that correspond to latents that were likely seen during training (e.g., latents with high probability under the prior). The PULSE paper does this by constraining the search space to a hypersphere centered at the origin. $S$ here would be measuring how much the downscaled image deviates.
This paper does something similar. This paper also has an additional perceptual loss term using a VGG model. $S$ here would be measuring the reconstruction loss using both a simple pixel-wise MSE loss and the perceptual loss.
Finally, an early way of generating images from text-prompts was optimizing the input embeddings for an image generator like StyleGAN using CLIP as a loss function.
I'm not too sure about your second and third suggestion, though. The second sounds a bit like gradient descent -- how do you know what "the direction of $S$" is? The third sounds like Conditional GANs, but I'm not aware of work that conditions on the "quality" of the input.