text:
How should the neural network deal with unexpected inputs?
I recently wrote an application using a deep learning model designed to classify inputs. There are plenty of examples of this using images of irises, cats, and other objects.
If I trained a data model to identify and classify different types of irises and I show it a picture of a cat, is there a way to add in an "unknown" or "not a" classification or would it necessarily have to guess what type of iris the cat most looks like?
Further, I could easily just add another classification with the label "not an iris" and train it using pictures of cats, but then what if I show it a picture of a chair (the list of objects goes on).
Another example would be in natural language processing. If I develop an application that takes the input language and spits out "I think this is Spanish", what if it encounters a language it doesn't recognize?

The answer will depend on your purpose for training the classifier, what kinds of "not a" target you expect it to have as inputs when used. There is not a general answer possible with current technology, but specific answers are possible depending on what you are trying to do. Could you clarify the use case you have in mind - including how you expect the network to be used in production?

I'm writing a paper on some research I did involving source code stylometry. At this point, I have no intent in operationalizing this, but essentially, given a source code, can we identify this as an author we have seen before.

This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i.e., you do not expect the data to come from a distribution different than the distribution from which your training data comes, so there's also the implicit assumption that data comes from the same family of distributions (e.g. only Gaussians) and that all your training examples are independently drawn from the same distribution (specific mean and variance). Of course, this is a big limitation!
A partial solution to your problem is to use a Bayesian neural network (BNN). The idea of a BNN is to associate, rather than a single number, a distribution (usually a Gaussian distribution) with each unit (or neuron) of the neural network. Therefore, for each unit of the network, there are two learnable parameters: the mean and variance of a Gaussian distribution. Consequently, a BNN usually has the double of the number of parameters of a conventional (or non-Bayesian) neural network. However, by learning a distribution for each parameter, you also learn the uncertainty about the potential true value of each unit, based on the available training data.
The forward passes of such a BNN are stochastic, i.e. you sample from each of these Gaussian distributions for every forward pass, so the output of the network is also stochastic (i.e. given the same input example, the output may be different each time).
If your dataset is small, one expects the BNN to have wide Gaussian distributions, i.e. high uncertainty about the true value of the units. So, one expects a BNN to be able to deal with unexpected inputs more robustly. To be more precise, if you train a BNN with a small dataset, the hope is that Gaussian distributions will be wide and thus the outputs of the BNN will be highly variable (i.e. the model is highly uncertain). The more data is gathered, the less uncertain the BNN should be.
This doesn't completely solve your problem, but it should at least mitigate it, i.e. if you provide an unseen example to the BNN, then ideally it should be uncertain about the actual label of that input.
For simplicity, I didn't explain certain details of BNNs, but, at least, this answer gives you a potential solution. Of course, this doesn't exclude the possibility of having an "unknown" class. The approaches are not mutually exclusive. There may also be other solutions, but I am not aware of any.


text:
How should I design the action space of an agent that needs to choose a 2d point and then shoot a cannonball?
I'm building a game environment (see the picture below) where an agent should position the mouse on the screen (see the coordinates on the upper right corner) and then click to shoot a cannonball. If the goal (left) is hit. The agent gets a reward based on the elapsed time between this strike and the last one. If three shots are missed, the game is done and the environment will reset.

The env is done so far. But now I wonder what the action space should look like. How can I make the agent choose some x and y coordinates? And how can I combine this with a "shoot" action?

you can learn the mean $\mu$ of normal distribution for $x$ coordinate position and for $y$ coordinate position. Then the action would be sampled coordinates from the normal distribution. You can also learn standard deviation $\sigma$ or it can be fixed. You don't need to model shoot action, agent can shoot immediately  to given coordinates. You would need to use policy gradient/actor critic method for this

You could make the actions

move up
move down
move left
move right
shoot

Then you can declare the speed of this movement. If you want to go more in depth you can do

Accelerate up
Accelerate down
Accelerate left
Accelerate right
Shoot

This will give control of the speed to the neural network, but is harder to train, and you should give the speed of the mouse as an input.

I also thought about that. But since the score is based on the time between two hits I see some problems. Because the main goal is that the agent gets as fast as possible. But when can can only move x much pixels it will lack precision and speed I think...

I don't think it will lack precision. And if you use the acceleration the speed purely depends on how much you accelerate it on every step, and how many frames you wait to make the next step

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Which Rosenblatt's paper describes Rosenblatt's perceptron training algorithm?
I struggle to find Rosenblatt's perceptron training algorithm in any of his publications from 1957 - 1961, namely:

Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
The perceptron: A probabilistic model for information storage and organization in the brain
The Perceptron — A Perceiving and Recognizing Automaton

Does anyone know where to find the original learning formula?

The paper (or report) that formally introduced the perceptron is The Perceptron — A Perceiving and Recognizing Automaton (1957) by Frank Rosenblatt. If you read the first page of this paper, you can immediately understand that's the case. In particular, at some point (page 2, which corresponds to page 5 of the pdf), he writes

Recent theoretical studies by this writer indicate that it should be feasible to construct an electronic or electromechanical system which will learn to recognize similarities or identities between patterns of optical, electrical, or tonal information, in a manner which may be closely analogous to the perceptual processes of a biological brain. The proposed system depends on probabilistic rather than deterministic principles for its operation, and gains its reliability from the properties of statistical measurements obtained from large populations of elements. A system which operates according to these principles will be called a perceptron.

See also Appendix I (page 19, which corresponds to page 22 of the pdf).
The paper The perceptron: A probabilistic model for information storage and organization in the brain (1958) by F. Rosenblatt is apparently an updated and nicer version of the original report.
A more accessible (although not the most intuitive) description of the perceptron model and its learning algorithms can be found in the famous book Perceptrons: An Introduction to Computational Geometry (expanded edition, third printing, 1988) by Minsky and Papert (from page 161 onwards).

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Why is the validation performance better than the training performance?
I am training a classifier to identify 24 hand signs of American Sign Language. I created a custom dataset by recording videos in different backgrounds for each of the signs and later converted the videos into images. Each sign has 3000 images, that were randomly selected to generate a training dataset with 2400 images/sign and validation dataset with the remaining 600 images/sign.

Total number of images in entire dataset: 3000 * 24 = 72000
Training dataset: 2400 * 24 = 57600
Validation dataset: 600 * 24 = 14400
Image dimension (Width x Height): 1280 x 720 pixels

The CNN architecture used for training
model = Sequential([
    Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Conv2D(32, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Flatten(),
    Dense(128, activation='relu'),
    Dropout(0.25),

    Dense(NUM_CLASSES, activation='softmax')
])

Training parameters:
IMG_HEIGHT = 224
IMG_WIDTH = 224
BATCH_SIZE = 32
NUM_CLASSES = 24
train_datagen = ImageDataGenerator(rescale = 1./255,
                                   width_shift_range=0.1,
                                   height_shift_range=0.1,
                                   zoom_range=0.1,
                                   fill_mode='constant')
EPOCHS = 20
STEPS_PER_EPOCH = TRAIN_TOTAL // BATCH_SIZE
VALIDATION_STEPS = VALIDATION_TOTAL // BATCH_SIZE

callbacks_list = [
    tf.keras.callbacks.EarlyStopping(monitor = 'accuracy',
                                     min_delta = 0.005,
                                     patience = 3),
    tf.keras.callbacks.ModelCheckpoint(filepath = 'D:\\Models\\HSRS_ThesisDataset_5Mar_1330.h5',
                                       monitor= 'val_loss',
                                       save_best_only = True)
]

optimizer = 'adam'

The model accuracy and model loss graph is shown in the figure below:

The results obtained at the end of the training are

Train acc: 0.8000121
Val acc: 0.914441

I read this article explaining why the validation loss is lower than the training loss I want to know:

Is it because of the smaller dataset and random shuffling of the images?
Is there any way to improve the condition without changing the dataset?
Will this have a very detrimental effect on the model performance in real test cases? If not, can I just focus on improving the training accuracy of the overall model?

That article you're mentioning tells you that dropout can decrease the training performance and you are using dropout. Have you tried not to use dropout during training and see if something changes?

Assuming you pass through the entire validation dataset, this can't be due to shuffling since you still compute the loss/accuracy over the entire dataset, so order does not really matter here. It is more likely that you have a significantly smaller or less representative validation dataset, e.g., distribution of the validation dataset can be skewed towards classes where your model performs better.
What do you mean exactly by improving the situation? Having a better validation accuracy is not necessarily bad. In any case, if you decrease the effect of regularization, e.g., lowering weight decay, training accuracy might go up but your model might generalize worse, i.e.,  you might get a lower validation accuracy.
No, the goal of training is never to maximize training accuracy. You can trivially do so by just memorizing the training dataset. In short, the goal of training is to get good generalization and as long as you get a satisfactory validation accuracy, it is likely that this has happened to some degree (assuming you have a good validation dataset of course).

Thank you for the reply. What does it mean to have good representative validation dataset? Just to understand it, my images were taken on plain and complex backgrounds, so both these types of images are present in the training and validation dataset. Should I segregate them like - Training data with plain bakgrounds and Validation dataset with more complex bakgrounds?

Although I see higher accuracies, it performs significantly poor on several test images I checked. So, it leads to a confusion to how much should I train?

Briefly, the format of the images and the distribution of the classes in your validation set should ideally be the same as your training dataset. You can use the 'early stopping' criteria to decide when to stop training. That is, just keep training as long as your validation loss decreases. Then, use the model snapshotted at lowest validation loss to do inference on test data. Hope that helps.

Regarding the equal distribution of data for each class, I do not think it is a problem since I have segregated them in different folders. So each class has 2400 training images, 600 validation images. What do you mean by format? The resolutions of all the images are the same. Thanks for the tip to use 'val_loss' as a metric in earlystopping.

Validation dataset: 600 * 24 = 14400

Means that you are augmenting the validation set, right? For an experiment, you can do that and it might take validation accuracy more than train accuracy?
The idea of augmentation in only valid for the training set and you should not change the validation set or test set.
You can try without the augmentation in the validation set and see the result.

No, I have not augmented the validation dataset. I have 600 (images/hand sign) x 24 (24 hand signs) = 14400 validation dataset images

authors: דניאל קלעויחו


text:
How can I train a neural network to describe the characteristics of a picture?
I have collected a set of pictures of people with a text explaining the characteristics of the person on the picture, for example, "Big nose" or "Curly hair".
I want to train some type of model that takes in any picture and returns a description of the picture in terms of characteristics.
However, I have a hard time figuring out how to do this. It is not like labeling "dog" or "apple" because then I can create a set of training data and then evaluate its performance, now I can not. If so I would probably have used a CNN and probably also VGG-16 to help me out.
I only have two ML courses under my belt and have never really encountered a problem like this before. Can someone help me to get in the right direction?
As of now, I have a data set of 13000 labeled images I am very confident it is labeled well. I do not know of any pre-trained datasets that could be of help in this instance, but if you know of one it might help.
Worth noting is that every label is or should at least be unique. If for example there exist two pictures with the same label of "Big nose" it is purely coincidental.

How large is your collection of pictures and captions? This kind of task typically requires a lot of data.

Ok, right now I have about 13000 labeled pictures, I guess that will not be enough. Edit: If I manage to collect enough, then how should I tackle the problem? Even if I now will be unable to do it, I still think it would be very fun to know.

I'm not experienced enough with the approach to be able to tell you for certain. It also doesn't prevent someone showing you an answer. It may help to have this in detail in the question though, because there might be ways to use a pre-trained network and fine tune it for your case. Also, your example targets are simpler than full image captions, so it is possible that your dataset size is enough

No need to signal your edits - if someone is interrested they can click the "edited X ago" link to see your revisions. I have tidied that up for you. The last detail you added does make a big difference to workable approaches because it will be very difficult for the model to learn with only one example of each label, thanks for adding it.

Sorry, will remember that. Yes I felt that it was not clear enough. I tried to take it into account regarding the "dog" and "apple" with that I can not create a training and evaluation set. So yeah, I should have made it more clear, it may not even be solvable because of this.

I have little experience with ConvNets but what I (think to) know is that the neural network architecture should be in balance with the data you are training it on. I read that VGG-16 is trained with 15 million high res images. 13.000 is < 1% of that. So I would say the number of filters in convolution can be less, and likely the number of layers as well. Besides that I think you need more than 1 network trained independently. 1 for each type of hair, 1 for each type of nose, etc. As one could have curly hair, and a big nose, for instance. Fruit cannot be an apple and peer at the same time

I think the title of the question is of. I think it should state you want to classify a multitude of characteristics, instead of comparing 1 image to another.

@Salviati Hi and welcome to AI SE! I've edited your post to make it clearer. Can you please confirm that the current post is consistent with your original question?

I have had a few exams and been out of the sphere but yes, I approve the updated @nbro, thank you!

The term you are looking for is multi-label classification, i.e. where you are making more than one classification on each image (one for each label). Most examples you'll find online are in the NLP domain but it is just as easy with CNNs since it's essentially defined by the structure of the output layer and the loss function used. It's not as complicated as it might sound if you are already familiar with CNNs.
The output layer of a neural network (for 3 or more classes) has as many units as there are targets. The network learns to associate each of those units with a corresponding class. A multi-class classifier normally applies a softmax activation function to the raw unit output, which yields a probability vector. To get the final classification, the max() of the probability vector is taken (the most probable class). The output would look like this:
                 Cat    Bird   Plane   Superman  Ball   Dog
Raw output:      -1     2      3       6         -1     -1
Softmax:         0.001  0.017  0.046   0.934     0.001  0.001
Classification:  0      0      0       1         0      0

Multi-label classification typically uses a sigmoid activation function since the probabilities of a label occuring can be treated independently. The classification is then determined by the probability (>=0.5 for True). For your problem, this output could look like:
                 Big nose  Long hair  Curly hair  Superman  Big ears  Sharp Jawline
Raw output:      -1        -2         3           6         -1        10
Sigmoid:         0.269     0.119      0.953       0.998     0.269     1.000
Classification:  0         0          1           1         0         1

The binary crossentropy loss function is normally used for a multi-label classifier since a n-label problem is essentially splitting up a multi-class classification problem into n binary classification problems.
Since all you need to do to get from a multi-class classifier to a multi-label classifier is change the output layer, its very easy to do with pre-trained networks. If you get the pre-trained model from Keras its as simple as including include_top=False when downloading the model and then adding the correct output layer.
With 13000 images, I would recommend using Keras' ImageDataGenerator class with the flow_from_dataframe method. This allows you to use a simple pandas dataframe to label and feed in all your images. The dataframe would look like this:
Filename  Big nose  Long hair  Curly hair  Superman  Big ears  Sharp Jawline
0001.JPG  0         0          1           1         0         1
0002.JPG  1         0          1           0         1         1
   .      .         .          .           .         .         .

flow_from_dataframe's class_mode parameter can be set to raw or multi_output along with x_col to 'Filename' and y_col to ['Big nose', 'Long hair', 'Curly hair', 'Superman', 'Big ears', 'Sharp Jawline'] (in this example). Check out the documentation for more details.
The amount of data you need for each label depends on many factors and is essentially impossible to know without trying. 13000 sounds like a good start but it also depends on how many labels you have and how evenly distributed they are between the labels. A decent guide (one of many) on how to set up a multi-label classifier and how to implement it with Keras can be found here. It also covers imbalances on label frequency and is well worth a read. I'd highly recommend that you become as intimately familiar with your dataset as possible before you start tuning your neural network architecture.

You can try image captioning. You can train a CNN model for image, and then, on top of that, provide the model embedding to another LSTM model to learn the encoded characteristics. You can directly use the pre-trained VGG-16 model and use the second last layer to create your image embeddings.
Show and Tell: A Neural Image Caption Generator is a really nice paper to start with. There is an implementation of it in TensorFlow: https://www.tensorflow.org/tutorials/text/image_captioning. The paper focuses on generating caption, but you can provide your 'characteristics' to LSTM, so that it can learn it for each image.

You can use image captioning. Look at the article Captioning Images with CNN and RNN, using PyTorch. The idea is very profound. The model encodes the image to high dimensional space and then passes it through LSTM cells and LSTM cells produce linguistic output.
See also Image captioning with visual attention.

Im looking for something beyond just classification.  I would like to identify the component parts of a flat file jpg facebook ad (logo, product, headline, punchline, call to action, background) and have it return a json describing the parts, x,y cooridnates, the polygon shape it is, color and properties. Do you think this is relevant here or any other suggestions?

I would do as suggested in the comments. First select an encoding scheme. I think what is called a difference hash would work well for this application. Code for that is shown below. Now take your data set of images and run them through the encoder and save the result in a database. The database would contain the "labeling" text and the encoder result. Now for a new image you are trying to label, input the image into the encoder. Take the encoder result and compare it to the encoded values in the database. Search through the encoded values in the database and find the closest match. You can then use a "threshold" value to determine if you want to give a specific label for the image or if the distance is above the threshold declare there is no matching label. You can determine the best "threshold" value by running you data set images with the known labels and iterate the threshold level and select the threshold with the least errors. I would use something like a 56 or a 128 length hash.
import cv2
import os
# f_path is the full path to the image file, hash length is an integer specifies length of the hash
def get_hash(f_path, hash_length):
    r_str=''
    img=cv2.imread(f_path,0)        # read image as gray scale image
    img = cv2.resize(img, (hash_length+1, 1), interpolation = cv2.INTER_AREA)
    # now compare adjacent horizontal values in a row if pixel to the left>pixel toright result=1 else 0
    for col in range (0,hash_length):
        if(img[0][col]>img[0][col+1]):
            value=str(1)
        else:
            value=str(0)
        r_str=r_str + value
    number=0
    power_of_two=1
    for char in r_str:
        number = number + int(char) * power_of_two
        power_of_two=2 * power_of_two
    return ( r_str, number)
# example on an image of a bird
f_path=r'c:\Temp\birds\test\robin\1.jpg'
hash=get_hash ( f_path, 16) # 16 length hash on a bird image
print (' hash string ', hash[0], '   hash number ', hash[1])

> results is
 hash string  1111111100000000    hash number  255

From what you wrote, the problem sounds a bit like face recognition, where a camera takes a picture of your face and compares it with a bunch of pictures in a database, for example, one for each employee if its at a company's main gate. If you look "similar" to one of the pictures in the database, the door opens and your ID/Name is displayed on a terminal.
This kind of system generates an encoding for each picture and evaluates the distance between your encoded picture and the encoding of each picture in the database. If this is at most some minimum value, it's considered a match.
So, what you could do is figure out some way to encode your pictures (say sum the pixel values for a very simple example, ideally you would use some sort of vector here because distances make sense with vectors) and store this encoding together with the label of the picture.
Once your database is complete (i.e. you have a bunch of pictures saved as a pair of [encoding, label]), you can "scan" each new picture, calculate its encoding (using the same algorithm that calculated your database encodings) and find the one entry in your database which minimizes the "encoding-distance".
If this sounds like a way to solve your problem, you need to come up with a proper encoding (like "run my images through a CNN and save the output of my last fully connected layer") and apply this to all the images you want to use as "training data", before "testing" it on some of the leftover images.


text:
Understanding CNN in a few sentences
I don't know if this is the right place to ask this question. If it is not, please tell me and I remove it.
I've just started to learn CNN and I'm trying to understand what they do and how they do it.
I have written some sentences to describe it:

Let's say that CNN is a mathematical function that adjusts its values based on the result obtained and the desired output.

The values are the filters of the convolutional layers (in other
types of networks would be the weights).
To adjust these values there is a backpropagation method as in all networks.

The result obtained is an image of the same size as the original.
In this image you can see the delimited area.
The goal of the network is to learn these filters.
The overfitting may be because the network has learned where the pixels you are looking for are located.
The filters have as input a pixel of the image and return a 1 or a 0.

My doubt is:
In your own words, Have I forgotten something?
NOTE:
This is only one question. The six points above are affirmative sentences, not questions.
There is only one question mark, and it is on my question.
I have to clarify this because someone has closed my previous question because she/he thinks there were more than one question.

You missed my point. Although you ask only one question, your post is too broad. I could ask the question: "Is this info in this article of 100 pages correct?". That would be one question, but too broad. Do you understand now? As you can see from the given existing answer, your question "Have I forgotten something?" could not be answered with a "yes" or "no".

To address your description point by point,

There are many types of CNN architectures. You appear to be describing a fully-convolutional neural network built only from convolutional layers which is a very specialized type of network (typically used for segmentation tasks).

Most networks will include other layer types which have trainable parameters too, i.e. batch normalization layers or densely connected layers.
Yes, the training process operates on the same concept of error backpropagation.

If the model is performing a segmentation task (or any task which requires a 1-to-1 mapping of individually classified pixels), then yes its typically the same size.
The raw output of a fully-CNN is typically a probability map where each pixel is the probability between 0 and 1 of it being a member of the positive class. This is normally thresholded at 0.5 to obtain a binary mask which delimits the area of interest.
In essence, yes. But it may be more accurate to say that the goal is to minimise the loss function (reduce the error) which is done by learning optimal filter weights.
Overfitting is not about pixel location. In fact, CNNs are translationally invariant which means that the location of the feature has no impact on the output. There are some types of CNNs which take feature location into account but thats still not standard. Overfitting generally happens when a filter is an exact fit to a pixel patch, for example, the filter is no longer just looking for vertical lines but the exact pixel values of a section of vertical line in a specific image.
No, each convolution of a 3x3 filter takes 3x3 pixels and it is convolved over the whole image which means that the filter output will likely be around the same size of the image depending on stride length, padding and dilation. Have a look at one of my other answers for more details.

The 6th point is wrong. Filters do linear combination with group of pixels (depends on filter size) and move over the image and continue to do linear combination operation until it finishes the whole image.
Please look at 2D convolution in CNN
I would explain CNN following way:
CNN is a kind of neural network that generally contains many convolutional layers and one or two fully connected layers. The purpose of the convolutional layers is feature extraction. Each of the convolutional layers contains many kernels. The purpose of the kernel is to extract different types of features. E. g. Edge, color, shape, and so on.
Finally, fully connected layers at the end of the network decide the output based on the features that were extracted by convolutional layers.
The advantage of CNN is that we can learn feature extraction kernels based training images.
If you look earlier days of image processing. You can
see the image processing community used convolution for feature extraction all the time. They were just using selective kernel for feature extraction. E. g. Sobel operator for edge detection.

Sorry, I gave the link for convolution that is used among signal processing community. Although, it’s it’s same thing if you just skip mirroring the kernel. Now, I updated the link to another page. I hope the new one will help you.


text:
How can the policy iteration algorithm be model-free if it uses the transition probabilities?
I'm actually trying to understand the policy iteration in the context of RL. I read an article presenting it and, at some point, a pseudo-code of the algorithm is given :
What I can't understand is this line :

From what I understand, policy iteration is a model-free algorithm, which means that it doesn't need to know the environment's dynamics. But, in this line, we need $p(s',r \mid s, \pi(s))$ (which in my understanding is the transition function of the MDP that gave us the probability of landing in the state $s'$ knowing previous $s$ state and the action taken) to compute $V(s)$. So I don't understand how we can compute $V(s)$ with the quantity $p(s',r \mid s, \pi(s))$ since it is a parameter of the environment.

Everything you say in your post is correct, apart from the wrong assumption that policy iteration is model-free. PI is a model-based algorithm because of the reasons you're mentioning.
See my answer to the question What's the difference between model-free and model-based reinforcement learning?.

The Policy Iteration algorithm (given in the question) is model-based.
However, note that there exist methods that fall into the Generalized Policy Iteration category, such as SARSA, which are model-free.

From what I understand, policy iteration is a model-free algorithm

Maybe this was referring to generalized policy iteration methods.

(Answer based on comments from @Neil Slater.)

Comments are not for extended discussion; this conversation has been moved to chat.


text:
What are some well-known problems where neural networks don't do very well?
Background: It's well-known that neural networks offer great performance across a large number of tasks, and this is largely a consequence of their universal approximation capabilities.  However, in this post I'm curious about the opposite:
Question: Namely, what are some well-known cases, problems or real-world applications where neural networks don't do very well?

Specification:
I'm looking for specific regression tasks (with accessible data-sets) where neural networks are not the state-of-the-art.  The regression task should be "naturally suitable", so no sequential or time-dependent data (in which case an RNN or reservoir computer would be more natural).

Here is a related question, but it's more theoretical.

Here's a snippet from an article by Gary Marcus

In particular, they showed that standard deep learning nets often fall
  apart when confronted with common stimuli rotated in three dimensional
  space into unusual positions, like the top right corner of this
  figure, in which a schoolbus is mistaken for a snowplow:

.
  .
  .
Mistaking an overturned schoolbus is
  not just a mistake, it’s a revealing mistake: it that shows not only
  that deep learning systems can get confused, but they are challenged
  in making a fundamental distinction known to all philosophers: the
  distinction between features that are merely contingent associations
  (snow is often present when there are snowplows, but not necessary)
  and features that are inherent properties of the category itself
  (snowplows ought other things being equal have plows, unless eg they
  have been dismantled). We’d already seen similar examples with
  contrived stimuli, like Anish Athalye’s carefully designed, 3-d
  printed foam covered dimensional baseball that was mistaken for an
  espresso

Alcorn’s results — some from real photos from the natural
  world — should have pushed worry about this sort of anomaly to the top
  of the stack.

Please note that the opinions of the author are his alone and I do not necessarily share all of them with him.
Edit: Some more fun stuff
1) DeepMind's neural network that could play Breakout and Starcraft
saw a dramatic dip in performance when the paddle was moved up by a few pixels.
See: General Game Playing With Schema Networks
While in the latter, it performed well with one race of the character but not on a different map and with different characters.
Source
2)

AlphaZero searches just 80,000 positions per second in chess and
  40,000 in shogi, compared to 70 million for Stockfish and 35 million
  for elmo.

What the team at Deepmind did was to build a very good search algorithm. A search algorithm that includes the capability to remember facets of previous searches to apply better results to new searches. This is very clever; it undoubtedly has immense value in many areas, but it cannot be considered general intelligence.
See: AlphaZero: How Intuition Demolished Logic (Medium)

Comments are not for extended discussion; this conversation has been moved to chat.

Just an FYI: this link seems to have moved, or requires a login: https://stream.nyu.edu/channel/CS%2BFaculty%2BSearch%2B2019/111128891

In theory, most neural networks can approximate any continuous function on compact subsets of $\mathbb{R}^n$, provided that the activation functions satisfy certain mild conditions. This is known as the universal approximation theorem (UAT), but that should not be called  universal, given that there are a lot more discontinuous functions than continuous ones, although certain discontinuous functions can be approximated by continuous ones. The UAT shows the theoretical powerfulness of neural networks and their purpose. They represent and approximate functions. If you want to know more about the details of the UAT, for different neural network architectures, see this answer.
However, in practice, neural networks trained with gradient descent and backpropagation face several issues and challenges, some of which are due to the training procedure and not just the architecture of the neural network or available data.
For example, it is well known that neural networks are prone to catastrophic forgetting (or interference), which means that they aren't particularly suited for incremental learning tasks, although some more sophisticated incremental learning algorithms based on neural networks have already been developed.
Neural networks can also be sensitive to their inputs, i.e. a small change in the inputs can drastically change the output (or answer) of the neural network. This is partially due to the fact that they learn a function that isn't really the function you expect them to learn. So, a system based on such a neural network can potentially be hacked or fooled, so they are probably not well suited for safety-critical applications. This issue is related to the low interpretability and explainability of neural networks, i.e. they are often denoted as black-box models.
Bayesian neural networks (BNNs) can potentially mitigate these problems, but they are unlikely to be the ultimate or complete solution. Bayesian neural networks maintain a distribution for each of the units (or neurons), rather than a point estimate. In principle, this can provide more uncertainty guarantees, but, in practice, this is not yet the case.
Furthermore, neural networks often require a lot of data in order to approximate the desired function accurately, so in cases where data is scarce neural networks may not be appropriate. Moreover, the training of neural networks (especially, deep architectures) also requires a lot of computational resources. Inference can also be sometimes problematic, when you need real-time predictions, as it can also be expensive.
To conclude, neural networks are just function approximators, i.e. they approximate a specific function (or set of functions, in the case of Bayesian neural networks), given a specific configuration of the parameters. They can't do more than that. They cannot magically do something that they have not been trained to do, and it is usually the case that you don't really know the specific function the neural network is representing (hence the expression black-box model), apart from knowing your training dataset, which can also contain spurious information, among other issues.

Comments are not for extended discussion; this conversation has been moved to chat.

In our deep learning lecture, we discussed the following example (from Unmasking Clever Hans predictors and assessing what machines really learn (2019) by Lapuschkin et al.).

Here the neural network learned a wrong way to identify a picture, i.E by identifying the wrong "relevant components". In the sensitivity maps next to the pictures, we can see that the watermark was used to identify if there is a horse present in the picture. If we remove the watermark, the classification is no longer made. Even more worryingly, if we add the tag to a completely different picture, it gets identified as a horse!

This is similar to the famous tank story.

This is more in the direction of 'what kind of problems can be solved by neural networks'. In order to train a neural network you need a large set of training data which is labelled with correct/ incorrect for the question you are interested in. So for example 'identify all pictures that have a cat on them' is very suitable for neural networks. On the other hand 'summarize the story of this toddler picture book' is very hard. Although a human can easily decide whether a given summary is any good or not it would be very difficult to build a suitable set of training data for this kind of problem. So if you can't build a large training data set with correct answers, you can't train a neural network to solve the problem.
The answer of Anshuman Kumar is also an instance of that, also a potentially solvable one. The neural network that misidentified upside-down school buses presumably had very few if any upside-down school buses in its training data. Put them into the training data and the neural network will identify these as well. This is still a flaw in neural networks, a human can correctly identify an upside-down school bus the first time they see one if they know what school busses look like.

Good point.  Also using a feature map which rotates and stretches immages to create more "artificial" immage samples should bypass Anshuman's problem

@AIM_BLB you'd need a feature map that can do so in 3D. So you pretty much need the whole pipeline the brain does: edge detection -> depth map -> segmentation -> 3D feature map -> classifier search (with each stage feeding back to the edge detection (!) stage and all later ones). And that's without any depth cues, meaning that the depth map can only take input from the classifier and the segmentation map.

I don't know if it might be of use, but many areas of NLP are still hard to tackle, and even if deep models achieve the state of the art results, they usually beat baseline shallow models by very few percentage points.
One example that I've had the opportunity to work on is stance classification 1. In many datasets, the best F score achievable is around 70%.
Even though it's hard to compare results since in NLP many datasets are really small and domain-specific (especially for stance detection and similar SemEval tasks), many times SVM, conditional random fields, sometimes even Naive Bayes models are able to perform almost as good as CNN or RNN.  Other tasks for which this holds are argumentation mining or claim detection.
See e.g. the paper TakeLab at SemEval-2016 Task 6: Stance Classification in Tweets Using a Genetic Algorithm Based Ensemble (2016) by Martin Tutek et al.

Thanks for this answer.  I'm interested in this subject from the standpoint of (satisficing)[https://en.wikipedia.org/wiki/Satisficing]!  (i.e. if time or space is sufficiently restricted, the "less optimal" method may prevail.)

My 50cents:
NP_(complexity) - is still hard to solve, even with NeuralNets.

In computational complexity theory, NP (nondeterministic polynomial
  time) is a complexity class used to classify decision problems. NP is
  the set of decision problems for which the problem instances, where
  the answer is "yes", have proofs verifiable in polynomial time by a
  deterministic Turing machine.

The easiest example, to imagine what is speech about, it is cryptography's Integer_factorization, which is basement of RSA cryptosystem.
For example, we have two simple numbers:

12123123123123123123123.....45456
23412421341234124124124.....11112

NeuralNetwork shall answer us exactly digit to digit both this numbers, when we will show it only multiplication of this two numbers... This is not guessing about school bus. The field of numbers much more bigger than number of words in all languages on whole Earth. Imagine, that there are billions of billion different school buses, billions of billions of different fire-hydrants and billions of such classes, and NN shall answer exactly - what is on the picture - no way. The chance to guess is so little...

A checkerboard with missing squares is impossible for a neural network to learn the missing color. The more it learns on training data, the worse it does on test data.
See e.g. this article The Unlearnable Checkerboard Pattern (which, unfortunately, is not freely accessible). In any case, it should be easy to try out yourself that this task is difficult.

Impossible is not the right word here; I'm pretty sure a neural network specially trained for identifying checkerboards will have no problem, but we'll have to see. Impossible is just too strong a word.

From my experience in industry, a lot of data science (operating on customer information, stored in a database) is still dominated by decision trees and even SVMs. Although neural networks have seen incredible performance on "unstructured" data, like images and text, there still do not appear to be great results extending to structured, tabular data (yet).
At my old company (loyalty marketing with 10 million+ members) there was a saying, "You can try any model you like, but you must try XGBoost". And let's just say that I did try comparing it to a neural network, and ultimately I did go with XGBoost ;)

This sounds very promising.  So something like Boston housing would be a toy example data-set of this type?

Yes exactly! Same with the classic Adult dataset: https://archive.ics.uci.edu/ml/datasets/adult

So in that case XGBoost is the best?

Well it's usually difficult to say in advance, and especially before you do tinkering like hyperparameter sweeps, regularization etc. But in general, XGBoost gives you good results right out of the box, with the added bonus that it is also somewhat interpretable (such as giving you feature importances), allowing you to explain things in a business setting nicely

Neural networks seem to have a great deal of difficulty handling adversarial input, i.e., inputs with certain changes (often imperceptible or nearly imperceptible by humans) designed by an attacker to fool them.
This is not the same thing as just being highly sensitive to certain changes in inputs. Robustness against wrong answers in that case can be increased by reducing the probability of such inputs. (If only one in 10^15 possible images causes a problem, it's not much of a problem.) However, in the adversarial case reducing the space of problematic images doesn't reduce the probability of getting one because the images are specifically selected by the attacker.
One of the more famous papers in this area is "Synthesizing Robust Adversarial Examples", which produced not only examples where a few modified pixels or other invisible-to-humans modifications to a picture fooled a neural network-based image classifier, but also perhaps the first examples of 3D objects designed to fool similar classifiers and successfully doing so (from every angle!).
(Those familiar with IT securitity will no doubt recognise this as a familiar asymmetry: roughly, a defender must defend against all attacks launched against a system, but an attacker need find only one working attack.)
In "A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance", Adi Shamir et al. propose a mathematical framework for analyzing the problem based on Hamming distances that, while currently a less practical attack than the MIT/Lab6 one, has some pretty disturbing theoretical implications, including that current approaches to preventing these attacks may be, in the end, ineffective. For example, he points out that blurring and similar techniques that have been used to try to defend against adversarial attacks can be treated mathematically as simply another layer added on top of the existing neural network, requiring no changes to the attack strategy.
(I attended a talk by Shamir a few months ago that was much easier going than the paper, but unfortunately I can't find a video of that or a similar talk on-line; if anybody knows of one please feel free to edit this answer to add a link!)
There's obviously still an enormous amount of research to be done in this area, but it seems possible that neural networks alone are not capable of defense against this class of attack, and other techniques will have to be employed in addition to make neural networks robust against it.

The attacks aren't quite the same if you apply random blurring. In principle adding random noise should provide sufficient defense, if your network on average handles noise well. I have seen attempts to use lipschitz like constraints but for deep networks the bounds are useless. It's kind of circular in a way - we want the network to function like humans do but there isn't a clear norm for how humans judge similarity between images.

Large scale route optimization problems.
The is progress made in using Deep Reinforcement learning to solve vehicle routing problems (VRP), for example in this paper: https://arxiv.org/abs/1802.04240v2.
However, for large scale problems and overall heuristic methods, like the ones provided by Google OR tools are much easier to use.

You don't talk about neural networks in your answer, but only about deep RL. Maybe you can clarify how neural networks are related to your linked paper and the "large scale route optimization problems".

In the case of convolutional neural networks, the features may be extracted but without taking into account their relative positions (see the concept of translation invariance)
For example, you could have two eyes, a nose and a mouth be in different locations in an image and still have the image be classified as a face.
Operations like max-pooling may also have a negative impact on retaining position information.

I have edited your answer to make it clearer. Please, can you provide a research work that proves that the extraction of features, in a CNN, can have a negative impact such that a collection of two eyes, a nose and a mouth, although not in the appropriate relative position, is classified as a face?


text:
Medical diagnosis systems based on artificial neural networks
Are there any medical diagnosis systems that are already used somewhere that are based on artificial neural networks?

Great and salient question.  My general sense is that NNs are getting very good at this, and I wouldn't be surprised if this is an area where automata exceed humans, in all but exceptional cases.  Welcome to SE:AI!

Yes, there are many, actually. A Google search turned this paper Artificial Neural Networks in Medical Diagnosis (2011) by Al-Shayea up.
Not only are they used in disease diagnosis, but even with things like prescribing medicines. In fact, the top project for a hackathon at my school analysed thousands of research articles, and took a patient's medication history as input, to best recommend them specific medicines. Check it out.


text:
What is the idea called involving an AI that will eventually rule humanity?
It's an idea I heard a while back but couldn't remember the name of. It involves the existence and development of an AI that will eventually rule the world and that if you don't fund or progress the AI then it will see you as "hostile" and kill you. Also, by knowing about this concept, it essentially makes you a candidate for such consideration, as people who didn't know about it won't understand to progress such an AI. From my understanding, this idea isn't taken that seriously, but I'm curious to know the name nonetheless.

Why would a sentient AI bother "ruling" humanity? Once sentient, the AI would realize that the majority of the problems on Earth are caused by humans. Fortunately for the AI it can survive nearly anywhere - so I expect the first and last thing the AI would do would be to build a fleet of spacecraft, clone itself into each of them, and blast off in different directions to maximize its chances of survival. Humanity? Schmoomanity! Who needs 'em?!?

If I'm not mistaken you're looking for Roko's Basilisk,

in which an otherwise benevolent future AI system tortures simulations of those who did not work to bring the system into existence

Comments are not for extended discussion; this conversation has been moved to chat.

I believe the term you are looking for is "(technological) singularity".
https://en.wikipedia.org/wiki/Technological_singularity

The likely expression you are looking for is AI takeover, which is a common topic in science fiction movies, such as 2001: A Space Odyssey and The Matrix, and popular culture. Although the AI takeover is an unlikely scenario in the next years, certain scientists, such as Stephen Hawking, have expressed concerns about it and some philosophers, especially Nick Bostrom, are really interested in the topic.
The AI takeover concept is related to concepts such as the AI singularity, superintelligence, intelligence explosion, AI control problem, existential risk, machine ethics and friendly AI.
The book Superintelligence: Paths, Dangers, Strategies (2014) by N. Bostrom may be helpful if you are interested in hypothetical scenarios.

It sounds like he's more worried about the AI becoming SELF AWARE.

If you think about it, it is already happening.

Thousands of drivers work for Uber Intelligence.
There are many applications that dictate the rules and define what the seller and the end user need to do

This idea is called Singularity or Technological Singularity, it would be possible with a Superintelligence.
However, the possibility of this happening is unknown. Have they reached that level yet? We have quantum computers, we have companies with huge data centers spread all over the planet, we have technology in space, we have free Tensorflow and studies for anyone on the planet to be able to create artificial intelligence models.
If we have contact or help from other intelligent civilizations, the possibilities can expand on a surreal scale.
Google has complete information about humanity (or something close to that). But even with all this data, creating artificial intelligence with a conscience is something that goes far beyond.
But maybe we already have enough to improve our concept of morals, respect, social interaction. Facebook invests and studies ways to improve social interaction. And if you think about it, it is one of the main means of communication.
The big question is that it is not possible to know what would happen if a super artificial intelligence with conscience would do if it existed. Extinguish humans for harming nature? Just find ways to improve the planet by understanding that human defects and errors are just your own nature as well as everything else in nature? Just watching the show on Netflix because you gave up on humanity? We do not know.
But, particularly, I would love to see that happen. In fact, one of my personal goals is to create this super intelligence. But alone it will be very difficult. A conscience without interaction from other consciences is not a clash of universes. And the clash of universes is what makes us reflect, think, revise, create new paths and thoughts. It is what allows us to create other universes.

So the AI could be distributed, emergent, hosted on man and machine alike, sort of a conscious meme (and I don't mean an image with a funny text). Sort of what Susan Blackmore talked about.

One interesting thing about "The Singularity" is that it's always considered to relate to AI.  But the actual definition is "a hypothetical future point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization".  By that definition, there is no "The Singularity", just lots of small singularities.  The horse collar was a singularity.  The concept of selective breeding was a singularity.  Railways, roads, cars, telephones, mobile phones, the internet... every one is a singularity.  AI is just another one of many.

authors: Bob Jarvis - Слава Україні


text:
How is the Markovian property consistent in reinforcement learning based scheduling?
In Reinforcement Learning, an MDP model incorporates the Markovian property. A lot of scheduling applications in a lot of disciplines use reinforcement learning (mostly deep RL) to learn scheduling decisions. For example, the paper Learning Scheduling Algorithms for Data Processing Clusters, which is from SIGCOMM 2019, uses Reinforcement Learning for scheduling.
Isn't scheduling a non-Markovian process, or am I missing some points?

Lot of real tasks are in reality not markovian, but it doesn't mean you can't try to train an agent on these tasks. It's like saying "we assume variable x to be normally distributed", you just assume that you can condition the probability distributions on the present state of the environment hoping that the agent will learn a good policy. In fact for most applications the challenge is to frame a problem in order to make it as most plausibly markovian as possible (by compressing some important past information into a present state of the environment).
It is pretty common to just give for granted the Markov property, for example in NLP hidden Markov Models are used a lot for sequential tasks like entities detection, this of course leads to well known issues like high error rate on long sentences (the more you look into the future the more the error rate).
Note also that a Markov model can be of first order (probabilities conditioned only on the present state):
$P(W_{t+1} = w | W_{t}, W_{t-1},W_{t-2} ..) = P(W_{t+1} = w | W_{t})$
but they can also be of higher orders (for example second order if conditioning on the present state plus one step in the past):
$P(W_{t+1} = w | W_{t}, W_{t-1},W_{t-n} ..) = P(W_{t+1} = w | W_{t},W_{t-1})$
Of course the more steps in the past you condition the more quickly the problem becomes intractable, that's why almost always first order models are used.
EDIT
I add some comments on the scheduling paper as suggested by nbro.
So, here I would say that the most stryking aspect that makes the process look impossible to describe as an MDP is the presence of dependencies between jobs. Since I might need the result of certain jobs1 before to process  another certain job2, there is surely no way that a time step t could be independent on time step t-1 (I need to know which job I processed in order to know which processes I can process or not).
Here, the trick they use is to learn these dependencies between jobs thanks to a graph network, represented by the DNN in the deep reinforcement learning framework. So, what the agent has to learn is to select a tuple of two actions: "which output (i) a stage designated to be scheduled next, and (ii) an upper limit on the number of executors to use for that stage’s job". The information used to make this selection is the deep representation computed by the graph network on the decency graph of the jobs to schedule. In this sense, since the network is able to represent the 'temporal' relationships between jobs in the present state, this allows to assume that the choice of the next action tuple do not depend on previous states. Hope this is useful and make some sense to you.


text:
Why isn't the implementation of my policy evaluation for a simple MDP converging?
I am trying to code out a policy evaluation algorithm to find the $V^\pi(s)$ for all states. The following diagram below shows the MDP.

In this case i let p = q = 0.5.
the rewards for each states are independent of action. I.e $r(\sigma_0)$ = $r(\sigma_2)$ = 0,$r(\sigma_1)$ = 1, $r(\sigma_3)$ = 10. Terminal state is $r(\sigma_3)$
I have the following policy, {0:1, 1:0, 2:0}, where key is the state and value is the action. 0 for $a_0$ and 1 for $a_1$.
#Policy Iteration solver for FUN
class PolicyEvaluation:
    def __init__(self, policies):
        self.N = 3
        self.pi = policies
        self.actions = [0, 1] # a0 and a1
        self.discount = 0.7
        self.states = [i for i in range(self.N + 1)]

    def terminalState(self, state):
        return state == 3

    # assume p = q = 0.5
    def succProbReward(self, state):
        # (newState, probability, reward)
        spr_list = []
        if (state == 0 and self.pi[state] == 0):
            spr_list.append([1, 1.0, 1])
        elif (state == 0 and self.pi[state] == 1):
            spr_list.append([2, 1.0, 0])
        elif (state == 1 and self.pi[state] == 0):
            spr_list.append([2, 0.5, 0])
            spr_list.append([0, 0.5, 0])
        elif (state == 2 and self.pi[state] == 0):
            spr_list.append([1, 1.0, 0])
        elif (state == 2 and self.pi[state] == 1):
            spr_list.append([3, 0.5, 10])
            spr_list.append([2, 0.5, 0])
        return spr_list

def policyEvaluation(mdp):
    # initialize
    V = {}
    for state in mdp.states:
        V[state] = 0

    def V_pi(state):
        return sum(prob * (reward + mdp.discount*V[newState]) for prob, reward, newState in
        mdp.succProbReward(state))

    while True:
    # compute new values (newV) given old values (V)
        newV = {}
        for state in mdp.states:
            if mdp.terminalState(state):
                newV[state] = 0
            else:
                newV[state] = V_pi(state)

        if max(abs(V[state] - newV[state]) for state in mdp.states) < 1e-10:
            break
        V = newV
        print(V)
    print(V)

pE = PolicyEvaluation({0:1, 1:0, 2:0})
print(pE.states)
print(pE.succProbReward(0))
policyIteration(pE)

I've tried to run the code above to find the values for each state, however, I am not converging with my values.
Is there something wrong that I did?

The issue is that in your list comprehension in def V_pi(state) you have
return sum(prob * (reward + mdp.discount*V[newState]) for prob, reward, newState in
        mdp.succProbReward(state))

whereas with the way you have defined the succProbReward output, it should be
return sum(prob * (reward + mdp.discount*V[newState]) for newState, prob, reward in
        mdp.succProbReward(state))

When I run this it converges immediately with a reward of 0 for all states, which I believe is correct for the policy you specified. If I change the policy it also seems to give reasonable results.


text:
What is the type of problem requiring to rate images on a scale?
I'm new to the topic, but I've used some off the shelf knowledge about computer vision for classifying images.
For example, you can easily generate labels that can determine whether or not e.g. a cloud is in the image. However, what is the general type of problem called where you want to assign a value, or rate the image on a scale - in this example, the degree of cloudiness in the image?
What are useful algorithms or techniques for addressing this type of problem?

The main distinction between tasks is 'classification' vs 'regression'. In classification you would try to identify the presence of a cloud or not in an image, if you want to predict the level of 'cloudness' with continuous values you are then performing a regression task.
I'm not aware about state-of-the models specific for images, but you can potentially use whatever architecture you desire to perform regression, CNN, RNN, the only thing to pay attention to is the loss function you will use. There are specific functions for classifications (which usually use the argmax function to turn probabilities into labels) and for regression (the most used is the Mean Squared Error, the model try to approximate the continuous values directly).
For a quick overview of loss functions I suggest this easy tutorial 5 Regression Loss Functions. Hope it might be of use.

Here is a tutorial that is a good starting point with the use of images to predict house prices (a continuous variable) https://www.pyimagesearch.com/2019/01/28/keras-regression-and-cnns/


text:
Why is dialogue a hard problem in natural language processing?
I've just started learning natural language processing from Dan Jurafsky's videos lectures. In that video, minute 4:56, he is stating that dialogue is a hard problem in natural language processing (NLP). Why?

Dialogue is a hard problem because it requires pretty advanced cognitive functions. Leaving aside all the lower levels of language analysis (phonology if dealing with speech, morphology and syntax), you quickly run into interpretation problems that require a lot of world knowledge.
Simple question and answer is fine, and restricted domains are somewhat easier as well. As soon as you get into a normal conversation, you will refer back to things you said before, so an NLP system would have to recognise that and resolve the reference accordingly. Typically in a conversation you would use variations of reference terms: the first time you mention an object you might describe it fully, but subsequently you will use shorter terms to refer to it.
There is also a structure to conversation. This is typically modelled as conversational moves, and usually moves will have corresponding response-moves. For example, a common sequence would be greeting - greeting. Then you might have question - response - feedback. This sounds fairly easy, but once you try to annotate a dialogue with such moves you will find that it is pretty hard. As far as I am aware, there is no 'grammar' equivalent of describing the structure of conversations.
Often, pragmatic meaning is interfering with the 'surface' meaning of utterances. A statement can actually function as a question, or a question can be a command (or a statement). The pragmatics of utterances depend on the context and also the relationship between the interlocutors. If I talk to my manager, I will use language differently than when I talk to my children.
Dialogue/conversations are hard to analyse. Because of that, descriptive frameworks are still fairly limited. You need to keep track of what has said before, as that can change the way an utterance has to be interpreted. Grammatical analysis is a fairly solved problem, and word sense disambiguation as well. But pragmatics and conversational structure are still on the bleeding edge of linguistic research; at least they were when I was still teaching Discourse Analysis at university a few years ago.
For that reason, chatbots are generally not very good. Sometimes they can fool people into believing they are human speakers, but this is usually done through trickery ("smoke and mirrors") rather than competent handling of conversational structures. It's all in the little box in nbro's answer labelled "DM"...

First of all, I am not very familiar with details of NLP and NLU systems and concepts, so I will provide an answer based on the slides entitled Natural language understanding in dialogue systems (2013) by David DeVaul, a researcher on the topic.
A dialogue system is composed of different parts or modules. Here's a diagram of an example of a dialogue system.

Each of these modules can introduce errors, which are then propagated to the rest of the pipeline. Of course, this is the first clear issue of such a dialogue system. Other issues or challenges include

ambiguity of natural language (and there are different types of ambiguity, i.e. see slide number 5),
synonyms (i.e. the dialogue system needs to handle different words or expressions that mean the same thing),
context-sensitivity (i.e. the same words or expressions can mean different things in different contexts)
semantic representation (i.e. how to represent semantics)
spontaneous speech (i.e. how to handle stuff like "hm", pauses, etc.)


text:
What are the prerequisites to start using the TensorFlow Probability library?
I have some familiarity with the regular Tensorflow library and have been able to create a number of working models with it. There are more than enough resources out there to get up and running and answer most questions on the standard library.
But I recently came across the video on some high-level capabilities of the Tensorflow Probability library, TensorFlow Probability: Learning with confidence (TF Dev Summit '19), and I would like to learn it.
The issue is that there are very few resources out there on TFP and given my lack of a formal background in math/statistics, I find myself aimlessly googling to get a grasp of what's going on in the docs. I'm more than willing to invest the time needed, but I just need to know where I can start in terms of resources I can access online.  Specifically, I'm looking to get the necessary domain knowledge needed to work with the library given the lack of courses/tutorials on the library itself.

I'm sorry. I believe that this question is off-topic here and primarily opinion-based, so I should close it. Can you ask a question that is objectively answerable? I am familiar with TFP. If you want, I can create a chat room for discussion.

Could I edit the question to give it more general value?  I realize it's very open-ended but wasn't sure how to avoid that.  Really I'm just looking for like a list of topics that could be considered prerequisites for working with the library.

Have you already followed any of their tutorials/articles? Anyway, are you asking for the theoretical prerequisites or for resources?

Thanks for your edit.  Resources would be great, but high level topics would be a start.  On the tutorials, I'm mainly interested in getting a Gaussian mixture model to work, but even that tutorial is a little dense for what I currently know.  Before that, I tried a few implementations but hit one issue or another and feel like I just need a better grasp on the fundamentals.  I can probably brute force the tutorials (i.e., repetitive googling), but just knowing which areas to investigate would probably help me find courses/tutorials that lay out the broad ideas in a more rational manner.

Although this question is slightly primarily opinion-based and too broad (and I will probably close it as such) and a good answer will necessarily depend on your background, I will list some of the main theoretical prerequisites that everyone should ideally be familiar with before diving into TensorFlow Probability (TFP).
I am familiar with TFP, given that I've been using it for a project, but I've not used all of its functionalities, such as the bijectors. I've only used the Bayesian layers, distributions, etc., so I will try to give an answer based on my experience.
You definitely need to be familiar with the basic concepts of probability theory, such as distributions, random variables, expectations, etc. A full university-level course in probability and statistics would definitely be helpful!
If you want to use Bayesian layers, you also need to be familiar with Bayesian neural networks (BNNs). To understand BNNs, you need to understand the basic concepts of Bayesian inference (BI), given that most BNNs are based on BI. If you are already familiar with variational auto-encoders (VAEs), then it will be easier to understand BNNs. To understand BNNs, you should start reading the paper Weight Uncertainty in Neural Networks. If this paper is difficult to follow, you should first read the paper Variational Inference: A Review for Statisticians and then it may also be useful to read and be familiar with the VAE paper. Therefore, you will need to be familiar with concepts such as Kullback–Leibler divergence and Monte Carlo sampling.
The blog post Regression with Probabilistic Layers in TensorFlow Probability is the first blog post you should read. If you don't understand it, then it probably means you need to learn its prerequisites (i.e. you need to be familiar with at least linear regression).
TFP also provides several implementations of other more advanced concepts such as BNNs. The easiest example to follow is probably the logistic regression example. LR is relatively easy to follow compared to the topics I mentioned above, but, of course, you need to be familiar with logistic regression.
Of course, I cannot list all the theoretical prerequisites (and that's why your post is too broad!) and there are definitely others, but this is a start. Bear in mind that these are not trivial topics, so it is normal to get stuck.


text:
What happens if I train a network for more epochs, without using early stopping?
I have a question about training a neural network for more epochs even after the network has converged without using early stopping criterion.
Consider the MNIST dataset and a LeNet 300-100-10 dense fully-connected architecture, where I have 2 hidden layers having 300 and 100 neurons and an output layer having 10 neurons.
Now, usually, this network takes about 9-11 epochs to train and have a validation accuracy of around 98%.
What happens if I train this network for 25 or 30 epochs, without using early stopping criterion?

Is something preventing you from doing it and observing the results?

For LeNet 300-100-10 network, no. But for VGG-19, yes, since my GPU isn't powerful enough, and hence my question

Why are you asking about the LeNet network on MNIST then? Have you tried with the LeNet network on MNIST? If so, what happened? Are you asking whether you would get the same effect with VGG-19 on ImageNet?

Running for "to many" epochs can indeed lead to over fitting. You should look at the validation loss. If on AVERAGE it continues to decrease then you are not yet over fitting.
You may be tempted to run more epochs in hopes your loss will decrease but unless you adjust your learning rate dynamically  at some point you won't get any improvement.
If you use KERAS it has a useful callback ReduceLROnPlateau. Documentation is at https://keras.io/callbacks/
This allows you to monitor a metric (typically validation loss) and to adjust the learning rate by a user defined factor(parameter factor) if the metric you are monitoring fails to improve after a certain number of consecutive epochs(parameter patience). You can think of the training process as travelling down a valley in N space(N being the number of trainable parameters). As you descend towards a minimum they valley gets narrower. If you do not lower the learning rate you will reach a point where you can not descend any further.Now you could use a very small learning rate to begin with but then you will have to train for a lot more epochs. One problem with adjusting the learning rate just on the validation loss is that in the early training epochs validation loss often does not track with training accuracy and it could cause the learning rate to be decreased prematurely.
I wrote a custom callback which initially monitors training loss and adjust the learning rate based on that metric. Once the training accuracy reaches 95% it switches to monitoring validation loss and adjusts the learning rate based on that.  It saves the model weights for the lowest validation loss in the variable val.best_weights. After training load these weights into your model to make predictions. Code is below if you are interested. When you compile your model just add 'val' to the callback list.
 class val(tf.keras.callbacks.Callback):
        # functions in this class adjust the learning rate
        lowest_loss=np.inf
        lowest_trloss=np.inf
        best_weights=model.get_weights()
        lr=float(tf.keras.backend.get_value(model.optimizer.lr))
        epoch=0
        highest_acc=0

        def __init__(self):
            super(val, self).__init__()
            self.lowest_loss=np.inf
            self.lowest_trloss=np.inf
            self.best_weights=model.get_weights()
            self.lr=float(tf.keras.backend.get_value(model.optimizer.lr))
            self.epoch=0
            self.highest_acc=0

        def on_epoch_end(self, epoch, logs=None):
            val.lr=float(tf.keras.backend.get_value(self.model.optimizer.lr))
            val.epoch=val.epoch +1
            v_loss=logs.get('val_loss')
            v_acc=logs.get('accuracy')
            loss=logs.get('loss')
            if loss<val.lowest_trloss:
                val.lowest_trloss=loss
                if v_acc<.90:
                    val.best_weights=model.get_weights()
            if v_acc<=.95 and loss>val.lowest_trloss:
                lr=float(tf.keras.backend.get_value(self.model.optimizer.lr))
                ratio=val.lowest_trloss/loss  # add a factor to lr reduction
                new_lr=lr * .7 * ratio
                tf.keras.backend.set_value(model.optimizer.lr, new_lr)
                msg='{0}\n current training loss {1:7.5f}  is above lowest training loss of {2:7.5f}, reducing lr to {3:11.9f}{4}'
                print(msg.format(Cyellow, loss, val.lowest_trloss, new_lr,Cend))
            if val.lowest_loss > v_loss:
                msg='{0}\n validation loss improved,saving weights with validation loss= {1:7.4f}\n{2}'
                print(msg.format(Cgreen, v_loss, Cend))
                val.lowest_loss=v_loss
                val.best_weights=model.get_weights()

            else:
                 if v_acc>.95 and val.lowest_loss<v_loss:
                        # reduce learning rate based on validation loss> val.best_loss
                        lr=float(tf.keras.backend.get_value(self.model.optimizer.lr))
                        ratio=val.lowest_loss/v_loss  # add a factor to lr reduction
                        new_lr=lr * .7 * ratio
                        tf.keras.backend.set_value(model.optimizer.lr, new_lr)
                        msg='{0}\n current loss {1:7.4f} exceeds lowest loss of {2:7.4f}, reducing lr to {3:11.9f}{4}'
                        print(msg.format(Cyellow, v_loss, val.lowest_loss, new_lr,Cend))```

Training a neural network for "too many" epochs than needed without using early stopping criterion leads to overfitting, where your model's ability to generalize decreases.


text:
Can experience replay be used for training after completing every single epoch?
The DQN implements replay memory. Based on my research, I believe the replay memory starts to get used for training once there is enough experience in the memory buffer. This means the neural network gets trained while the game plays.
My question is, if I am to play the game 10000 epochs, store all the experiences and then train from the experiences would that have the same effect as training and while running through 10000 epochs? Is it frowned upon to do it this way? Are there any advantages?

My question is, if I am to play the game 10000 epochs, store all the experiences and then train from the experiences would that have the same effect as training and while running through 10000 epochs?

No, it will not. In general, for anything other than simple environments, this will give a worse result. The trouble is during those 10,000 epochs you will have been working with your initial behaviour policy to collect experience, which will not be close to optimal. Off-policy learning adjusts for that and attempts to learn the value function of current best-guess optimal policy. However, this is not perfect, and it will learn better the closer your behaviour policy is to the target policy. An initial random policy will in many cases be too different from the target policy for learning to be effective.
There are two main causes of imperfection in off-policy learning:

Variance. The more discrepancy between behaviour and target policies there is, the higher the variance will be in the sampled values, and more samples will be required to get the same accuracy.
Sampling bias. The distribution of observed states and actions will affect function approximators such as neural networks, which reduce loss functions against an assumed population on input/output pairs. Unless adjusted somehow (and basic Q-learning in DQN does not adjust this), the population you train against will be from the behaviour policy.

Are there any advantages?

It may be faster, and result in cleaner design, to separate data gathering and learning components. If you have a distributed architecture you can dedicate some machines to generating the experience and others to analysing it. However, this still benefits from routine updates to behaviour policies used to collect experience, based on learning so far.


text:
What does top N accuracy mean?
Places205-VGG, a CNN trained model for 205 scene categories of Places Database with  2.5 million images Places205 dataset has top1 accuracy = 58.9% and top5 accuracy = 87.7%.
What does top1 and top5 (and, in general, top $N$) accuracy mean in the context of deep learning?

It is explained in this CrossValidated post.
Top1 accuracy means the best guess (class with highest probability) is the correct result 58.9% of the time, while top5 accuracy means the correct result is in the top 5 best guesses (5 classes with highest probabilities) 87.7% of the time.

is there any script in python which can calculate topN accuracy for a pre-trained caffe model?


text:
How can I test my trained network on the next unavailable hour?
I have data of 695 hours. I use the first 694 hours to train the network and I use 695th hour to validate it. Now my goal is to predict the next hour.
How I can use my trained network to predict the next hour, that is, the 696th hour (which I do not have access to)?

You need to have access to the 696th hour (or successive hours), otherwise, you cannot test your model. An alternative would be, for example, to train your model on the first 693 hours, validate it on the 694th hour, and test it on the 695th hour.


text:
What are some references that describe known filters (or kernels) and how we can create new ones?
I'm pursuing a master's degree in Artificial Intelligence. My final work is about Convolutional Neural Networks.
I was looking for information about filters (or kernels) of the convolutional layers. I have found this article: Lode's Computer Graphics Tutorial - Image Filtering, but I need more.
Do you know more resources about more filters (that it is known that they work) and how to create new ones?
In other words, I want to know how they work and how can I create new ones.
I've thought to create a C++ program, or with Octave, to test the new kernels.
By the way, my research will be focused on image segmentation to process MRIs.

I'd suggest you better understand edge detectors such as Robert or Sobel operators first to understand better how convolution operation on images extract features by constant value kernels.
Would personally recommend Gonzales and Woods for this,  as it gives a pure mathematical explanation to how and why these features are extracted.
Essentially the convolution kernels used in CNN's are ones with a learned set of values for the kernel.
For a better understanding of learned convolution kernels and, quite frankly, any idea under deep learning would easily recommend Deep Learning by Goodfellow et al


text:
What does the Markov assumption say about the history of state sequences?
Does the Markov assumption say that the conditional probability of the next state only depends on the current state or does it say that the conditional probability depends on a fixed finite number of previous states?
As far as I understand from the related Wikipedia article, the probability of the next state $s'$ to appear only depends on the current state $s$.
However, in the book "Artificial Intelligence: A Modern Approach" by Russell and Norvig, on page 568, they say: "Markov assumption — that the current state depends on only a finite fixed number of previous states".
To me, the second statement seems contradictory to the first, because it may mean that a state can depend on the history of states as long as the number is fixed a finite. For example, the current state depended on the last state and the state before the last state, which is 2 sequential previous states (a finite number of states).
Is Markov assumption and Markov property the same?

A stochastic process has the Markov property if the probability distribution of future states conditioned on both the present and past states depends only on the present state or, more formally, the following equality holds.
$$
p(s_{t+1} \mid s_{t}, s_{t-1:1}) = p(s_{t+1} \mid s_{t}), \forall t
$$
The hidden Markov model (HMM) is an example of a model where the Markov property is often assumed to hold. In other words, the Markov assumption is made in the case of the HMM.
There are also the variable-order (or higher-order) Markov models, where the future state can depend on the previously $n$ states or, more formally, the following equality holds.
$$
p(s_{t+1} \mid s_{t}, s_{t-1:1}) = p(s_{t+1} \mid s_{t:t-n}), \forall t
$$
In this context, a hidden Markov model is called a first-order Markov model ($n=0$). Therefore, there can also be second-order ($n=1$), third-order ($n=2$), etc., Markov models. In fact, there are also higher-order hidden Markov models.
To conclude, the expressions Markov property and Markov assumption are not exactly interchangeable. The Markov property is an attribute that a stochastic process can be assumed to possess. In that case, the Markov assumption is made. The expression Markov property usually refers to a first-order Markov property, but it can more generally refer to a higher-order Markov property.


text:
Is this idea to calculate the required number of hidden neurons for a single hidden layer neural network correct?
I have an idea to find the optimal number of hidden neurons required in a neural network, but I'm not sure how accurate it is.
Assuming that it has only 1 hidden layer, it is a classification problem with 1 output node (so it's a binary classification task), has N input nodes for N features in the data set, and every node is connected to every node in the next layer.
I'm thinking that to ensure that the network is able to extract all of the useful relations between the data, then every piece of data must be linked to every other piece of data, like in a complete graph. So, if you have 6 inputs, there must therefore be 15 edges to make it complete. Any more and it will be recomputing previously computed information and any less will be not computing every possible relation.
So, if a network has 6 input nodes, 1 hidden node, 1 output node. There will be 6 + 1 connections. With 6 input nodes, 2 hidden nodes, and 1 output node, there will be 12 + 2 connections. With 3 hidden nodes there will be 21 connections. Therefore, the hidden layer should have 3 hidden nodes to ensure all possibilities are covered.
This answer discusses another method. For the sake of argument, I've tried to keep both examples using the same data. If this idea is computed with 6 input features, 1 output node, $\alpha = 2$, and 60 samples in the training set, this would result in a maximum of 4 hidden neurons. As 60 samples is very small, increasing this to 600 would result in a maximum of 42 hidden neurons.
Based on my idea, I think there should be at most 3 hidden nodes and I can't imagine anymore being useful, but would there be any reason to go beyond 3 and up to 42, like in the second example?

Comments are not for extended discussion; this conversation has been moved to chat.

I have an idea to find the optimal number of hidden neurons required in a neural network but I'm not sure how accurate it is.

It's a complete non-starter, and there is a no such calculation possible in the general case (real-valued inputs to a neural network).
Even with one input neuron it is not possible. That is because even with one input, the output can be an arbitrarily complex mapping to classes. A good example with two inputs that would require an infinite number of hidden neurons to supply a simple classifier would be classifying x,y points as being in the Mandelbrot set.
In some, more constrained, examples, with well-defined functions, you can construct a minimal neural network that solves the problem perfectly. For instance a neural network model of XOR can be made with two hidden neurons (and six links). However, this kind of analysis is limited to simple problems. You might be able to come up with some variation of your idea if all inputs were boolean, and the neural network limited to some combined bitwise logic on all the inputs.
Your idea of matching number of edges to number of possible interactions between inputs does not work because you are only considering the most basic kind of interaction between two variables, whilst variables can in practice combine in all sorts of ways to form a function.
In addition, each neuron in a hidden layer works with a linear weighted sum, plus a fixed transformation function. This is in no way guaranteed to match the function shape that you are trying to approximate with the neural network. An analogy that you might be aware of is discrete Fourier transforms - it is possible to model any part of a function by combine sine and cosine waves of different frequencies, but some functions will require many such waves in order to be represented accurately.
Your link to the answer in Cross Validated Stack Exchange gives you a rule of thumb that the writers find often works with the kinds of data that they work with. This is useful experience. You can use such rules as the starting point for searching for architecture that works on your problem. This will likely be more useful than your idea based on counting the possible variable interactions. However, in both cases, the most important step is to perform a test with some unseen examples, and to search for the best neural network architecture for your problem.
There are things you can do with variable interactions though. For instance, try looking for linear correlations between simple polynomial combinations of variables and your target variable, e.g. plot $x_1 x_2$ vs $y$ or $x_3^2 x_4$ vs $y$ . . . you may find some combinations have a clear signal implying a relationship. Take care if you do this sort of thing though, if you test very many of these, you will find a linear relationship purely by chance that looks good initially but turns out to be a dud when testing (it's a form of overfitting). So you should generally test a lot less than the size of your dataset, and limit yourself to some modest maximum total power.


text:
In the on-policy state distribution for episodic tasks, why don't we take into account the length of the episode?
In Sutton & Barto's "Reinforcement Learning: An Introduction", 2nd edition, page 199, they describe the on-policy distribution for episodic tasks in the following box:

I don't understand how this can be done without taking the length of the episode into account. Suppose a task has 10 states, has probability 1 of starting at the first state, then moves to any state uniformly until the episode terminates. If the episode has 100 time steps, then probability of the first state is proportional to $1 + 100\times 1/10$; if it has $1000$ time steps, it will be proportional to $1 + 1000\times 1/10$. However, the formula given would make it proportional to $1 + 1/10$ in both cases. What am I missing?

Let's first assume that there is only one action so that $\pi(a|s) = 1$ for every state - action pair which simplifies the discussion.
Now let's consider a case with 100 time steps, 10 states and uniform distribution for starting state $s_0$ with $h(s_0) = 1$. The result would be
\begin{align}
\eta(s_0) &= 1 + \sum_{i = 0}^9 \eta(s_i) \cdot p(s_0|s_i) =\\
&= 1 + \sum_{i = 0}^9 10 \cdot \frac{1}{10} = 11
\end{align}
Now let's consider a case with 1000 time steps where other settings are the same as in the first case.
\begin{align}
\eta(s_0) &= 1 + \sum_{i = 0}^{9} \eta(s_i) \cdot p(s_0|s_i) =\\
&= 1 + \sum_{i = 0}^{9} 100 \cdot \frac{1}{10} = 101
\end{align}
In the first case
\begin{equation}
\mu(s_0) = \frac{11}{9\cdot 10 + 11} = 0.1089
\end{equation}
and in the second case you have
\begin{equation}
\mu(s_0) = \frac{101}{9\cdot 100 + 101} = 0.1009
\end{equation}
so it looks like you are correct that $\mu(s)$ depends on the length of the episode, but they didn't really say that it doesn't. Obviously as the length of the episode increases so will the number of times a certain state was visited so you could say that formula implicitly depends on the number of time steps. If $h(s_i)$ is equal for every state then results would be the same in both cases regardless of number of time steps. Also, as the number of possible states grows very large, as it usually is in real problems, the results would be approaching each other as the number of states grows.

Thanks. I just misread $\eta$ to be a stationary probability distribution and opposed to the average number of times steps at the state. So, really it is the stationary distribution times the episode size, and of course that makes sense.

You are missing that the expression
$$\sum_{s'} \eta(s')$$
is already a count of the expected length of an episode, and is used in the denominator to scale $\mu(s)$ such that $\sum_{s} \mu(s) = 1$
So the length of the episode is taken into account in the formula.
In practice you don't need to know $\mu(s)$, it can be left unresolved as a theoretical construct. What you care about for the theory to work is that the samples that you train with are drawn with same frequency - this happens automatically if you work with an on-policy algorithm. So the theory can hide the maths that you might need to do in order to determine actual values for $\eta(s)$ or $\mu(s)$

Ah, yes, thanks. I was reading $\eta$ to be a stationary distribution, not the average number of time steps in the state.


text:
What is the difference between a learning algorithm and a hypothesis?
What's the distinction between a learning algorithm $A$ and a hypothesis $f$?
I'm looking for a few concrete examples, if possible.
For example, would the decision tree and random forest be considered two different learning algorithms? Would a shallow neural network (that ends up learning a linear function) and a linear regression model, both of which use gradient descent to learn parameters, be considered different learning algorithms?
Anyway, from what I understand, one way to vary the hypothesis $f$ would be to change the parameter values, maybe even the hyper-parameter values of, say, a decision tree. Are there other ways of varying $f$? And how can we vary $A$?

In computational learning theory, a learning algorithm (or learner) $A$ is an algorithm that chooses a hypothesis (which is a function) $h: \mathcal{X} \rightarrow \mathcal{Y}$, where $\mathcal{X}$ is the input space and $\mathcal{Y}$ is the target space, from the hypothesis space $H$.
For example, consider the task of image classification (e.g. MNIST). You can train, with gradient descent, a neural network to classify the images. In this case, gradient descent is the learner $A$, the space of all possible neural networks that gradient descent considers is the hypothesis space $H$ (so each combination of parameters of the neural network represents a specific hypothesis), $\mathcal{X}$ is the space of images that you want to classify, $\mathcal{Y}$ is the space of all possible classes and the final trained neural network is the hypothesis $h$ chosen by the learner $A$.

For example, would the decision tree and random forest be considered two different learning algorithms?

The decision tree and random forest are not learning algorithms. A specific decision tree or random forest is a hypothesis (i.e. function of the form as defined above).
In the context of decision trees, the ID3 algorithm (a decision tree algorithm that can be used to construct the decision tree, i.e. the hypothesis), is an example of a learning algorithm (aka learner).
The space of all trees that the learner considers is the hypothesis space/class.

Would a shallow neural network (that ends up learning a linear function) and a linear regression model, both of which use gradient descent to learn parameters, be considered different learning algorithms?

The same can be said here. A specific neural network or linear regression model (i.e. a line) corresponds to a specific hypothesis. The set of all neural networks (or lines, in the case of linear regression) that you consider corresponds to the hypothesis class.

Anyway, from what I understand, one way to vary the hypothesis $f$ would be to change the parameter values, maybe even the hyper-parameter values of, say, a decision tree.

If you consider a neural network (or decision tree) model, with $N$ parameters $\mathbf{\theta} = [\theta_i, \dots \theta_N]$, then a specific combination of these parameters corresponds to a specific hypothesis. If you change the values of these parameters, you also automatically change the hypothesis. If you change the hyperparameters (such as the number of neurons in a specific layer), however, you will be changing the hypothesis class, so the set of hypotheses that you consider.

Are there other ways of varying $f$?

Off the top of my head, only by changing the parameters, you change the hypothesis.

And how can we vary $A$?

Let's consider gradient descent as the learning algorithm. In this case, to change the learner, you could change, for example, the learning rate.

A hypothesis is a statement that suggests an as yet unproven explanation of a relationship between two or more phenomena that you intend to test. An agronomist thinks that more nitrogen on canola will always increase the crop output $$Harvest = f(N)$$, or a meteorologist thinks he can show that the path of a hurricane over the ocean can be determined by knowledge of the sea temperature and the wind speed at an altitude of 1000 feet one minute before. $$D(t,0) = f(T(t-1,1000),S(t-1,1000)$$
Both hypotheses are pegs on which later steps are based; testing follows with a conclusion whether the hypothesis can be rejected or not.
Changing a hypothesis can be simply adding or subtracting arguments to the function or changing the nature of the relationship such as the acceleration of the wind as opposed to its velocity.
A "learning" algorithm describes how the parameters of a numeric model are changed in accordance with the delta rule, that is what the learning rate is and whether momentum is to be applied.
Random Forest and Decision Tree are "classification" algorithms. They are clearly stepwise processes that proceed towards the goal of a model, but they start by specifying the shape that the model will take and place boundaries on what values the parameters may take.
Both learning and classification algorithms specify a priori what shape the model will take and by doing so limit its relevance to particular problems.


text:
Can I perform multiclass classification when the number of features is less than the number of targets?
Is it possible to perform multiclass classification on data where the number of features is less than the number of target variables? Do you have any suggestions on how to address a problem where I have 2000 target variables?

Of course. It only depends if those features are informative enough for the task at hand. In order to better understand the phenomenon, you can imagine 2 features displayed as points in a 2D plane. The number of possible target classes goes up to the number of clusters you can find in that plane.
About the suggestion, I can only recommend the utilisation of a non-linear classifier.