text:
How does the automated temperature adjustment step work in Soft Actor-Critic?
In section 5 of the paper Soft Actor-Critic Algorithms and Applications, it is proposed an optimization problem to obtain an optimal temperature parameter $\alpha^*_t$. First, one uses the original evaluation and improvement steps to estimate $Q_t^*$ and $\pi_t^*$, and then one somehow solves the optimization problem:
$$\alpha_t^* = \arg\min_{\alpha_t} \mathbb E _{a_t\sim\pi^*}\left[\alpha_t(-\log\pi_t^*(a_t|s_t;\alpha_t)-H)\right]\text .$$
As far as I understand, we should use our current estimate of $\pi_t^*$ to solve that problem. Since it was obtained from a previous $\alpha_{t-1}^*$, in practice, it is not dependent on $\alpha_t$ and so the optimization problem becomes a linear problem with the only restriction being $\alpha_t\geq0$.
Here comes my problem: under this rationale, if $\alpha_t$ is a scalar independent of both state $s_t$ and action $a_t$, the value of the cost function is just proportional to $\alpha_t$ and so the solutions are either $0$ or $\infty$, depending on the sign of the expected value (something similar happens if $\alpha_t^*=\alpha_t^*(s_t,a_t)$). However, the whole idea of introducing this parameter is to account optimally for the exploration of the policy.
What is the correct way to solve this optimization problem along with the evaluation and improvement steps? I am particularly interested in the tabular case. Also, is there any explanation why they use a negative minimum entropy $H$ when the entropy is always positive?
By the way, in the approximate case, the current official implementation seems to be doing just that: moving $\alpha_t^*$ up or down a little bit (closer to $\infty$ or 0, respectively), depending on the magnitude of the expected value. I guess one could do the same for the tabular case, modifying the $\alpha_t^*$ only a little bit in each step, but this seems rather suboptimal.


text:
If the accuracy of my current model is low ($50 \%$) and we want to minimize time in collecting more data, should we try other models?
Suppose we have a data set with $4,000$ labeled examples. The outcome variable is trinary (three possible categorical values). Suppose the accuracy of a given model is "bad" (e.g. less than $50 \%$).

Question. Should you try different traditional machine learning models (e.g. multinomial logistic regression, random forests, XGBoost,
etc.), get more data, or try various deep learning models like
convolutional neural networks or recurrent neural networks?

If the purpose is to minimize time and effort in collecting training data, would deep learning models be a viable option over traditional machine learning models in this case?

This post may need more details to be answered properly. What model were you using at the time? What were its hyper-parameters? What loss was you using?

Accuracy can sometimes be a very coarse metric. When it is applied to three class problems, people often take the class label with maximum predicted probability and predict that. The probabilities of the individual labels are ignored. I'd recommend that as well as accuracy you calculate sensitivity and specificity for each class and the area under the ROC curve. For both of these, you can take a 1 vs rest (i.e. class 1 vs classes 2 & 3, class 2 vs classes 1 & 3 etc) approach to calculating the metrics. Even if your accuracy is under 50%, the model may still be predicting at least one class well, so I recommend doing more analysis before making a decision.
I also recommend comparing your model (if it is deep learning-based) to a traditional type of model, as deep learning usually works best with big data and with such a small dataset as you have, there may be no benefit to using deep learning (unless you are leveraging transfer learning).

To know if your model needs more training data, try to plot out "learning curves", that are based on increasing size of the training set.
Basically, you calculate training and validation accuracy metrics for 1, 2, 3, 4, 5, ..., m  training samples. Size of validation set may be constant over time. If the accuracy is still rising when your data set is fully used, then you need more training data.

If data collection is expensive, it is better to first try to improve your model.
You say your accuracy is bad, but have you using tried better performance metrics? A confusion matrix could help. Another potential problem is that your data may be imbalanced. What if your model is performing badly because, for example, there aren't enough samples from class 2. Now you know that to improve your model you need mode class 2 samples. This can be acquired by collecting more data, or by other class imbalance methods e.g. SMOTE.
You can also check whether the performance is bad only on the test set. Is the model overfitting? Or does it perform badly on the training set too? Underfitting. 4000 samples should be enough for a neural network, but not a very large deep learning model, maybe a smaller one. Testing other machine learning models on your data is definitely a good idea. Not only you can provide your model with a benchmark, but you also may find a better model.
If you think your data is good enough and your model is adequate, you can try hyperparameter optimization. This can be quire useful for getting the most out of neural networks, but it is trial and error, and it can take some time to find the best hyperparameters.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Can neural networks be used to find features importance?
I am wondering if I can use neural networks to find features importances in similar manner as it can be done for random forests or decision trees and if so, how to do it?
I would like to use it on tabular time series data (not images). The reason why I want to find importances on neural networks not on decision trees is that NNs are more complicated algorithms so using NNs might point out some correlations that are not seen by simple algorithms and I need to know what features are found to be more useful with that complicated correlations.
I am not sure if I made it clear enough, please let me know if I have to explain something more.

It's possible. I've used the olden() function from NeuralNetTools.
Please take a look at this example found online:
http://blogs2.datall-analyse.nl/2016/02/19/rcode_variable_importance_neural_network/

ofteт it is partially arguable that "NNs are more complicated algorithms so using NNs might point out some correlations that are not seen by simple algorithms", - though it  (NN) often can improve accuracy of prediction

This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to train on. I would us a rnn for with a softmax output layer that ranks features by performance.
You can find a good explanation of softmax here: https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax
basically it will assign probability values for each output node with all of these values adding up to 1.0

Could you please explain the procedure in a bit more detailed way or give some reference link?

I added a bit of info on softmax, what are you using for optimization? knowing that i may be able to provide a bit more insight into how i would set this up

Thanks for more info!
I am using tensorflow library, mostly with Adam optimizer (trying others as well) with custom learning rate.

at a ten thousand foot level this could be done by deciding the amount of features or the min probability as a cut of and then creating a net with softmax output layer with a node for each feature, you will likely want to use a reinforcement learning optimization technique as you will need to evaluate if how well the features the net picks perform in order to optimize, you could use neuroevolution with a fitness function for this or i imagine other rl algos would work too.

There are multiple standard ways of feature selection, for example ranking features by information gain, that you could use, and then you can train the neural network on just those features.
However, let's assume you have trained a neural network on all of the features and now want to estimate their importance. One approach you could take is to perform a sensitivity analysis on the inputs: add random noise in a controlled fashion to different features and see what effect it has. If the training dataset has been centered (so each feature has zero mean) then you could set the inputs to zero (the "average" training example) and then perturb each feature in turn to see what the effect is. You could also fix a feature permanently to zero and then run your validation data through the network and see how accuracy changes. There should be no major effect for insignificant features, but important features being zeroed should lead to a decrease in accuracy. You can also do something like this when predicting specific examples: perturb the example's features and see how much the prediction changes. LIME does something like this to explain why a black box like a neural network makes the predictions it does.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to express accuracy of a regression ANN that uses MSE loss function?
I have a regression MLP network with all input values between 0 and 1, and am using MSE for the loss function. The minimum MSE over the validation sample set comes to 0.019. So how to express the 'accuracy' of this network in 'lay' terms? If RMSE is 'in the units of the quantity being estimated', does this mean we can say: "The network is on average (1-SQRT(0.019))*100 =  86.2% accurate"?
Also, in the validation data set, there are three 'extreme' expected values. The lowest MSE results in predicted values closer to these three values, but not as close to all the other values, whereas a slightly higher MSE results in the opposite - predicted values further from the 'extreme' values but more accurate relative to all other expected values (and this outcome is actually preferred in the case I'm dealing with). I assume this can be explained by RMSE's sensitivity to outliers?

You can not use error to reliably measure accuracy. Error is best used as a measure of how fast the model is currently learning.
As an example, using different loss functions (cross entorpy vs MSE) results in massively different values for the error at similar accuracy.
Also considering this, an error of 0.0000000001 quite often has lower validation set accuracy then and error of 0.1, as the prior is likely over trained.
As for you second question, yes this is because MSE has a huge bias towards outliers. I have personally found regression networks to struggle in most circumstances, so if it is at all possible to turn the network into a classifier, you may see an improvement.

Not entirely correct; 1) accuracy in the technical sense used in ML has to do with classification (and not regression) problems, where MSE loss would be completely meaningless 2) in regression settings, it is very common to use the loss function itself (MSE, MAE  RMSE etc) also as the "business" metric, in contrast with classification problems where this is never done.

Just as a general remark, notice that technically we don't use the term "accuracy" for regression settings, such as yours - only for classification ones.

If RMSE is 'in the units of the quantity being estimated', does this mean we can say: "The network is on average (1-SQRT(0.019))*100 = 86.2% accurate"?

No.
The advantage of the RMSE, as you have correctly quoted, is that it is in the same units with your predicted quantity; so, if this quantity is, say, USD, you can say (to the business user) that the error of the model is 0.019 USD, and this can be perfectly fine by itself. But you cannot convert it to a percentage - it would be meaningless.
If required to give the performance of a regression model in a percentage, your best option would be the Mean Absolute Percentage Error (MAPE).


text:
Unable to overfit using MLP
I'm building a 5-class classifier with a private dataset. Each data sample has 67 features and there are about 40000 samples. Samples of a particular class were duplicated to overcome class imbalance problems (hence 40000 samples).
With a one-vs-one multi-class SVM, I am getting an accuracy of ~79% on the validation set. The features were standardized to get 79% accuracy. Without standardization, the accuracy I get is ~72%. Similar result when I tried 50-fold cross validation.
Now moving on to MLP results,
Exp 1:

Network Architecture: [67 40 5]
Optimizer: Adam
Learning Rate: exponential decay of base learning rate
Validation Accuracy: ~45%
Observation: Both training accuracy and validation accuracy stops improving.

Exp 2:
Repeated Exp 1 with batchnorm layer

Validation Accuracy: ~50%
Observation: Got 5% increase in accuracy.

Exp 3:
To overfit, increased the depth of MLP. A deeper version of Exp 1 network

Network Architecture: [67 40 40 40 40 40 40 5]
Optimizer: Adam
Learning Rate: exponential decay of base learning rate
Validation Accuracy: ~55%

Thoughts on what might be happening?

Such results are not unexpected. 67 dim vector is small, multiclass SVM would likely produce better results then NN. NN start to shine with dimensionality >=1000. If you actually want overfit, just for fun, you probably need bigger network with 128 to 512 layer size, at least 3 layers (not counting softmax), one 1024 layer may work too. NN should have wide hidden layers, at least in the bottom.

Does your deep layer (exp 3) have skip connections / residual connections? If not, then the gradient is likely vanishing, and early layers aren't learning anything much useful

I would use SGD over Adam, as my experience with Adam on small network and small dataset is that it finds a worse optimum than SGD. Additionally, what batch size are you using for neural network? It affects the optimum as well. Smaller gives better generalization, so try something small like 8.

Would be helpful to know if exp1/exp2 is overfitting the data more so than exp3. What is both the training accuracy and test accuracy for exp1, exp2, and exp3?

I guess you are using linear activation functions,  maybe you are not initializing your weights, or you are regularizing your model enough.
Initialize weights with glorot, insert dropout layers in between, use Relu as your activation function, stop the training process based on Early Stopping. and just experiment with one hidden layer.
side note: if you use Adam, don't mess with the learning rate.
in SGD optimizer you could use decay because there is a single learning rate for all weight updates and the learning rate does not change during training.
In Adam, the learning rate is maintained for each network weight (parameter) and separately adopted as learning unfolds.

I'm using pytorch to model MLP. I'm assuming weight initialization is handled by pytorch. Also, I'm using ReLU and regularization is 0 by default in pytorch.
Also, what sense does it make to use dropout layer when I'm not able to overfit on train data?
I'll look into the learning rate aspect you mentioned.

For the multiclass SVM, there will be an ensembling effect since you are learning 5*4=20 1vs1 classifiers. It could be an interesting experiment to try the same thing with simple neural networks. Also, since you are standardizing the inputs you could try tanh activations on the first layer after the input. I presume you are using softmax on the output layer.


text:
Over- and underestimations of the lowest and highest values in LSTM network
I'm training an LSTM network with multiple inputs and several LSTM layers in order to set up a time series gap filling procedure. The LSTM is trained bidirectionally with "tanh" activation on the outputs of the LSTM, and one Dense layer with "linear" activation comes at the end to predict the outputs. The following scatterplot of real outputs vs the predictions illustrates the problem:
Outputs (X-axis) vs predictions (Y-axis):

The network is definitely not performing too bad and I'll be updating the parameters in the next trials, but the issue at hand always reappears. The highest outputs are clearly underestimated, and the lowest values are overestimated, clearly systematic.
I have tried min-max scaling on inputs and outputs and normalizing inputs and outputs, and the latter performs slightly better, but the issue persists.
I've looked a lot in existing threads and Q&As, but I haven't seen something similar.
I'm wondering if anyone here sees this and immediately knows the possible cause (activation function? Preprocessing? Optimizer? Lack of weights during training? ... ?). Or, and in that case, it would also be good to know if this is impossible to find out without extensive testing.

Try using sigmoid layer in the last layer instead of linear activation.

RNN is a deeply non-linear function over time, how the black linear line is deduced?
Assuming you are doing just linear regression if the least square error was used as the loss function, it will have a probabilistic interpretation
$$y^{(i)}|x^{(i)};\theta \sim \mathcal N(\theta^Tx^{(i)}, \sigma^2)$$
$Y$ is conditioned on $X$ parameterized by $\theta$, with a Gaussian distribution, thus for every data point $x$, there is a corresponding $y$ which if you are doing maximum likelihood estimation is just the mean of the Gaussian, hence variance is introduced to express the noise. if you are doing mini-batch is not guaranteed to reach a global minimum.
Side note: If you normalize the data with min-max scaling, please make sure to use only the train set, if you include the dev/test set, you are doing some kind of data snooping, generalization error will be biased.

The black line is a reference 1:1 line. The blue dots are predictions vs. true values. I do use mini-batches. MSE is the loss function and standard adam is the optimizer. Implementation is done in Keras with TF backend. I'm not seeing clearly yet what your answer states?
And yes, scaling parameters were calculated based on the training set only.

Umm, I see, normalize the inputs to have zero mean and unit variance instead of min-max scaling. augment the LSTM with Mixture Density Outputs as proposed in Generating Sequences With Recurrent Neural Networks (Grave-2013).

Be careful of the output formulation, if you could define relative outputs instead of absolute real values, it will be way better.

What do you mean by relative as opposed to absolute real values? min-max scaling or standard scaling was already included in the procedure.

Use the relative difference between subsequent timesteps instead of their corresponding absolute values, e.g. x2= x2-x1, don't forget to renormalize them to have zero mean unit variance


text:
What does the argmax of the expectation of the log likelihood mean?
What does the following equation mean? What does each part of the formula represent or mean?
$$\theta^* = \underset {\theta}{\arg \max} \Bbb E_{x \sim p_{data}} \log {p_{model}(x|\theta) }$$

Hi Arash. This is a legitimate question. However, next time, please, provide some context, that is, describe where you saw this formula.

This equation and more information of it can be found in Expectation Maximization Wikipedia site and the explanation there was as follows (formula there in two parts):

Some more explanation from same page:

In statistics, an expectation–maximization (EM) algorithm is an iterative method to find maximum likelihood or maximum a posteriori (MAP) estimates of parameters in statistical models, where the model depends on unobserved latent variables. The EM iteration alternates between performing an expectation (E) step, which creates a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters, and a maximization (M) step, which computes parameters maximizing the expected log-likelihood found on the E step. These parameter-estimates are then used to determine the distribution of the latent variables in the next E step.

Mathematically, E in your equation stands for Expectation Value,  x|theta is conditional probability and x~data and model are sub-titles of source of probability in either case. The arg max theta is argument theta that maximizes the equation.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Problem extracting features from convolutional layer where the dimensions are big for feature maps
I have trained a convolutional neural network on images to detect emotions. Now I need to use the same network to extract features from the images and use them to train an LSTM. The problem is: the dimensions of the top layers are: [None, 4, 4, 512] or [None, 4, 4, 1024]. Therefore, extracting features from this layer will result in a 4 x 4 x 512 = 8192 or 4 x 4 x 1024 = 16384 dimensional vector for each image. Clearly, this is not what I want.
Therefore, I would like to know what to do in this case and how to extract features that are of reasonable size. Should I apply global average pooling to the activation or what?
Any help is much appreciated!

I think it's okay to let CNN extract a big number of feature maps, then you can reduce the features from it. To reduce features from CNN result, you can do feature selection or feature extraction. Some people prefer to use feature selection and select the most significant feature maps, for examples:

Action unit selective feature maps in deep networks for facial expression recognition
Convolutional Neural Network Simplification Based on Feature Maps Selection

Hope it helps.

In addition to the first answer about feature selection, you could also add a global max or average pooling layer at the end of your network. This would reduce the dimensionality to 512 or 1024. If that's still too much, another option would be to add an additional convolutional layer with reduced channels and then do the global pooling. You will have to experiment with which option is best for your data.


text:
Machine mathematical reasoning by clever substitutions, How to do with AI
I have three equations that relates five variables {a, b, c, r, s} with a sum and two ratios.

Eq. 1: a = b + c;
Eq. 2: s = b / a;
Eq. 3: r = b / c.

Given two values for any of the five variables I get a solution. But, this is not the automation problem I want to solve.
I can have the solution of variable r by simply knowing s. This is solved by a "human algorithm" as follows.

Substitute a of Eq. 1 in Eq. 2.
Divide the second term of the new Eq. 2 by the variable c.
Replace b/c by the expression of Eq. 3.

That means s = r / (r+1).
The questions is -How can an AI algorithm solve this?, i.e. the machine should recognize that given the variable r she can obtain directly the variable s and do no require another variable.

What you're asking here is actually not reasoning but rather symbolic computation.  Sympy is an example of a library which does it.
Explaining computer algebra systems could be a series of lectures and this is too broad for a single question. To give you an idea: (1) represent equations symbolically (2) bring them in a normal form (3) substitute.
Although it is not elegant, you can simplify the problem by brute Force: try all combinations of solving for a variable and substituting it in the other equations.
Related:

How do symbolic math software work? | Mathematics Stack Exchange

I would turn this problem into graph search in which vertices are "equivalent sets of equations" and edges are "valid operations" such as such as substitution, division and so on, which always ensure that all states are mathematically equivalent. A very simple example of a few states and edges (to requote from your question):
State 0:
  Eq. 1: a = b + c;
  Eq. 2: s = b / a;
  Eq. 3: r = b / c.

Operation 0-->1:
  Multiply Eq. 3 by c

State 1:
  Eq. 1: a = b + c;
  Eq. 2: s = b / a;
  Eq. 3: rc = b.

Operation 0-->2:
  Substitute Eq 1 into 2

State 2:
  Eq. 1: a = b + c;
  Eq. 2: s = b / (b+c);
  Eq. 3: r = b / c.

Operation 2-->3:
  Rearrange Eq 3 so B is on right hand side.

etc etc

Depending on how many operations you have, this does mean that the graph could be quite large, so you may not want to generate the entire graph (actually, the graph will be infinite if the number of operations allowed is unlimited). It's also possible that the graph will contain cycles, since there may be multiple ways to reach the same state.
You will have to think about how to develop data structures for representing the states and operations. Since this is a search problem, it will help to define what your "goal" states look like (e.g. states where s is defined in terms of r).
To search the graph, I recommend one of these approaches:

Breadth-first search: this will find the shortest number of operations needed to reach any goal states
Best-first search: if the graph is too large for breadth first, you may need to guide the search to the goal states using a heuristic, e.g. you could rate states by how few variables are needed on the LHS of equations or something like that -- some experimentation will be needed here.

The exact choice of search algorithm will depend on how big the graph is, which in turns depends on how many operations/edges you want.


text:
Why do we need both encoder and decoder in sequence to sequence prediction?
Why do we need both encoder and decoder in sequence to sequence prediction?
We could just have a single RNN that, given input $x$, outputs some value $y(t)$ and hidden state $h(t)$. Next, given $h(t)$ and $y(t)$, the next output $y(t+1)$ and hidden state $h(t+1)$ should be produced, and so on. The architecture shall consists of only one network instead of two separate ones.

Could you maybe draw a diagram of your suggested RNN architecture? It is not 100% clear to me what you mean.The encoder/decoder architecture used in seq2seq language translation already uses a feedback loop from output to input in the decoder. I think you are asking why you cannot use the same network/parameters for both stages? But I'd like to see a picture or something more concrete in order to understand what you are proposing

Precisely, why not just use one network? I updated the original post by describing what I mean

Sorry I still don't get your proposed architecture. You don't give indication of time steps for x, which is also a sequence. Are you suggesting that you run the network just like a seq2seq network, where you feed in a sequence of $x$ (ignoring the predictions), then after some point - perhaps after an end token - start reading $y$ and feeding that in to the same inputs as $x$? What if $x$ and $y$ are sequences of different things? Are you proposing to input "blank" $y$ whilst feeding in $x$, then blank $x$ when feeding in $y$?

I am assuming batch size of 1. So, you feed in x1 (the first training sample), ignore the output and just keep the last hidden state of the RNN. Then, you run a loop that goes up to some max_len value and starts producing the output values based on the last hidden state and prev output value. Then you repeat the process for the next training sample x2. x is sequence of numbers and y(t) (at single time step) is just a number. Hope this clears things up

In your case do $x$ and $y$ represent the same kind of measurement? Typically you would use a seq2seq model if that was not the case (e.g. English and French words are not the same data type). However, you might use your proposed architecture if $x$ and $y$ were from the same series and your goal is to predict a continuation of the series - e.g. a financial series or NLP language model of a single language.

Thanks, I convert the output in the representation space of the input. The output consists of two values that have separate output vocabulary. My input space is word embeddings. Output space is only 10 words (so I have output space ranging from 0 to 10) Later I convert the output number to a word in the input space since there is one-to-one mapping. Is this ok?

(Old question, I know...)
It is not that we need both an encoder and decoder for sequence-to-sequence models - this decoupling of "reading" and "generating" just works better very often.
Example for Sequence-to-sequence without two RNNs
To prove my point above, here is an example from machine translation. Current machine translation systems are sequence-to-sequence models, and virtually all models have the bipartite structure of encoder and decoder.
Approaches like Eager Translation break this implied convention. They learn translation models that do not encode and decode with separate RNNs, but at every time step 1) read a source token and 2) produce a target token - with a single RNN.
Why encoder-decoder works better very often
Sequence-to-sequence modeling with encoder-decoder structure almost always implies attention in-between encoder and decoder. Attention relays information between the encoder and decoder, in the sense that every time the decoder has to generate the next item in the target sequence, an attention network computes a dynamic, useful "summary" of all encoder states.
This attention summary is different and recomputed for every decoding step. On the other hand, encoding the source sequence is done only once and then all encoder states are kept in memory.
The ability to have a direct view of the source sequence (using as a proxy the entire sequence of encoder states) via attention is what makes the encoder-decoder approach superioir to a single RNN.
In comparison, a single RNN only has a direct view on one element of the input sequence. Some interesting scenarios for a single RNN:

At every time step, read one source token, then write one target token: Previous elements in the source sequence are represented only in lossy recurrent states, while future elements cannot be accessed at all.
First read all source tokens, then write all target tokens.: the meaning of the entire source sentence has to be compressed into a fixed-size recurrent state vector.


text:
Facial Recognition + Database + Compare & Identify - is it complicated?
The last week I've been looking for freelancers who are able to do this project for me but they weren't that experienced in it, so I would like to know whether my idea is complicated or is it their lake of experience.
Scenario:
1. The facial recognition system will be installed on a vertical screen where a camera would be attached to it and it will be assigned on the entrance of the room.
2. Once a visitor come to the entrance, and looks at the screen, a text on the screen would say: " Welcome! It seems like it's your first visit! Please enter your name then a keyboard should popup"
3. The visitor would enter his first name into the keyboard, and it would be saved alongside his face in the database, and it would say thank you, {name}.
4. If the same visitor visits again, the system should say: " Welcome back {name}, happy to see you again"!

Depending on the level of accuracy/speed you want your system to have, you could try using an existing python API. First thing I found on google, the README looks adapted to your project. Otherwise ask your question with more accurate wanted specifications perhaps.


text:
Face liveness detection using face landmark points
How to detect liveness of face using face landmark points?
I am getting face landmarks from android camera frames. And I want to detect liveness using these landmark points.
How to tell if a human is making a specific movement that can be useful for liveness detection?

One approach would be to obtain labeled ("yes/no") samples, e.g. via Mechanical Turk, it costs though) and approach it as supervised learning task. E.g. a neural network will discover itself through the learning what features (landmark positions) are helpful for detection.
There should be examples around for mood detection etc.

BioID has a liveness detection algorithm that you can test here and it is free!
You can get their model at their GitHub. I think it is open-source (check the license).
Their model relies on facial landmarking. They also have algorithms for:
- Face Verification
- Photo Verification
- Cloud Based Solutions in BWS

Detecting liveness of a face using face landmark points is a common technique in computer vision-based liveness detection systems. It involves analyzing the movement patterns of facial landmarks to distinguish between a live human face and a static (e.g., printed or digital) representation. There are various methods you can employ to achieve this. Here's a general approach you can follow:
1: Collect a Dataset: First, you need a dataset of face images or video frames with corresponding ground truth labels indicating whether each face is live or fake (spoofed). This dataset should cover a variety of facial expressions, movements, and environmental conditions.
2: Face Detection and Landmark Extraction: In your Android app, use a face detection algorithm (e.g., Haar cascades, Single Shot Multibox Detector (SSD), or deep learning-based face detectors) to locate and extract facial regions. Then, a face landmark detection model is applied to obtain the key facial landmarks' coordinates.
3: Define Liveness Indicators: Identify specific facial movements or characteristics that indicate a live face. Some common liveness indicators include blinking, head rotation, talking, eye movement, and facial expressions.
4: Temporal Analysis: Many liveness detection methods rely on the temporal analysis of facial landmark movements over a sequence of frames. By tracking the movement of facial landmarks over time, you can assess the consistency and dynamics of facial expressions.
5: Feature Engineering: Extract relevant features from the landmark data. For instance, you can calculate the mean displacement of the facial landmarks over time or the standard deviation of landmark positions.
6: Train a Classifier: Use the labeled dataset to train a machine learning classifier (e.g., SVM, Random Forest, or deep learning models like CNN) to distinguish between live and fake faces based on the extracted features.
7: Validation and Testing: Split your dataset into training and testing sets. Evaluate the performance of your liveness detection model on the testing set using metrics like accuracy, precision, recall, and F1-score.
8: Model Optimization: Iterate on your model to improve its performance by adjusting hyperparameters, trying different feature representations, or experimenting with different classifiers.
9: Deployment: Integrate your trained liveness detection model into your Android app, where you can use it to assess the liveness of the face in real-time camera frames.
I also Found KYChub Face Liveness solution, you can check here, hope this will you

authors: Андрей Винтер


text:
What is the most statistically acceptable method for tuning neural network hyperparameters on very small datasets?
Neural networks are usually evaluated by dividing a dataset into three splits:

training,
validation, and
test

The idea is that critical hyperparameters of the network such as the number of epochs and the learning rate can be tuned by testing the network on the validation data while keeping the test data completely unseen until a final evaluation that happens only after the hyperparameters have been tuned.
However, if the amount of data is very small (e.g. 10-20 examples per class), then dividing the dataset into three splits may negatively impact the model due to lack of training data, and two splits is therefore preferable. A two split approach that makes a reasonable amount of data available for training is ten-fold stratified cross validation.
My question is -- is it statistically sound to tune hyperparameters by repeatedly evaluating hyperparameter sets using cross validation? Keep in mind that there is no held-out test data in this case, as the amount of available data is too small.  I'd like some evidence/citations if possible showing that specifically for small datasets, this is the best approach for estimating the best hyperparameters that lead to the best generalizable model. Or if there is another approach that is better, I'd like to learn about that too.


text:
Any RL approaches for this 2D space optimization problem?
I have a list of rectangles, they are in a certain order in 2D at the beginning. The task is to move them to get the boundary (rectangular) of the minimal area. It's OK to push off the dotted border as long as the area is minimal.
The starting state may look similar to this (top view):

Any reinforcement learning approaches to this problem? I'm thinking of some actions called 'rotate 90 degs', 'push east', 'push west', 'push south', 'push north', but these actions are still not clear how to be applied, which to push, how far to push.
The 2D state can be mapped to a grid of zeros (free) and ones (occupied) to utilize conv2D layers. Before feeding to conv2D, all rectangle coords should be translated to make the ($x_{min}$,$y_{min}$) be at the origin.

Why do you need to use reinforcement learning and CNNs specifically for this? This is the 2D stock cutting problem with rectangles, which can be solved using metaheuristics and an objective function (to minimise) that is proportional to the maximum height of the structure.

i know it can be solved near perfect with regular algorithms, just wanna try RL for this problem


text:
Dissection of a depth map
I am curious about how depth maps work. While searching I came across this website which contains some images and their depth maps. I took this depth map and tried to study it using a python pillow.

from PIL import Image
import numpy as np

image = Image.open('elephant_depth_s.png')
img = np.asarray(image)

print(img.shape)

The depth map shape is (400, 400, 3) with 3 channels. Contrary to my assumption this depth map has three channels instead of one. Even though most of the values are zeros some are not. This len(np.where(img>0)) shows that all the channels have some values greater than zero. My question is;

In color images, RGB channel values are used for creating corresponding color pixels. Example  RGB (255,255,0) creates yellow.

In this depth map how these three channel corresponds to the depth?
Can you please give us some more information on depth maps and their real-world applications?

Depth maps are created using principles of photometry (method of measuring light).
The depth maps (rather images) you took from the website are "images" not exact depth "maps". So by default when you pull out a png image from a webpage, it will be saved in "RGB". That is the reason you got an array with 3 layers. In practice, it will always be a single layer that simply shows relative brightness at each point.
The webpage you referred to is talking about "ray tracing" using software called PoVRay (Persistence of Vision Ray Tracing). What it does is creates a 3D surface with a source of light at a point in the frame and simply measures the intensity of light falling on the surface. Remember this is 3D. When you capture this surface from a point using a camera you get a 2D image that represents a "depth map" as seen from a point of reference with respect to the source of light.
A depth map has plenty of applications in computer vision, photography, and ray tracing. Illuminating a frame, measuring depth are few such applications.


text:
The results changed even though seed is fixed
I am using a reinforcement learning model for some tasks.
and for the model, I am using stable_baselin3 and for the environment, I am using the gym.
I made a small change in the environment and the results became poor now when I rolled back that change the results didn't match the previous results even though I had made no change in the parameters or data.
The seed of the model is fixed so there is no chance that this could be due to random initialization and I have tested this on my model before by running it multiple times to see if gives the same results for the same seed and it did.
I checked that version of NumPy and the torch I used didn't change during my two runs.
and I also tried changing them manually and the results were the same for all the versions.
I am sure that the change I made is rolled back properly because it was a very small change.
Can anyone suggest a list of things that can cause results to change that will help me debug the problem?
I am setting seed for NumPy, torch(used by stable_baseline3), and random. I am not running anything in parallel. I am using the PPO model of stable_baseline3.
I am using below-given function for seed, it is provided by the stable_baseline3 library.
def set_random_seed(seed: int, using_cuda: bool = False) -> None:
    """
    Seed the different random generators.

    :param seed:
    :param using_cuda:
    """
    # Seed python RNG
    random.seed(seed)
    # Seed numpy RNG
    np.random.seed(seed)
    # seed the RNG for all devices (both CPU and CUDA)
    th.manual_seed(seed)

    if using_cuda:
        th.backends.cudnn.deterministic = True
        th.backends.cudnn.benchmark = False

If you use NumPy, PyTorch and Regular python, you may well have 3 (or more) seeds needing setting to manage all the PRNGs. You don't say which seeds you are setting? Also, some libraries cope badly with RNG seeds in multi-threaded code, so worth mentioning if you are using multi-threading or multi-process to run environments in parallel.

@NeilSlater Thanks for the reply and suggestions. The thing is I checked running the code multiple times before, and results never changed for the same data and parameters, and I am not able to re-create this again it just happened once.

This seems to be a programming issue, so it's off-topic here. We focus on the theoretical aspects of AI, which also includes RL, of course. See https://ai.stackexchange.com/help/on-topic for more details.


text:
Which machine learning algorithm can be used to identify patterns in a large file of numbers?
I'm new to machine learning and have many questions, but today I want to know if my case can be solved by machine learning, and if the answer is yes, I would like to know what to learn first and which lessons should I follow to accomplish that.
So my case is such: I have numbers, from 0 to 65.535. These numbers are randomly written in the doc (text file, any format). For example: ,  15.623, 14, 0, 64.322, 5, 5, 15.623, 14, 0, 64.322, 5, 35.2323, 123, 532, 5.764, and etc. They are not limited in the quantity and can be repeated.
I want to take these numbers, find patterns inside the whole file and shrink the data, but remember the positions for reverse engineering.
For example: 5,  15.623, 14, 0, 64.322, 5, 5, 15.623, 14, 0, 64.322, 5, 35.2323, 123, 532, 5.764.
I can see that 5 is followed by 15.623 in 2 cases, and their positions are 0, 1, and 7, 8. I want to assign 5, 15.623 a new character with a marker (not to confuse with the original numbers) = *1.
So my new text would look like this: *1, 14, 0, 64.322, 5, *1, 14, 0, 64.322, 5, 35.2323, 123, 532, 5.764.
So I will have a new file, much smaller in character count, and another code where I save the assigned numbers to new characters.
The reason I want to use machine learning is the following: I want to reduce character count in my original file and finding patterns in millions of numbers can drastically reduce the count. Also, there will be MANY files with millions of numbers inside and I want to train the AI to faster identify patterns and deliver the best method possible.
How to do it, what I should learn and what direction I should take to accomplish that?
Mention in the comments if you need any additional information.

Hello. I think your solution is almost similar to Huffman coding for lossless compression (using strings of characters instead of single ones). Take a look here, you might find some ideas:
https://www.reddit.com/r/algorithms/comments/hsolg3/multicharacter_huffman_coding/ https://www.geeksforgeeks.org/huffman-coding-greedy-algo-3. I think using AI here would be computationally expensive compared to traditional algorithms.


text:
How to align or synchronize Youtube caption with audio accurately
I need to use the automatic caption from Youtube to precisely isolate excerpts from the video aligned to text and generate the dataset to train a model in French.
So I've already written the script, but when I compare the audio with the matching text, I noticed that the text is often delayed (positive or negative). For example, the text reads "1 2 3 4" and the audio says "0 1 2 3" ("0" comes from the previous clip).
If you have a look at a Youtube video in French, when you click on "open transcript", you can also notice this delay.
Here is an example that is very noticeable on short clips: The audio says "conditions de travail" whereas the transcript reads "de travail".
I measured the delay in Audacity and it is not consistent across the clips. Please note that it does not seem to happen in English videos.
If I use Google Speech Recognition in Python (recognize_google) on audio clips, there are no such delays (also because the clips are already separated) but the punctuation is missing which is not good for training my model.
Why can't Google align more accurately the audio and the text (caption)?
Can you suggest a better way of aligning audio with text?


text:
Is there any way to force one input have more effect on model?
Now I am working on building a deep learning model for a regression problem. I used 50 inputs and try to add one new categorical input. The problem is that this one input is much more important than other inputs. I want to make it more influential than others and all I can think of now are the following three.

just add at first layer as other inputs
Add new categorical input to each layer (Now model has 5 layers)
Fit it to the embedding layer first and increate its dimension and concatenate it with other inputs.

Do these seem fine and are there any other ways to give more power to one input?

loss reweighing


text:
Why Acme is using own uniform initializer?
Why is Acme using own initializer for both tanh and ELU, when commonly used for tanh is Xavier and for ELU is He initializer? What mathematics is behind them?
Here is the code.
uniform_initializer = tf.initializers.VarianceScaling(
    distribution='uniform', mode='fan_out', scale=0.333)


text:
Generating automatic sports commentary (NLG)
I am trying to develop a "simple" announcer for sports segments that mainly consists of events like goals, fouls, substitutions, and many other events that could happen in many sports. The idea is that I already have key info like the player who does the action, the location in the court, the time that it takes place, and more extra info. I also have information like the sport being played and the type of event, so this task is purely focused on NLG.
The naïve idea that I had was to extract commentaries of soccer, which can be found on so many websites, and extract the key info of these commentaries that will act as the ground truth in the model for getting the input, i.e.,:
['football', 'goal', 'Bob', 'fourth minute'] -> Goal! that was a nice goal from Bob in the fourth minute of the match.
I would say that the first two words are being used for steering the model to generate sports according to phrases (a cricket player kicking the ball doesn't make sense).
The comments generated by the fine-tuned model on this input-output are acceptable.
The problem is that I have to build a dataset for many sports (or at least 2 with the same quality as soccer ones like in Flashscore) and I can't find any.
I have also been looking for Plug and Play methods to generate sentences.
What do you think? Is fine-tuning a must-do in this situation or can it be thought of in another way?

Hello. Could you please put your specific question in the title? From the title, it seems that you want to generate comments, but it's not specific enough.

I am not sure you need deep learning. The main question is how do you represent the sports events inside the computer. Do you have some frames? See these slides. You could be interested by RefPerSys whose objects are frames

Your question should tell what is the input of your system? Do you parse English written text (e.g. coming from Reuters)? Do you have a video camera, with your software "understanding" the event?

Since sport commentaries are a fairly restricted domain, and the language does not vary much, I would go for a canned text approach.
Analyse what kind of events you get, and what variables you're dealing with. Then write some template sentences with placeholders for the variables. The more you write for the same data, the more varied your text will be. You could then structure them by pre-conditions, such as a goal that equalises a previous goal, as you will want to say something different than if it was a repeated goal by the same team.
You could probably look at interactive fiction for tools, such as Twine -- effectively you treat your sporting event like an interactive story, that is driven by what happens in the sporting event you are describing.
You will have the effort of writing the templates, but in return your output text will be of higher quality, and you have more control over what is generated than if you were using a machine learning approach.

Thanks for answering! I'll consider your idea for sure. Nevertheless, if I were to consider a deep learning approach, do you think that this can be accomplished without relying on building a dataset for all sports? I still consider than fine-tuning a model would be the optimal way on terms of time and all based on my idea posted in the question

There is no free lunch -- I doubt that you will get any good results without putting in your domain knowledge.

Thanks, Oliver. I just wanted to make sure of that dependency. Cheers.


text:
Difficulty in agent's learning with increasing dimensions of continuous actions
I have been working on some RL project, where the policy is controlling the robot using its joint angles.Throughout the project I have noticed some phenomenon, which caught my attention. I have decided to create a very simplified script to investigate the problem. There it goes:
The environment
There is a robot, with two rotational joints, so 2 degrees of freedom. This means its continuous action space (joint rotation angle) has a dimensionality of 2. Let's denote this action vector by a. I vary the maximum joint rotation angle per step from 11 to 1 degrees and make sure that the environment is allowed to do a reasonable amounts of steps before the episode is forced to terminate on time-out.
Our goal is to move the robot by getting its current joint configuration c closer to the goal joint angle configuration g (also two dimensional input vector).
Hence, the reward I have chosen is e^(-L2_distance(c, g)).
The smaller the L2_distance, the exponentially higher the reward, so I am sure that the robot is properly incentivised to reach the goal quickly.
Reward function (y-axis: reward, x-axis: L2 distance):

So the pseudocode for every step goes like:

move the joints by predicted joint angle delta

collect the reward

if time-out or joint deviates too much into some unrealistic configuration: terminate.

Very simple environment, not to have too many moving parts in our problem.
RL algorithm
I use Catalyst framework to train my agent in the actor-critic setting using TD3 algorithm. By using a tested framework, which I am quite familiar with, I am quite sure that there are no implementational bugs.
The policy is goal-driven so the actor consumes the concatenated current and goal joint configuration a= policy([c,g])
The big question
When the robot has only two degrees of freedom, the training quickly converges and the robots learns to solve the task with high accuracy (final L2 distance smaller than 0.01).
Performance of the converged 2D agent. y-axis: joint angle value, x-axis: no of episodes. Crosses denote the desired goal state of the robot.:

However, if the problem gets more complicated - I increase the joint dimensions to 4D or 6D, the robot initially learns to approach the target, but it never "fine-tunes" its movement. Some joints tend to oscillate around the end-point, some of them tend to overshoot.
I have been experimenting with different ideas: making the network wider and deeper, changing the action step. I have not tried optimizer scheduling yet. No matter how many samples the agent receives or how long it trains, it never learns to approach targets with required degree of accuracy (L2_distance smaller than 0.05).
Performance of the converged 4D agent. y-axis: joint angle value, x-axis: no of episodes. Crosses denote the desired goal state of the robot.:

Training curve for 2D agent (red) and 4D agent (orange). 2D agent quickly minimises the L2 distance to something smaller than 0.05, while the 4D agent struggles to go below 0.1.:

Literature research
I have looked into papers which describe motion planning in joint space using TD3 algorithm.
There are not many differences from my approach:
Link 1
Link 2
Their problem is much more difficult because the policy needs to also learn the model of the obstacles in joint space, not only the notion of the goal. The only thing which is special about them is that they use quite wide and shallow networks. But this is the only peculiar thing.
I am really interested, what do you guys would advise me to do, so that the robot can reach high accuracy in higher joint configuration dimensions? What am I missing here?!
Thanks for any help in that matter!

Are all episodes the same length regardless of getting close to the goal state? E.g. they are all 75 time steps long, and a perfect, lucky agent could score +75 total reward if it started in the goal state and did not move?

Yes, the setup is exactly as you describe it.
This perfect, lucky agent could observe that situation in theory. But the probability of such an episode is so low that I do not consider this harmful by any means. Could you elaborate on your comment? Do you think there might be a problem with the setup?

I don't think the possible +75 episode by accident is a problem. I just wanted to confirm that the episode does not end due to getting the correct value, and that is the extreme case. I originally wrote an answer assuming that reaching the goal would terminate an episode - that answer is not correct for your setup, so I deleted it

Well, now when you explained your point of view: I have currently this "termination on success" condition implemented for 4D robot. But to be honest, IMO, it does not change anything, because the required L2 distance is so low, that the "success" condition never gets triggered in the 4D case.
EDIT: So to be clear, agent always reaches 75 steps before termination and never reaches the success state. It only collects the reward based on the exponential of l2 distance.

"termination on success" will be a major problem for you, and if you decide you want to keep that rule, you will need to change your reward function. When you say it "never reaches the success state" is that strictly true during all training episodes, or are there a small fraction of times when it happens (perhaps apparently by accident and early on during training)?

Yes, its strictly true, I keep logging that.
Why termination on success is a problem? I don't see how this could harm the training.

Termination on success is a problem in your case - with your reward signal - because it zeroes out the remaining rewards. In your reward scheme, the agent is incentivised to get close to the goal but not too close otherwise it would lose rewards from remaining time steps. There are a few ways to fix this - for instance you could end with +75 reward, guaranteed to be better than anything the agent could get by waiting. However, you are not hitting that problem yet for some other reason that I cannot quite see.

I understand your point, I see how this can be a problem in theory, but it does not explain why 2D agent "gets it right" and 4D does not.


text:
Can XGBoost solve XOR problem?
I've read that decision trees are able to solve XOR operation so I conclude that XGBoost algorithm can solve it as well.
But my tests on the datasets (datasets that should be highly "xor-ish") do not produce good results, so I wanted to ask whether XGBoost is able to solve this type of problem at all, or maybe I should use a different algorithm like ANN?
EDIT: I found a similar question with a negative answer here.
Could someone please confirm that XGBoost cannot perform XOR operation due to "greedy approach" and whether this can maybe be changed in parameters?


text:
How do you measure multi-label classification accuracy?
Multi-label assignment is the task in machine learning to assign to each input value a set of categories from a fixed vocabulary where the categories need not be statistically independent, so precluding building a set of independent classifiers each classifying the inputs as belong to each of the categories or not.
Machine learning also needs a measure by which the model may be evaluated. So this is the question how do we evaluate a multi-label classifier?
We can’t use the normal recall,  accuracy and  F measures since they require a binary is it correct or not measure of each categorisation. Without such a measure we have no obvious means to evaluate models nor to measure concept drift.

I like this discussion of accuracy: https://stats.stackexchange.com/q/312780/247274. Two example of the strictly proper scoring rules that Kolassa discusses are crossentropy loss (log loss) and Brier score.

Your intuition is correct. We do use other metrics for multi-label classification. The meaning of evaluation itself changes. Apart from grading the classifier on whether it classifies correctly or not, we also have to penalize it, if it chooses the wrong class appropriately. You could use the following metrics:

micro/macro averaging Recall/Precision, etc.
Hamming Loss
Subset Accuracy

Maybe you should briefly describe how they are calculated (or what's the intuition behind them) or, at least, provide a link to a research paper (or reliable article) that explains these metrics/measures.

Even with a binary classifier, one number does not fully represent the behaviour - the confusion matrix has three degrees of freedom. Even more with a multi-class problem, it is best to print our the whole confusion matrix. Then you can pick up problems like "large class A is well classified, many Bs are wrongly classified as C, and the few Ds are wrongly assigned to A,B, or C".
Even better, printing the confusion matrix helps you think about what the real business goals are: which of these errors matters most in practice?


text:
Is stable learning preferable to jumps in accuracy/loss
A stable/smooth learning validation curve often seems to keep improving over more epochs than an unstable learning curve. My intuition is that dropping the learning rate and increasing the patience of a model that produces a stable learning curve could lead to better validation fit.
The counter argument is that jumps in the curve could mean that the model has just learned something significant, but they often jump back down or tail off after that.
Is one better than the other? Is it possible to take aspects of both to improve learning?

There is an approach to machine learning, called Simulated Annealing, which varies the rate: starting from a large rate, it is slowly reduced over time. The general idea is that the initial larger rate will cover a broader range, while the increasingly lower rate then produces a less 'erratic' climb towards a maximum.
If you only use a low rate, you risk getting stuck in a local maximum, while too large a rate will not find the best solution but might end up close to one. Adjusting the rate gives you the best of both.

Thank you Oliver. It hadn't occurred to me that having too low a learning rate could result in getting stuck in a local maximum. I'm aware of learning decay in Tensorflow and have been using it but not aware how important it is to use both appropriately. Am I right in saying we would want big jumps early on and then a stable learning later as it optimises on hopefully the best maxima?

@OliverP Yes, exactly.

That should have been local minimum, @OliverP

@DavidHoelzer No, local maximum is correct.

If you have an erratic loss landscape, it can lead to an unstable learning curve. Thus, it's always better to choose a simpler function which creates a simple landscape. Sometimes even due to uneven dataset distribution, we can observe those jumps/irregularities in the training curve.
And yes, those jumps do mean it might've found something significant in the landscape. Those jumps can arise while the model is exploring the multiple local minima of the landscape.
During Machine Learning Optimization, we usually use algorithms like Stochastic Gradient Descent and Adam to find Local Minima's whereas approaches like Simulated Annealing find global minima. There have been multiple discussions around why to use local minima's instead of global minima. Some argue that local minima are just as useful as global minima in case of machine learning problems.
Thus, Stable Learning is preferable as it symbolizes that the model is converging to local minima.
References

You can read A Survey of Optimization Methods from
a Machine Learning Perspective by Shiliang Sun, Zehui Cao, Han Zhu, and Jing Zhaoto et al. and read about all the optimization functions commonly used in Machine Learning.


text:
How to extract the main text from a formatted text file?
My idea is to model and train a neural network that receives a text version of a PDF file as the input and gives the content text as output.
Take the scenario:

One prints a PDF file to a text file (the text file does not have images, but has the main text, headings, page numbers, some other footer text, and so on, and keeps the same number of columns  - two for instance - of text);

This text file is submitted to a tool that strips everything that is not the main content of the text in one single text column (one text stream), keeping the section titles, paragraphs, and the text in a readable form (does not mix columns);

The tool generates a new version of the original text file containing only the main text portion, ready to be used for other purposes where the striped parts would be considered noise.

How to model this problem in a way a neural network can handle it?
Update 1
Here are some clarifications on the problem.
PDF file
The picture below shows two pages of a pdf version of a scientific paper. This is just to set the context, the PDF file is not the input for this problem, it is just to understand where the actual input data comes from.

The color boxes show some parts of interest for this discussion. Red boxes are headers and footers. We are not interested in them. Blue and green boxes are content text blocks. Different colors were used to emphasize the text is organized in columns and that is part of the problem. Those blue and green boxes are what we actually want.
Text file
If a use the "save as text file" feature of my free PDF reader, I get a text file similar to the image below.

The text file is continuous, but I put the equivalent of the first two pages of the PDF file side-by-side just to make things easier to compare. We can see the very same colored boxes. In terms of words, those boxes contain the same text as in the PDF version.
Understanding the problem
When we read a paper, we are usually not very interested in footers or headers. The main content is what we actually read and that will provide us with the knowledge we are looking for. In this case, the text is inside blue and green boxes.
So, what we want here is to generate a new version of the input (text) file organized in one single text stream (one column if you will), with the text laid-out in a form someone can read it, which means, alternating the blue and the green boxes.
However, if the original PDF has no footers, it should work in the same way, providing the main text content. If the text comes in three of four columns, the final product must be a text in good condition to be read without losing any information.
Any pictures will be simply stripped off the text version of the paper and we are fine with that.

How many years can you afford spending on such a project?

After a long wait and some digging, I accidently found what I was looking for. In 2015, polish researcher Dominika Traczyk publish an article presenting CERMINE, a solution for the posted problem.
His solution is SVM-based, but the article gives good insights for an alternate Neural Network version.
The article is open access and can be found on the Springer website, while all the source code is available on GitHub.

How to extract the main content text from a formated text file?

I am not sure that just a neural network is the best approach to your problem.
Traditional natural language processing software are using something else, and generally using a complex mix of several techniques. I am supposing you are processing written text available as some file (in a file format you are very familiar with, e.g. OOXML or PDF or HTML5).
Read the wikipage on natural-language understanding and the one on parse trees (or concrete syntax trees).
BTW, you might use LaTeX or the Lout formatter to produce some PDF file. Both are open-source software (easily available on most Linux distributions, including Debian or Ubuntu). I recommend you to try generating some PDF file using them, and experiment on the generated PDF file. And a lot of AI papers are available (as preprints) in PDF form.
You could also use, as a PDF input to experiment your software, this or that draft reports (you might enjoy reading them too...). If in 2021 your software is capable of "understanding" and "abstracting/summarizing" these PDF files, please send me an email to basile@starynkevitch.net explaining (in written English) how you did build your neural network and what is the output of your software.
There are several issues:

extracting the non-textual things (e.g. HTML tags from HTML input, or strings from a PDF file, or some LaTeX one).

detecting the human language used in your text (e.g. French or English or Russian or Chinese). N-gram based techniques come to mind.

having a data structure or database representing a dictionnary of a thousand (at least) of significant words (in English or Russian or whatever human language you are interested in) related to the domain you want to handle (that dictionary would be different if you want to parse weather forecasts or documentation related to the automotive industry, since the word pressure or speed relates to different concepts. Notice also that "weather" and "time" are expressed in French by the same word: "temps" - as in "le temps qu'il fait" for ongoing weather and "le temps qui passe" for the flow of time). A "Queen" is not the same for a chess player and an historian. A program translating -or just analyzing- chess comments from English won't use the same word for translating / understanding "bishop" (in chess, "fou" in French, literally the crazy guy, unrelated to religion; in Russian chess books it would be "слон",  literally an elephant) than another program translating / analyzing historical comments from English (e.g. about Mary Stuart).

modeling inside your software some domain-specific knowledge related to your analyzed text, since you would handle differently weather forecast text to textual comments of chess competitions, or textual exercises in any computer science or programming book (like CLRS). You could use some frame-based representations, like in RefPerSys or in CyC.

building a semantic network representing the input text. I believe you might need some prior one representing domain-specific knowledge in the area of the analyzed text (e.g. a program analyzing comments on chess games needs to know the rule of chess; another program analyzing StackOverflow answers probably needs to know something about operating systems in general). In think that in English "overflow" or "overheating" means very different concepts to software developers and to weather forecasters or climate experts.

Look also for inspiration into this blog of the late Jacques Pitrat. He did wrote an interesting book on your topic.
You might look inside the DECODER European project, and read more about expert systems and their inference engine and knowledge bases.
Your project could give you some PhD.
You certainly need several years of work to achieve your goals. I suggest contacting some academic in your area to be your advisor.
Notice that on Linux the pdf2text software is extracting text from PDF files. It is open-source, but I won't say it is an AI software. However, you could use it thru popen(3). See also regex(7).
BTW, the PDF specification is public as ISO 32000-2:2017 (and is related to PostScript). Get it and read it, and see also this youtube video or this 978 pages document. On Linux, most PDF files can usually be inspected with od(1) or less(1).
My HP Office Pro 8610 printer (connected to a Linux desktop) is capable of printing some PDF and of scanning into a PDF file. But if I print on paper some PDF file and scan it, the PDF file did change a lot, even if visually it looks the same.
Notice that some drawings -or photos- could be embedded in a PDF file, and appear to a non-blind human reader as letters.

AFAIK, when using LaTeX to produce some PDF file, the generated PDF does not know about header and footers and columns. Of course, the LaTeX source file does know about them.

BTW, if your software is capable of using NLP techniques on PDF, could you please send me an email explaining that. I am not asking for your software, just for you to explain me one of its successful runs. Honestly, I am skeptical about you being able to achieve your goal in one year of full time work.

Please do so. When it works, send me an email. My intuition is that a full year of work would be needed, but I hope for you that is is less, or that you have enough money to live during that while working on your interesting problem.

Sorry, but nota really what I'm trying to do. Mu problema hás nothing to do with the information inside the texto, just identifying the text portions.

@AlexSC: please edit your question to define in several paragraphs of written English what exactly is the text portion of a PDF file. In particular, a PDF file can contain photos  of objects having some text of them, or drawings (i.e. figures) with text. An example is  this draft report. What do you think are the text portions of it? Is the figure 1 on page 10 some text of that file?

@BasileStarynkevitch, I just added some clarifications.

It seems similar to extracting a support email question and support email provided answer leaving all the previous responses in those emails and email signatures out. Imagine 1000s of questions and replies in the support box and you want to extract only actual questions and answers leaving all the noise to train ChatGPT. Maybe solving this problem is an image recognition problem as humans we can look at documents and point out where the main content is. So first you teach your model to "see" the data blocks it needs to extract and after you extract what was seen by your model as a target text?

authors: Селенов Егор


text:
Where can I find pre-trained agents able to play games with multiple stages like exploration, dialog, combat?
My goal is to create an ML model to be able to classify different game stages, e.g., dialog with a non-player character, exploration, combat with enemy, in-game menu etc.
In order to do that, I am looking for an agent pre-trained on such a game. I am intending to develop a model using this pre-trained agent to produce a data set (frames-labels) and finally I will use that data set to train a model to classify those different stages.
I could only find a pre-trained model for the Doom, however, it is not much appropriate for my case because it does not have different game stages (it is merely based on running & shooting).
Training my own Reinforcement Learning Agent is a whole another workload in terms of both time and GPU such a game needs.
Any single idea could help me a lot. Thanks!

Welcome to AI:SE! These 2 repos have a collection of pre-trained RL agents on a variety of games: github.com/DLR-RM/rl-baselines3-zoo and github.com/araffin/rl-baselines-zoo

Hello @mugoh thank you for your answer! I cannot see any game related my criteria above. Instead of atari based or simple games, I need an agent trained on more complex game.

My thought is that, at a high level, the problems you identify (dialogue, exploration, combat) all have different structures, and would require different techniques.  (Exploration, for instance, is pathfinding.  Dialogue is natural language processing.  Combat is more like a card game with hidden information.)  So I doubt there is any single ML function that can accomplish them all, but it's an good question, imho, for that reason.


text:
How to recognize sequence of digits in an image
I am learning to program neural networks and others, and I would like to know how I can get the numbers that are in an image, for example, if I pass an image that has 123 written, get with my model that there are 123 written, I have tried to use PyTesseract is not very precise, and I would like to do it with a neural network, my current code is quite simple, it recognizes the digits of the mnist dataset such that:
import tensorflow as tf
from tensorflow.keras import Sequential, optimizers
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
import matplotlib.pyplot as plt

mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

print('train_images.shape:', train_images.shape)
print('test_images.shape:', test_images.shape)
plt.imshow(train_images[0])

train_images = train_images.reshape((60000, 28, 28, 1))
test_images = test_images.reshape((10000, 28, 28, 1))

train_images = train_images.astype('float32') / 255
test_images = test_images.astype('float32') / 255

train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels)

model = Sequential()

model.add(Conv2D(32, (5, 5), activation = 'relu', input_shape = (28, 28, 1)))
model.add(MaxPooling2D((2, 2)))

model.add(Conv2D(64, (5, 5), activation = 'relu'))
model.add(MaxPooling2D((2, 2)))

model.add(Flatten())

model.add(Dense(10, activation = 'softmax'))

model.summary()

model.compile(loss = 'categorical_crossentropy', optimizer = 'sgd', metrics = ['accuracy'])

model.fit(train_images, train_labels, batch_size = 100, epochs = 5, verbose = 1)

test_loss, test_accuracy = model.evaluate(test_images, test_labels)

print('Test accuracy:', test_accuracy)

but I would need to know how I can pass an image with a sequence of digits to it, and that it recognizes the digits in question, does anyone know how I could do it?

I don't know exactly what you're looking for and I didn't read your post carefully, but have you heard of optical character recognition? If not, that's probably something you want to look into.

@nbro Yes, it's what I'm using right now, but I can't make it very accurate

@JohnDoe your code is image classification, won't do object detection of digits

From your question there is no indication that there is any pattern to these digits.  If there were, the recommendation for an LSTM or RCNN would make sense.  In the case of random values, I have found that a two or three layer CNN that then descends through two parallel dense networks does an excellent job identifying CAPTCHA style random characters.  One path is primarily responsible for identifying bounding boxes, the other is determining which characters are present.
There are many other ways to solve this problem.  You might do well to research CAPTCHA solving via neural networks.  More specifically, text based CAPTCHAs.  If your task really is just OCR, then research OCR through neural networks.  The technique I describe here will work but will become cumbersome for a page of text, for example.  In that case a sliding window CNN coupled with a dense layer and an LSTM makes the most sense since you will be dealing with predictable sequences of characters.

Your task is text recognition, however your code is for classification task. So you need to use different approach for that. You mentioned that you're going to give model 123 and get 123. But you can not do that with just convolutional networks. Images with text are sequential, so you need to use CRNN(Convolutional-Recurrent-Neural-Networks), LSTM(Long-Short-Term-Memory), BiLSTM(Bidirectional-LSTM). In most of the research papers there are convolutional networks are being used just for feature extraction stage. For prediction stage they used recurrent units, such as LSTM cells or RNN.

This might be true generally, but I suspect it is not true in the OP case.  If you were looking for words, a recurrent or LSTM network would make sense.  The OP is looking for digits.  There is no indication in the question that these follow any pattern, so should be viewed as random values.


text:
Are there regularisation methods related only to architecture of the CNNs?
Are there any methods of regularisation of deep neural networks, particularly CNNs (or generally ANN but that will also work on CNNs) that are related only to the network's architecture and not the training itself?
I mean maybe something like how deep they are, amount of conv/pooling/fully connected layers, size of filters, size of steps of filters, etc. any pointers that would help with regularisation.
EDIT: To explain deeper what I mean I might add that I am exploring an experimental idea for the training of the CNNs that is not in any way related to typical gradient descent with backpropagation. That is why typical methods related to training will not work. I can see already that the models train satisfactorily on the training set but don't perform that well on a test set and since I didn't figure out any regularization methods for this type of training I thought maybe there are some related to architecture, that the training process will have to abide.

I am not sure I understand your question. If you use e.g. L2 regularization, you're limiting the values that the weights can take, so, in this way, this type of regularization is related to the "architecture" of the neural network. So, can you clarify what you're really asking and why regularization techniques like "L2" or "dropout" are not the answers to your question?

Without more info about your training procedure, it's still not clear (at least to me),  why dropout does not work in your case. Have you already tried it?

@nbro The training is based on evolutionary algorithms with a very specific domain based fitness function without use of standard error metrics such as binary crossentropy or MSE. I can't even imagine where would dropout or L2 go into this process.

maxnorm over weight vectors is a regularisation technique that is not architectural, but may help in your situation


text:
Predict time series from initial non-time dependant parameters
I'm trying to create an algorithm (neural network) that is able to predict a time series from a set of different parameters that are not given through time. Let's say I have a plane flying under the following conditions:

Parameters
Value

Angle of attack
8 degrees

Lateral angle
12 degrees

Wind speed
-20 m/s

Plane speed
200 m/s

From this point, I would like to predict the translational velocities in x-y-z axis for the next 2-3 seconds.
In order to train my model, I have a data base with different initial situations (input) and different motions of the plane (desired output) linked to their initial situation. Therefore, I want to train my model to predict these motions mentioned before, based only on the initial situation described.
In other words, the basics of what I'm trying to do could be summed up as the following:
Parameters describing the initial situation -> Model -> Time series of translational velocities.

I don't think you need a recurrent neural network for this. Why not just train a feedforward model with angle of attack etc as input and translation velocities as output? The size of the output will depend on how frequently you want updates to the velocities. e.g. if the update rate is 0.5 seconds and the network is predicting 2 seconds in advance it could have 12 outputs, being x,y,z values every 0.5 seconds.
You may need to massage the way you represent the outputs to find the best approach. For example, will the network be predicting absolute velocities or just the initial velocities plus changes to velocity over time? If the latter you can always get back to absolute velocities.


text:
How to handle equality constraints in the mutation operation of evolutionary algorithms?
I am new in evolutionary algorithms field. I have a chromosome of 6 variables (real variable), where the sum of these variables is equal to 1.
I am looking for mutation formulas that can generate a new chromosome    respecting the equality constraint: in my case, the sum of new chromosome should always equal to 1.

If X is your 6D vector and m(X) is the mutated version of X, then you can renormalise the mutant back to unity by dividing by the sum of X, i.e. X' = m(X)/sum(X).
However, I encourage you figure out how to mutate a vector while keeping the length of the vector at 1. One way to do this would be to randomly rotate your vector in 6D space. The length should stay the same, and you don't need to renormalise it afterwards.

A random small rotation is a neat idea

I think there are some typos in this answer, if I understand correctly your suggestion. For example, what is $f(X)$?


text:
Could an AI feel emotions?
Assuming humans had finally developed the first humanoid AI based on the human brain, would It feel emotions? If not, would it still have ethics and/or morals?

It depends on what you mean by "feeling emotions". I asked a similar question Is the simulation of emotional states equivalent to actually experiencing emotions? on Philosophy SE, and the answer does appears to hinge on the definition of "feeling" (especially when dealing with the concept of "qualia").

On the last part of your question, if it couldn't feel emotion, then it probably couldn't relate to other beings that have them. An inability (or unwillingness?) to consider other people's feelings is the #1 telltale sign of a sociopath. Whether or not a sociopath lacks morals or just has a corrupted moral code is a deep question unto itself, but suffice to say that either way, they end up dealing with others in a selfish manner. In the AI's case it would probably be the former problem, lacking a moral sense (in the case of humans it's probably the latter) but the result's the same

Machines can't take decisions based on human judgement. http://ai.stackexchange.com/questions/1354/are-there-any-textual-captcha-challenges-which-can-fool-ai-but-not-human/2143#2143

Have a look at the very related questions: Can an artificial intelligence suffer? and Can an AI learn to suffer?.

I have considered much of the responses here, and I would suggest that most people here have missed the point when answering the question about emotions.
The problem is, scientists keep looking for a single solution as to what emotions are. This is akin to looking for a single shape that will fit all different-shaped slots.
Also, what is ignored is that animals are just as capable of emotions and emotional states as we are:
When looking on Youtube for insects fighting each other, or competing or courting, it should be clear that simple creatures experience them too!
When I challenge people about emotions, I suggest to them to go to Corinthians 13 - which describes the attributes of love. If you consider all those attributes, one should notice that an actual "feeling" is not required for fulfilling any of them.
Therefore, the suggestion that a psychopath lacks emotions, and so he commits crimes or other pursuits outside of "normal" boundaries is far from true, especially when one considers the various records left to us from court cases and perhaps psychological evaluation - which show us that they do act out of "strong" emotions.
It should be considered that a psychopath's behaviour is motivated out of negative emotions and emotional states with a distinct lack of or disregard of morality and a disregard of conscience. Psychopaths "enjoy" what they do.
I am strongly suggesting to all that we are blinded by our reasoning, and by the reasoning of others.
Though I do agree with the following quote mentioned before: -
Dave H. wrote:

From a computational standpoint, emotions represent a global state that influences a lot of other processing. Hormones etc. are basically
just implementation. A sentient or sapient computer certainly could
experience emotions, if it was structured in such a way as to have
such global states affecting its thinking.

However, his reasoning below it (that quote) is also seriously flawed.
Emotions are both active and passive: They are triggered by thoughts and they trigger our thoughts; Emotions are a mental state and a behaviourial quality; Emotions react to stimuli or measure our responses to them; Emotions are independant regulators and moderators; Yet they provoke our focus and attention to specific criteria; and they help us when intuition and emotion agree or they hinder us when conscience or will clash.
A computer has the same potential as us to feel emotions, but the skill of implementing emotions is much more sophisticated than the one solution fits all answer people are seeking here.
Also, if anyone argues that emotions are simply "states" where a response or responses can be designed around it, really does not understand the complexity of emotions; the "freedom" emotions and thoughts have independently of each other; or what constitutes true thought!
Programmers and scientists are notorious for "simulating" the real experiences of emotions or intelligence, without understanding the intimate complexities; Thinking that in finding the perfect simulation they have "discovered" the real experience.
The Psi-theory seems to adequately give a proper understanding of the matter.
So I would say that the simulation of emotional states "is" equivalent to experiencing emotions, but those emotional states are far more complex than what most realize.

There is much discussion in philosophy about inner language and the ability to perceive pain (see Pain in philosophy article). Your question is in the area of philosophy and not science. If you define emotion as some state then you can construct simple automata with two states (emotion vs no-emotion). It can be a very complicated state with degrees of truth (percentage of emotion).
Basically, to mimic human emotion you need to make a living human-like organism, and still with todays understanding and technology you will not be able to recognize emotion in it. The only thing you can do is trust when it says "I'm sad". Now we are in the area of the Turing test, which is again philosophy, and not science.

It is certainly possible for AI to theoretically feel emotion.

There are, according to Murray Shanahan's book The Technological Singularity, two primary forms of AI:

1) Human based AI - achieved through processes such as whole brain emulation, the functioning of human based AI would likely be indistinguishable from that of the human brain, and, as a consequence, human based AI would likely experience emotion in the same manner as humans.

-

2) AI from scratch - with this form of AI, based on machine learning algorithms and complex processes to drive goals, we enter into uncharted territory as the development of this form of AI is inherently unpredictable and unlike anything we observe in the biological sample space of intelligence we have access to.
With this form of AI, there is no telling if and how it could experience emotion.

As the question references the former, it is very likely that human-based AI would indeed experience emotion and other human-like characteristics.

Assuming an AI was built out of a mechanical husk, mirroring the human brain exactly; complete with chemical signals and all. An AI should theoretically be capable of feeling/processing emotions.

Are you the Apple's Siri?

Dualists would argue that even then, an AI wouldn't be able to feel emotions, see the philosophical zombie argument and the problem of Qualia.

You should differentiate the ability to process emotions and mimics. Emotions are a much bigger deal than just chemical signals. It's also a matter of perception.

Although I agree with @kvfi that it is also a matter of perception(which depends on lot of factors), I believe with the current pace of advancements in the field of AI, as @Siri said, an AI should be able to feel emotions by taking into account and processing every piece of data related to it.

Emotions are a factor in humans having ethics/morals only because they are a factor in all human learning and decision-making.
Unless you are duplicating a human being exactly, there is no reason to think that an AI will learn the way a human learns, or make decisions in the same way a human makes decisions.
Therefore, whether it "feels emotion" just like we do, or whether it simply responds to outcomes "cost is greater = don't go there", the outcome of ethical BEHAVIOUR could be achieved. An AI could behave perfectly ethically without any need for feeling empathy, shame, etc.
You could also argue that a lot of UNETHICAL behaviour in human beings is driven by emotions, too, and that an unemotional but ethical AI may well do a better overall job than a human being.

This question is more the province of philosophy of mind than of AI, here are some detailed answers to your question from the philosophy SE: Is simulating emotions the same as experiencing emotions?, and What is the problem with physicalism?.
For the record, the accepted answer (by Siri) to the question is not entirely correct (The position in that answer corresponds roughly to John Searle's view on the question, and his is a minority view): Dualists would argue that even with a perfect replication down to the chemical level of brain interactions, an AI still wouldn't experience emotions, as it lacks the purely mental substance/properties that make a mind and not a machine.
On the completely opposite side of the spectrum, functionalists would answer that such a perfect replication is overkill: even a suitably programmed digital computer can experience emotion, particularly if one equips it with higher-order and self-referential states.

It is definitely a philosophical question in regard to simulation of emotions, but removing anthropomorphism, and focusing on fundamental, philosophical principles, it can shown to be mathematical: http://ai.stackexchange.com/a/2877/1671 .  To be truly felt, the emotion would have to arise out of something machines consider important, such as optimization, computing resources, calculability/intractability, and equilibria. However, it surely becomes philosophical again in terms of consciousness, or lack thereof, of the system. [Excellent answer, btw!]

Well, it depends of the level of the AI.
You can create an AI super autonomous with deep learning capabilities and so on, but in the robotic type only.
If you'd create an AI like EVA in the Ex-Machina movie, humanoid form, deep neural transmissions and with cognitive dissonance, then it could feel.
The 'AI' problem its not the chemical and neural transmissions, its the consciousness.

Yes and no. If you fully simulate a human brain and all of its functions, it would probably be able to feel emotions very similar to the way we do.
But we don't have enough capabilities and knowledge to do that, and maybe we could find a "shortcut" - a process that is intelligent without simulating a whole brain. In this case, emotions would probably represented by data values which say "this is good (make it happen again!)", or "this is bad (avoid it!)". This is just a very basic example (there are obviously many more emotions), but it would have a similar function and the AI would have similar solutions to the ones we have. But we don't know - and probably no one ever will know - if this data value 'bad' "feels" the same way for the AI the according emotion would feel to us.

IMHO
Definitely, yes!
Everything that person feels (physically or mentally) can be discovered by chemical signals processing in his body or brain. If we understand the policy and nature of such signals, we can program it.
There are a lot of pseudo-psychology and psychology works on this sphere, if you interested, I can suggest you:

Cognitive Psychology (Robert L. Solso)

describes cognitive apparat of human's mind in a simple words;

The Psychology of Emotions (Carroll E. Izard)

thorougly describes every kind of emotion by its looking on the human (both child and adult) face, low-level cognitive mechanism, related or adjacent emotions;

Books by Paul Ekman ("Telling Lies", "Emotions Revealed",
"Unmasking the Face")

practical detecting of human emotions by microexpressions language on face and body.
Upd 2023-01-05, to give more details
A person forms their own emotions through their own experience, social norms and other factors. Every emotion has its own strength and can vary from person to person. People behave in a different ways on the similar problems. You can learn to control your emotions and not allow some of them to progress (like anger - as society teaches us to do).
Need to mention that there are a lot of psychological problems that people could have like learned helplessness - that shows us that our experience can make us feel strange emotions as reaction to some triggers. And this is example that emotions like everyting else could be learned. More of that - you can control the emotions your AI will have.
Of course it could be the question if AI needs emotions and what benefits it can have from them.