text:
Is it possible to use a trained neural network to predict a feature, given other features and output?
I have a neural network that is already trained to predict two continuous outputs from a set of 7 continuous features.
Is there any way to apply the network to predict one of the input features, given other 6 features and the two outputs?

From your question, it appears that you would like to use other features in your data to predict one of the features. I am not sure I understood your question clearly, but anyways, either, you would be using the feature you want to predict as the output of the network.
Also, if you want to use the output of the network and other features to predict the new output, I think possibly you are trying to use a Recurrent Neural Network based approach, in which the past output is taken into account for future predictions. My personal experience with RNN's is that they are really good at learning such dependencies, as the RNN cells consider the present input, as well as the past output in modelling such tasks. If your problem is a sequence to sequence prediction task, I would definitely suggest trying RNN's (especially LSTMs, ideal for learning long term temporal dependencies) but ultimately, there is No Free Lunch. So try different approaches and see which one works for you best. Good Luck!


text:
Supervised K-means clustering doesn't appear to work
I have a data set containing actions taken by customers (e.g., view a product, add a product to cart, purchase product), the product bought (if any) and times of said actions. I am attempting to use K-means clustering to identify the customers who are more likely to purchase a product based on these actions (minus the purchase).
I'm currently clustering using: the number of products viewed, the number of products put in the cart, the mean time between the actions, the variance of the time between the actions, the standard deviation of the time between the actions (all of these values are normalized), as well as the product purchased (if any). The clusters I'm getting contain ~10% buyers and 90% non-buyers, but I'm trying to separate buyers and non-buyers.
Any thoughts on what else I can do? Or should I try another method completely?
Illustration: x axis are the clusters, y axis is the number of customers, red are buyers and blue are non-buyers

Update: I made a 3D  graph showcasing the clusters, the amount of customers and the mean time between actions (normalized because of reasons)

Yet another update: customers (not grouped by cluster, just as is) according to the average number of products they viewed and the average time between actions

I took some advice and tried using PCA (from this tutorial), and these are the results I got:

The raw data (x=number of items viewed/carted, y=average time between interactions)

Any tips on how to cluster this mess?

Comments are not for extended discussion; this conversation has been moved to chat.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What are Linear and Non-Linear Features of an image in the context of Convolutional Neural Network?
What features of image are linear or non-linear, any example ?

I don't know what you mean by "image features are non-linear" and "non-linear kernel". Please, edit your post to include the excerpt from the source where you heard or saw these terms/expressions (so provide some context).

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Complex systems constituting an entity unto itself
Introduction:
The notion that various social complex systems (e.g. society, family, business company, state, etc) could be regarded as ones exhibiting consistent traits of behaviour of their own - suggesting that they are entities unto themselves, some sort of organisms on their own or even intelligent entities on their own - is not new. I have personally stumbled upon papers of scholars who straightforwardly speak of such systems as if they were already proven to be singular entities.
That kind of assumption has entered the vernacular, as well, long time ago - e.g. "the state wants to..." , "society responds to conflict by...", "the family dynamics seeks balance through...", etc.
Therefore, we could even assume at one point, that such social systems are not only organisms of their own, but even some sort of artificial intelligence entities (as long as we could see them as an artificial product of human activity).
We are generally used to seeing ourselves as conscious entities and we are also good at exploring entities of less complexity than ourselves. But when it comes to entities which consists of us as mere components, we are not ready to mentally process that idea - it sounds as either too abstract or too sci-fi (think of Stanisław Lem's work).
Question:

While the average Joe could easily say "The state wants to..." or "Society responds to...", etc, how exactly do we prove (or at least gather some sort of supporting evidence) that a complex social system really exhibits a behaviour of its own?
Under what conditions could we regard it as some sort of spontaneously born artificial intelligence?
If that were true, how could we predict if that AI would procreate and bring about other social system forms which are also entities unto themselves? How could we possibly become aware if that has already happened?

Related reading: emergence and collective intelligence.


text:
When should I use Reinforcement Learning vs PID Control?
When designing solutions to problems such as the Lunar Lander on OpenAIGym, Reinforcement Learning is a tempting means of giving the agent adequate action control so as to successfully land.
But what are the instances in which control system algorithms, such as PID controllers, would do just an adequate job as, if not better than, Reinforcement Learning?
Questions such as this one do a great job at addressing the theory of this question, but do little to address the practical component.
As an Artificial Intelligence engineer, what elements of a problem domain should suggest to me that a PID controller is insufficient to solve a problem, and a Reinforcement Learning algorithm should instead be used (or vice versa)?

actually it's not too much work, its pretty standard in industry, using modern system design tools like MATLAB you can tune PID or any other controller relatively easy to satisfy your needs. Reinforcement learning is not applied in practice since it needs abundance of data and there are no theoretical garanties like there is for classic control theory. By the way, contoller design does not involve working directly with integrals/differentials, for linear systems all the work is done in Laplace domain which involves simple algebraic manipulations

As a good rule of thumb that helped me in past projects, if you cant explain explain the optimal policy (PID, RL, or otherwise) in a few sentences, PIDs will be really really hard. What's the optimal policy for Pacman?

Wouldn't it be great to lay a comparison between MPC (Model Predictive Control) and RL too ?

I think the comments are basically on the right track.
PID controllers are useful for finding optimal policies in continuous dynamical systems, and often these domains are also used as benchmarks for RL, precisely because there is an easily derived optimal policy. However, in practice, you'd obviously prefer a PID controller for any domain in which you can easily design one: the controller's behaviors are well understood, while RL solutions are often difficult to interpret.
Where RL shines is in tasks where we know what good behaviour looks like (i.e., we know the reward function), and we know what sensor inputs look like (i.e. we can completely and accurately describe a given state numerically), but we have little or no idea what we actually want the agent to do to achieve those rewards.
Here's a good example:

If I wanted to make an agent to maneuver a plane from in front of an enemy plane with known movement patterns to behind it, using the least amount of fuel, I'd much prefer to use a PID controller.
If I wanted to make an agent to control a plane and shoot down an enemy plane with enough fuel left to land, but without a formal description of how the enemy plane might attack (perhaps a human expert will pilot it in simulations against our agent), I'd much prefer RL.

Nice example. Can u also include MPC in this example ?

Personal example: in one of the labs I used to work in, years of halfhearted work with PID control had failed to achieve anything impressive to electrically stimulate human arms to go from a start position to a target position. Reinforcement learning quickly discovered much better control, since we just weren't good at applying control theory well enough. Might be called brutish, but it worked.


text:
How can Siamese Neural Networks accept a variable number of inputs?
Traditionally, Siamese Neural Networks have two inputs. With some tweaking, you can get them to accept any number of inputs. What I don't understand is how to get them to accept variable numbers of inputs. I've seen a couple of research papers (most notably this one) where they talk about doing this, but none explain exactly how.
Could someone please explain how to create a Siamese Neural Network with a variable number of inputs?

Hello. Welcome to Artificial Intelligence Stack Exchange. Can you please provide the links to the papers that you read so far (that you don't understand)?

After more research, I've found that magnet loss could present a solution. I would say more, but I can't find an implementation in keras (the framework I use) and my question about it is still unanswered. I'll make sure to edit this if I get a reply. I would love to combine the research done in both the magnet loss paper and the VIS-CNN paper, but I currently don't have the skill to


text:
Can Q(s,a) be replaced by V(s) when certrain requirements are met?
I read this post, was thinking about it and now I have a hypothesis but I am not sure whether or not its correct.
I claim that in Q-learning $Q(s,a)$ can be replaced by V(s) when

$p(s'|a,s)$ is deterministic, so $p(s'|A=a,S=s) = 1$
the policy is deterministic, so $\pi(A \mid S = s)$ is just $\pi(s)$
the reward function $R(s, a, s')$ is independent of the action and thus just $R(s, s')$
It is possible to enumerate all next states $s'$ for a given state $s$.

I firstly implemented that as a very basic Deep Q-Learning (or better Deep V-Learning) network and used OpenAI's gym environment frozen lake. Frozen lake has deterministic transition probabilities, the states are known and the rewards are bound to the state, independent of the action. So it fulfills my requirements.
I implemented only one neural network which is used for policy and target prediction. The network outputs only 1 value (V(s) rather than 1 value for each state action pair. I used a replay buffer and employed epsilon greedy. Epsilon greedy yields either a random action or it calculates $\max_{s'} V^\pi(s')$.
It learned an optimal policy so I tried to proof my claim formally.
Proof:
Per definition:
$Q^\pi(s,a) = \sum_{s'\in S}p(s'|a,s)[R(s, a, s') + \gamma V^\pi(s')]$
Since $p(s'|a,s) = 1$ and $R(s,a,s')=R(s,s')$, action $a$ vanishes and so:
$Q^\pi(s) = \sum_{s'}[R(s,s') + \gamma V^\pi(s')]$
So $Q^\pi(s) = Q^\pi(s, a)$.
Next, per definition:
$V^\pi(s) = \sum_{a \in A}\pi(a|s) * Q^\pi(s,a)$
Since $\pi(a| s) = 1$ we can remove the sum and use $Q^\pi(s)$.
So $V^\pi(s) = Q^\pi(s)$.
Per definition, the Q-formula:
$Q^\pi(s,a) \leftarrow Q^\pi(s,a) + \alpha[R(s, a, s') + \gamma \max_a Q^\pi(s',a)- Q^\pi(s,a)]$
Since we showed that $Q^\pi(s,a) = V^\pi(s)$ and $R(s,a,s') = R(s,s')$, we can say:
$V^\pi(s) \leftarrow V^\pi(s) + \alpha[R(s,s') + \gamma \max_{s'} V^\pi(s')- V^\pi(s)]$
What do you think?
UPDATE
Here is the code that shows that an optimal policy in Q-learning can be found with just V(s).

Your criteria #4:

It is possible to enumerate all next states $s'$ for a given state $s$.

makes me think you are describing afterstate representation.
Yes, you can really ignore actions for model-free RL in that case, but it's not a matter of "cancelling out" actions starting from definitions in Q-learning. Instead, actions and action-value functions simply don't exist in this subset of reinforcement learning problems. The agent's "action" is to select the next state directly, there is no separate action space, and nothing need get formally labelled with a separate type of action ($a \in \mathcal{A}$). This is common in board games for example, although these have different complications to deal with adversarial agents.
Instead of trying to prove anything about the action value function Q (and the idea of an action value function $Q(s)$ which does not evaluate an action), you should start with different fundamental objects, e.g. your (stochastic) policy function is $\pi(s'|s): \mathcal{S} \times \mathcal{S} \rightarrow \mathbb{R}$, and the Bellman equation for value function is $v_{\pi}(s) = \sum_{s' \in \mathcal{S}} \pi(s'|s)(r(s, s') + \gamma v_{\pi}(s'))$
Note your criteria 2 (that the policy is deterministic) is not required.

Neil, was I correct that what OP is suggesting is closely related to QSS learning? I haven't been able to look at the paper I'm thinking of since he has posted, but I really think it is.

@David I'm, not sure, but I think it is related, yes. It would be another way to code the action away from the agent's perspective. Using QSS, the agent could see and work with  afterstates, and the inverse dynamics model would convert the agent's state goals into actions. Importantly though, for model-free learning for the agent, the model would also need to provide a list of all possible target states for the next time step (or the agent would need access to an environment model that did this)


text:
Deduce properties of the loss functions from the training loss curves
I have two convex, smooth loss functions to minimise. During the training (a very simple model) using batch SGD (with tuned optimal learning rate for each loss function), I observe that the (log) loss curve of the loss 2 converges much faster and is much more smooth than that of the loss 2, as shown in the figure.
What can I say more about the properties of the two loss functions, for example in terms of smoothness, convexity, etc?

I'll ask the same question I asked in the data science SE : what make you think you can get overall properties of functions with a set of evaluations along an optimisation path ? From just your graph it's not possible to tell that much, yeah the minima of loss 1 appears lower than the one of loss 2. The path appears bumpier but it may be due on how you selected your 'optimal' learning rates. Even with a lot more details we probably won't be able to answer that question.

The loss 1 is in fact smaller than the loss 2, by construction. As I said, the learning rate seems to impact only the speed of convergence, not the smoothness of the loss curve. I've also tried other datasets and observed the same behaviors. I know that it's not enough to generalise from what I observe, but I want to understand why such behaviors persist in the datasets I'm working with.

Maybe start by giving us the form of your loss and be more precise about what you want to know (that would be not trivially given by the loss forms).


text:
Loss/accuracy on Synthetic data
I am trying to understand if there is any difference in the the interpretation of accuracy and loss on synthetic data vs real data.

So, what is your specific question? Can you elaborate more? Why do you think there would be a difference?

No, there is no difference.
Of course, you are likely not able to extrapolate results obtained from synthetic data to expect identical or similar results in real life to unless you have very compelling reasons to do so.
Without a more specific question, I'm afraid a more specific answer is not possible.


text:
Any guidance on learning rate / batch size for noisy data (high Bayes error rate)?
Is there any guidance available for training on very noisy data, when Bayes error rate (lowest possible error rate for any classifier) is high? For example, I wonder if deliberately (not due to memory or numerical stability limitations) lowering the batch size or learning rate could produce a better classifier.
I found so far some general recommendations, not specific for noisy data: Tradeoff batch size vs. number of iterations to train a neural network


text:
How is equation 8 derived in the paper "Self-critical sequence training for image captioning"?
In the paper "Self-critical sequence training for image captioning", on page 3, they define the loss function (of the parameters $\theta$) of an image captioning system as the negative expected reward of a generated sequence of words (Equation (3)):
$$L(\theta) = - \mathbb{E}_{w^s \sim p_{\theta}}[r(w^s)],$$
where $w^s = (w_1^s,..., w_T^s)$ and $w^s_t$ is the word sampled from the model at time step $t$.
The derivation of the gradient of $L(\theta)$ concludes with Equation (7), where the gradient of $L(\theta)$ is approximated with a single sample $w^s \sim p_\theta$:
$$\nabla_{\theta}L(\theta) \approx -(r(w^s) - b) \ \triangledown_{\theta}  log \ p_{\theta}(w^s),$$
where $b$ is a reward baseline and $p_\theta(w^s)$ is the probability that sequence $w^s$ is sampled from the model. Up until here I understand what's going on. However, then they proceed with defining the partial derivative of $L(\theta)$ w.r.t. the input of the softmax function $s_t$ (final layer):
$$\nabla_{\theta}L(\theta) = \sum^T_{t=1} \frac{\partial L(\theta)}{\partial s_t} \frac{\partial s_t}{\partial \theta}$$
I still understand the equation above.
And Equation (8):
$$\frac{\partial L(\theta)}{\partial s_t} \approx (r(w^s) - b) (p_\theta(w_t| h_t) - 1_{w^s_t}),$$
where $1_{w^s_t}$ is $0$ everywhere, but $1$ at the $w^s_t$'th entry. How do you arrive at Equation (8)?
I'm happy to provide more information if necessary. In the paper "Sequence level training with recurrent neural networks", on page 7, they derive a similar result.

First of all you made a mistake, equation 8 in the paper is defined with $\frac{\partial L(\theta)}{\partial s_t}$ not $\frac{\partial L(\theta)}{\partial\theta}$.
Loss is defined as:
$L(\theta) = - \mathbb{E}_{w^s \sim p_{\theta}}[r(w^s)]$
If we use definition of expectation (for discrete case):
$\mathbb{E}[X] = \sum\limits_{i} p_i(x_i)x_i$
we get
$L(\theta) = -\sum\limits_{w^s} p_\theta(w^s)r(w^s)$
probability $p_\theta(w_t)$ is defined as $softmax(s_t)$ = $S(s_t)$. If we calculate derivative of $L(\theta)$ with respect to $s_t$, we are basically calculating derivative of softmax $S(s_t)$ with respect to $s_t$.
It can be shown and i won't derive it here that derivative of softmax function is
$\frac{\partial S(s_i)}{\partial s_j} = S(s_i)(1_{ij} - S(s_j))$
where $1_{ij}$ here is defined as yours. If we plug this derivative into derivative of  $L(\theta)$ and replace $S(s_t)$ with $p_\theta(w_t)$ (remember that probability distribution is defined as softmax here) we get:
$\frac{\partial L(\theta)}{\partial s_t} = -\sum\limits_{w^s} p_\theta(w^s)(1_{w^s_t} - p_\theta(w_t))r(w^s)$
we extract the - from the equation above, use definition of expectation again and subtract baseline from reward and we have:
$\frac{\partial L(\theta)}{\partial s_t} = \mathbb{E}_{w^s \sim p_{\theta}}[(p_\theta(w_t) - 1_{w^s_t})(r(w^s) - b)]$
now, if we ignore expectation we get:
$\frac{\partial L(\theta)}{\partial s_t} \approx (p_\theta(w_t) - 1_{w^s_t})(r(w^s) - b)$
I probably butchered notation is some places, but base idea should be clear

Okay nice I understand, thank you! Thanks for pointing out the mistake, it was a typo & I corrected it :-)

By $p_\theta(w_t)$, you mean $p_\theta(w_t^s)$?

I'm still confused about this:  $w^s = (w_1^s,..., w_T^s)$ is a sequence (dim $T$) of words $w^s_t$. I know & understand the $S(s)$ derivative, but here $w^s$ and $w^s_t$ do not have the same dimensions and I don't really see how you would take the softmax of $w^s$... $p_\theta(w^s_t)$ is indeed $softmax(s_t)$ I think but $p_\theta(w^s)$ should rather be a product of probabilities (one for each word in the sequence) right? Now if it were a sum or if we would drop in a logarithm the other probabilities could be omitted because we they are constant deriving w.r.t. $s_t$ maybe?

I'm not entirely sure, I only glanced over the paper, I didn't actually read it. It is possible that I made some mistake especially in these middle step which they don't show in the paper. Maybe instead of $p_\theta(w^s)$  it would go $p_\theta(w_t^s)$ but they would differ in $t$, maybe you could still replace that with expectation over $w^s$ where each $w_t^s$ would be sampled


text:
Detect named entities inside words using spaCy
I am using rasa nlu for training an NLU system to detect intents and slots. Now, some languages have word endings with their nouns (like Finnish, e.g. "in Berlin" -> "Berliinissä"). I have tried to annotate the characters in the training data as entities, but then I run the model, it doesn't detect the characters inside the word. When those characters are a separate word, only then they're detected. I am unable to think of an implementation to effectively detect named entities within a word. Suggestions needed.

You could use a tool to decompose composite word. There are many open source softwares for various languages. For example : german-word-splitter


text:
Anomaly Detection in distributed system using generated log file
I am developing an AI tool for anomaly detection in a distributed system.  The system supports an interface that combines several individual logs into a single log file generating approx. 7000 entries/min. The logs entries are partially system generated (d-Bus, IPC, ….)  and human written statements (Status not received, initialized successfully, ….). The developers use the generated log for debugging. The entries have been configured to have a similar format depending on the generated system (timestamp, ids, component, context, verbosity level, description, ….).
Background:
1. The history of the identified anomalies is minimal and not archived.
2. Not many similar event templates in log files.
3. Software execution rules are not clearly documented.
4. The log events are co-related.
What are the recommended algorithms (Statistical, NLP, ML, Neural networks) that can be used to efficiently perform pattern extraction on the entries and identify existing and new anomalous behavior?

Are your categories only "anomalous" and "not anomalous" or you want to perform anomaly detection and categorisation (as two different tasks)?

The main goal is to perform anomaly detection. By categorization, I meant the extraction of features from the log events relevant to the identification of the anomalous behavior.

Is the data going to be continuously provided (that is, will you keep receiving a stream of data) or is the data contained in a set which will not change?

I am looking into the stream of data already present(offline). But in the future, it is desired to extend the method to perform on a stream of incoming data (online).

What percentage of user log are expected attack?

In the paper "Unsupervised real-time anomaly detection for streaming data" (by Subutai Ahmad, Alexander Lavin, Scott Purdy and Zuha Agha), 2017, an algorithm for anomaly detection (particularly suited for cases where a stream of data is continuously provided) is described. This algorithm is based on Numenta's Hierarchical Temporal Memory model.
I've actually never used it, but I know that Numenta's work is particularly suited for anomaly detection. You can have a look at it and see if it fits your needs. Have also a look at the Numenta Anomaly Benchmark (NAB).

Thank you for the reference. I am new to machine learning related topics. I am a bit confused here about the datasets(logs) I have. Do I need to start by thinking about how to label them (anomalous/normal behavior)? Or can I perform anomaly detection on the unlabeled dataset?

@Ben In this case, I don't think you will need labelled data. Anomaly detection only consists in finding certain patterns in the data (so this is a unsupervised learning technique).

Do you have suggestions for any other unsupervised learning techniques? The Numenta approach cannot be utilized for my work due to license issues.

@Ben I would not have other suggestions (right now). But why is the license an issue? I think they have an open-source option.

Unfortunately, it is blacklisted for use in my company.

@Ben I think you should report and discuss this issue at https://discourse.numenta.org/, if you have time. I also found Numenta's approach to licensing a little bit unclear. If you decide to post something on Numenta's forum, maybe link them to this question.

Can I not perform a clustering-based approach to recognize patterns of normal execution behavior from the logs and then identify anomalous behavior that does not adhere to the normal execution pattern? Or is this not an ideal solution to my scenario?

@Ben I think that a clustering-based approach might help you to find certain groups of data points in your data. But how are you going to define an anomalous data point?

You are right. It is difficult to define an anomalous point. I believe it depends on how clearly we define normal behavior.

There are a lot of ways to define this. Ultimately all machine learning is based upon an underlying set of assumptions and a model. The obvious thing is to fit some parametric model to your data and learn threshold for when things go wrong. Do you have examples of when things become anomalous?


text:
Will open LLM be subject to AI act / considered High Risk?
As I was reading the AI act I was wondering how it will apply to current open LLM (say Mixtral of Mistal AI).
It seems that there are some specific arguments regarding open LLM that would put them outside of the AI act scope:

They are free (are they still "put on the market" ?)
They are tools (not an AI system with a usage), it's there reusage in an AI system that would

Was the topic of LLM covered somewhere that I am missing ?
Bonus question: given their prevalence and potential impact, will they be considered high risk ? (or its their reusage ?)

I don't see a fundamental boundary in LLMs as tools or agents. If the system only answers questions, but does it always to the best of their "knowledge" it is a tool. But if it sometimes asks questions out of some curiosity (objectively describable as missing information "wanted"), sometimes answers but sometimes refuses to answer or lies on purpose due to some game constraint, is it an agent? I don't think the changes needed for those very different behaviours are any deep.

By "AI act", are you referring to https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence? Can you please include the link to the AI act in your own question? Thanks.

Yes.
The EU AI act has a specific section about generative AI. I'll quote it.

Generative AI, like ChatGPT, would have to comply with transparency requirements:

Disclosing that the content was generated by AI
Designing the model to prevent it from generating illegal content
Publishing summaries of copyrighted data used for training

So, clearly, LLMs (open or closed) are subject to the EU AI Act. So, I don't really know why you have doubts about it. Have you maybe read some unofficial document or are you referring to a different act?
Moreover, in the official linked document, these rules are not placed under the "High risk" section, but I suppose generative AI is also high-risk because in the "High risk" section we also have e.g. "Education and vocational training", which generative AI has been used for.
The act will probably evolve in the next months or years. See also the previous detailed one here.

My question related to open LLM (Mistral AI distributing weights), for which the provider  might be someone else. As such its not entirely clear that those bullet points apply to mistral / the entity using weights or both.

I think the AI Act and these rules still apply (there's nothing that suggests otherwise). I think your question is different. Your question is "who is responsible for ensuring this specific generative AI complies with these rules". If this is not your question, then I think that's the most interesting question anyway.


text:
What are evolutionary algorithms for topology and weights evolving of ANN (TWEANN) other than NEAT?
I wonder, if there are other than NEAT approaches to evolving architectures and weights of artificial neural networks?
To be more specific: I am looking for projects/frameworks/libraries that use evolutionary/genetic algorithms to simultanousely evolve both topology and train weights of ANNs other than NEAT approach. By 'other' I mean similar to NEAT but not based entirely on NEAT. I hope to find different approaches to the same problem.

The Wikipedia article on neuroevolution contains a list of neuroevolution techniques (e.g. NEAT). I will list below the examples that evolve both the parameters and the topology of the neural network.

Cellular encoding (1994)
GNARL (1994)
EPNet (1997)
Evolutionary acquisition of neural topologies (2007)
Interactively constrained neuro-evolution (2012)
Deus ex neural network (2012)
Spectrum-diverse unified neuroevolution architecture (2017)
Covariance matrix adaptation with hypervolume sorted adaptive grid algorithm (2017)

I would like to note that I am not familiar with any of these methods. This list is based on the info in the mentioned Wikipedia article.


text:
Is it possible to use deep learning to generate a 2D image from a few numerical values?
Is it possible to train a DL model that will generate a full resolution 2D image based on few numbers describing this image and what type of model or architecture would that be?

What I want to achieve is that I deliver to the model some numbers for example describing positions of objects on the screen and number describing how lit the scene is and I get back a 2D image with objects in their correct positions and proper lighting, but for one set of input data values I will get always one same image (see image above). These input data also could be anything else than positions and lighting, these are only examples helping to visualize what I mean.
This all, of course, assuming that I have a lot of annotated training data that consists of images and labels of the objects' positions and scene lighting values.
EDIT: The final model would be trained on real images taken from Full HD camera, not some simple shapes like presented here, that I did only to explain better my question.

I do not see the reason to refer to any kind of deep learning for this problem. You have objects and properties of the scene and just render the scene and output view from a certain angle.

@spiridon_the_sun_rotator Well, the reason is that this is only simplification of my problem that I done to help visualise what I am asking for. Later, I want to train suggested model on data of real images taken with Full HD camera and make sort of a realistic scene renderer, generated by DL.

Also, I don't see why, for experimental/research reasons, I should restrain myself from using other methods of doing tasks that can be done with regular/simple methods. My question was about whether such DL models exist, not how to render scene.

Did you take a look at GANs?

Probably closely related to the problem of interest would be
something akin Neural Radiance Fields (NeRF for short) https://www.matthewtancik.com/nerf.
The model takes several images from different angles and view of the scene and learns a 3d representation of the scene, that can be used to sample novel views of this scene (not present in the training data).

The knowledge of the scene is encoded in the weights of MLP. Namely, the renderer is a function that takes coordinates $(x, y, z)$ and viewing angle $(\theta, \phi)$ as an input, and outputs the color (RGB) and the density $\sigma$.

For your problem, one can modify, the network, that is takes in addition to the coordinates and angles, other properties of the scene, such as lightning (some scalar), e.t.c and outputs again (RGB) vector and density $\sigma$.
In order for procedure to be successful, I would expect, that model needs data with different lighting, and provided there are enough data points, one could synthesize the view with given lightning.
The presence of multiple objects can be probably taken into account, by passing a set of coordinates $(x_i, y_i, z_i)$ - but this seems to complicate things a lot, and the training procedure.

This is very close to what I need indeed, with the different output, I think their model outputs some data like points in cloud, and only later they make a scene from this by use of traditional rendering methods. What I want to do is to make such a renderer to produce ready 2D images directly from input data. But I think I will try something like reversed FC network or reversed CNN network, that instead of making predictions from images it makes images from predictions, sort of thing.


text:
What is the reasoning behind the number of filters in the convolution layer?
Let's assume an extreme case in which the kernel of the convolution layer takes only values 0 or 1. To capture all possible patterns in input of $C$ number of channels, we need $2^{C*K_H*K_W}$ filters, where $(K_H, K_W)$ is the shape of a kernel. So to process a standard RGB image with 3 input channels with 3x3 kernel, we need our layer to output $2^{27}$ channels. Do I correctly conclude that according to this, the standard layers of 64 to 1024 filters are only able to catch a small part of (perhaps) useful patterns?

what is the point of assuming that extreme case? Is it only for the sake of asking or you have a practical task where you are restricted to use only 0 and 1 filters?

@voo_doo Maybe instead of extreme, I should use the word edge or least complex. The more values a kernel can take, the higher the power base and the more filters are needed to capture all the patterns. So I wanted to say that the layer with the continuous kernel should use at least more filters than the layer with the binary kernel. This question came to me when I read about wide_resnet, which improved the results by increasing the number of filters in layers.

From mathematical point of view you are correct as are your calculations. To catch all the patterns you need that many filters, but this is where a whole idea of a training comes in. Main objective of the training in the CNNs is to find just a few good patterns from billions possible ones.
So the direct answer to your question is: The standard layers of 64 to 1024 filters are only able to catch a small part of (perhaps) useful patterns, yes but this is assuming no training taking place. If you conducted training on given data with given model, then 64 to 1024 filters could already extract a lot of useful patterns, perhaps more than needed.

Let $n=C*K_w*K_h$. Then you should only need $n$ filters. Not $2^n$ to keep all the information. If you just used the rows of the identity matrix as your filters than your convolution would just be making an exact copy so it definitely wouldn't be throwing away information. On the other hand, there will be a max pooling operation. To simplify the question let's suppose we have a 3 channels, and a 1 by 1 kernel. And then let's suppose it is just one convolution followed by global max pooling. Also, let's use your assumption that it's all binary. If you have $m$ filters then the final output will be $m$ dimensional no matter how many input points you have. So clearly information is being thrown away there. But that's not such a bad thing. Throwing away irrelevant information gets us closer to the features we need the problem at hand. The parts that get thrown away by max pooling correspond to features not being found in a particular part of the image.

Keeping information is not the goal here. The layer is supposed to add new information's about the relationship between the data in the input. To find all possible relationships between just two RGB pixels using binary kernel, we need $2^{2*3}=64$ filters. Dropping some of the information's may not be the problem in later layers working on high level features, but the first layers need to find a kernel, that basically fits all cases, because this kernel has to find some useful pattern in every single slice of the image.

If you take just 6 filters and plug the output of your convolution into a dense MLP then it should be able to approximate any function of your data assuming you have enough neurons. But if you first put it through maxpooling as is normally done then having 64 filters would let you extract every possible feature in the output of your maxpooling. But you have still lost information by doing max pooling. So using 64 filters would with maxpooling would let you approximate any function but only as long as you don't care about duplicates in the input

I'm a little lost in what you talk about. Why do you keep referring to using max pooling? The current trend is to move away from using it and replace it with strided convolutions. And I've never seen it used in front of a convolutions layer. I'm trying hard to understand what you're trying to explain to me, but it' s still eluding me.

i'm saying that what you said is true in the case where we use max pooling. Of course, 2^29 is way more than we'll need or be able to use anytime soon.


text:
Can dropout layers not influence LSTM training?
I am working on a project that requires time-series prediction (regression) and I use LSTM network with first 1D conv layer in Keras/TF-gpu as follows:
model = Sequential()
model.add(Conv1D(filters=60, activation='relu', input_shape=(x_train.shape[1], len(features_used)), kernel_size=5, padding='causal', strides=1))
model.add(CuDNNLSTM(units=128, return_sequences=True))
model.add(CuDNNLSTM(units=128))
model.add(Dense(units=1))

As an effect my model is clearly overfitting:

So I decided to add dropout layers, first I added layers with 0.1, 0.3 and finally 0.5 rate:
model = Sequential()
model.add(Dropout(0.5))
model.add(Conv1D(filters=60, activation='relu', input_shape=(x_train.shape[1], len(features_used)), kernel_size=5, padding='causal', strides=1))
model.add(Dropout(0.5))
model.add(CuDNNLSTM(units=128, return_sequences=True))
model.add(Dropout(0.5))
model.add(CuDNNLSTM(units=128))
model.add(Dense(units=1))

However I think that it has no effect on the network learning process, even though 0.5 is quite large dropout rate:

Is this possible that dropout has little/no effect on a training process of LSTM  or maybe I do something wrong here?
[EDIT] Adding plots of my TS, general and zoomed in view.

I also want to add that the time of training increases just a bit (i.e. from 1540 to 1620 seconds) when I add the dropout layers.

A couple of points:

Have you firstly scaled your data, e.g. using MinMaxScaler? This could be one reason why your loss readings remain high.
Additionally, consider that while Dropout can be useful for reducing overfitting, it is not necessarily a panacea.

Let's take an example of using LSTM to forecast fluctuations in weekly hotel cancellations.
Model without Dropout
# Generate LSTM network
model = tf.keras.Sequential()
model.add(LSTM(4, input_shape=(1, previous)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
history=model.fit(X_train, Y_train, validation_split=0.2, epochs=20, batch_size=1, verbose=2)

Over 20 epochs, the model achieves a validation loss of 0.0267 without Dropout.

Model with Dropout
# Generate LSTM network
model = tf.keras.Sequential()
model.add(LSTM(4, input_shape=(1, previous)))
model.add(Dropout(0.5))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
history=model.fit(X_train, Y_train, validation_split=0.2, epochs=20, batch_size=1, verbose=2)

However, validation loss is slightly higher with Dropout at 0.0428.

Make sure you have specified the loss function correctly. If you are forecasting a time series, then you are most likely working with interval data. Therefore, mean_squared_error is an appropriate loss function as one is trying to estimate the deviation between the predicted and actual values.

As a counterexample, binary_crossentropy would not be suitable as the time series is not a classification set. However, misspecifying the loss function is a common error. Therefore, you also want to make sure you are using the appropriate loss function and then work from there.

Yes, I scale data with StandardScaler
This might be the answer, but why this happens? What's the remedy?
Yes, I use MSE as a loss function

Can you please include a plot of the time series itself (if that's possible)? It will be easier for me to answer if I know the structure of the series.

I added plots to the main question :)

In your second graph, we can see a clear trend in the time series. From my experience, LSTMs don't tend to work very well with trend data. Here is an example: https://towardsdatascience.com/are-lstms-always-effective-at-time-series-analysis-a-case-study-of-oil-prices-97644f716a1c


text:
How to draw bounding boxes for gender classification?
I wonder what is the better way of drawing rectangles on images for gender classification. My task is to create a classifier (CNN based) to detect gender from pictures of entire bodies (not just faces). When I started labeling pictures I noticed that I am not sure whether I should draw it around an entire person like example 1 (including hands and legs and some background space between them) or just the inner part like example 2 (where there is almost no background), in order to achieve better results?
Example 1

Example 2

In state-of-the-art papers. researcher managed to achieve 90-100% accuracy on gender classification on just human faces, so both method may work just fine, with minor improvements maybe using the first method. However using the image of the entire body enhances overfitting as there is more input features and increases overfitting. For a very large dataset it might be better to use the first method as chances of overfitting are low while for a smaller one you should probably go for the second method or even just the face as that won't make such a huge differnce.
A better way to input the data to the network is through landmarks of a body.

Inputting the landmarks of the body helps reduce the features to the minimal and also keeps only the relevant features. The main difference in body between women and men is perhaps the torso and shoulder width, which you can get from body landmark detection. You can input both the image of the face and the body landmark as input to the network, and it would probably increase in accuracy. However this solution is not an one stage method, meaning it requires an extra model to predict the landmark.
Hope I can help you.


text:
Are there any public real-life code examples of ML applications in Python?
Problems I often face at work usually differ from tutorial or book-like examples so I end up with a code that works but it's not elegant and takes too much time to write.
I wanted to ask you if there are some publicly accesible examples or repositories of Python codes that deal with machine learning development and application process but were created in a real company or organisation to develop their real-life products or services?
EDIT: What I do not think about are libraries or packages repositories such as tensorflow. I would like to see some codes of projects that for example use tensorflow to create some other product or service.

This is an interesting question, but likely to be a very rare result, since any code that gives a company a commercial edge directly on a core task is likley to remain private and proprietary. I'm assuming you will not want to include e.g. TensorFlow or other frameworks published by the big tech companies? These could be interpreted to meet your criteria in theory. For instance Google really does use TensforFlow internally, that's the reason it has so many options for deployment. However, they don't generally make full code for their services that use TF models public.

Yes, tensorflow is pretty close to what I need, but in this question I wanted to ask about some code examples that for example already use packages like tensorflow to pursue other tasks.

Maybe some non-profit working projects?

If you are not looking for something like TF, edit your post to clarify what you are exactly looking for. In general, you should try to be more specific.

The best way is probably to Google it with "[org name] tensorflow github" and look what you get.
For instance I found:
Microsoft
Nvidia
Intel

If you search Papers with Code for python "machine learning" (or a more specific query) you will get numerous results. Note these will be mostly scientific applications or methods.

Not what I really wanted. Youre talking about libraries such as tf or pytorch just different, and I meant projects that use such libraries for some practical real life application on some real data set.

Find some opensource projects like Arxiv sanity by andrej karpathy which finds ML related papers from arxiv.org. You can find similar opensource applications.


text:
Can two neural networks be better instead of one with a categorical feature?
Let's assume, that I have a neural network with few numerical features and one binary categorical feature. The network in this case is used for regression. I wonder if such a neural network can properly adjust to two different states of the categorical feature or maybe training two separate networks, according to these two states can be a better idea in the sense of smaller achievable error(assuming I have enough data for each of the two states)? The new model will use simple 'if statement' on the begining of the regression process and use proper network accordingly.

Not exactly what you're asking, but training a neural network multiple times from different initial states on the same data, then taking the average of running it through all the resultant networks nets a roughly 4% increase in accuracy. So I would guess this would improve it as well, so long as you don't over train. (I got the 4% stat from one of the cs231n lectures, I just can't remember which one, it's one of lecture 2-10 I think)


text:
Should I use single or double view for gender recognition?
My project requires gender recognition of people shown on the given images, with more than one person per image. However, these people can be positioned in frontal or side view(passing by perpendicularly, no face visible). On the pictures there will be entire bodies shown, not only the faces. My idea is to firstly use object detection to point where people would be and next use CNNs to recognize gender of each person.
My question is: should I use one object detection algorithm for both frontal and side views of a person and then classify them with one CNN, or should I use object detection to separately find people positioned in frontal and side manner and use two different CNNs, one for classification of frontal views and one for side views?
I am asking this because I think it might be easier for one NN to classify only one view at a time, because side view might have different features than frontal, and mixing this features might be confusing for a network. However I am not really sure. If something is unclear, please let me know.
[EDIT] Since problem might be hard to understand only by reading, I made some illustrations. Basically I wonder if using second option can help in achieveing better accuracy for the subtle differences like those in gender recognition, especially when face is not visible:

Single detection and classification:

Two different classifiers:

Img Source

I edited the question giving some graphical explanation of what I am asking for.

I think your fundamental question is: can convolutional neural networks generalize to objects independent of the view?
The answer is mostly yes (given the dataset contains multiple views of objects). This is evident by looking at the results of various challenges: e.g. COCO object detection challenge results on youtube. You can see that no matter what the view is on the car, pedestrians, the detector is not biased towards any specific viewpoint.
Therefore, one can assume that you can build only one network to perform object detection and another network to perform classification.
If you really want to go even further:
- you can make a small change to the architecture of your detector (I guess you might be using something like SSD, YOLO or Faster-RCNN), in which you make gender classification for every bounding box prediction. If you think about it, it is intuitive because the detector is already doing classification (there is softmax + cross-entropy loss usually), you can just add another term in its tensor output and modify the loss. That way you don't even need another network! It would be much faster and simpler.
- you can predict the pose estimation of the object (and corresponding normals) with respect to the camera to capture the best viewpoint to perform classification.

Yes, you are right. What I was more thinking about (and now I see I did not write it) is that gender classification (and I am also thinking about extending this to age-group classification) can sometimes be very tricky because differences might be very subtle, especially with some weird fashion styles of people. So I thought that maybe by making this proposed two views division it could be easier for network to focus on these subtle details.

And can you please elaborate a bit more on your last point: - you can predict the pose estimation of the object (and corresponding normals) with respect to the camera to capture the best viewpoint to perform classification.
What excatly do you mean? Maybe some links?

Yeah I see. You can definitely try it and compare empirically. Also your point might be true in the case if you have little data. If you have enough/a lot of data (20k+ images) then having one architecture will do.

In terms of pose estimation I meant 2D pose estimation, for example this famous work. This does keypoint detection in the space of the image. One can also do it in 3D but thats' more complicated with monocular cameras. I meant that having the exact 3d Pose from a human, given you have a video you can for example select which frame (to get best viewpoint) to run your 2nd classification network on.


text:
What are modern state-of-the-art solutions in prediction of time-series?
I wanted to ask you about the newest achievements in time series analysis (mostly prediction).
What state-of-the-art solutions (as in frameworks, papers, related projects) do you know that can be used for analysing and predicting time series?
I am interested in something possibly better than just RNN, LSTM and GRU :)

An interesting model I encountered in a course is Facebook Prophet. Prophet takes into account trends, seasonality, and holidays for its predictions. As you can probably guess, this is a model that fits Facebook's needs very well. I'll give a brief introduction then provide a link where you can read more. Prophet fits a couple of functions of time represented by a few terms. The general form of the timeseries predictions are,
$$y(t)=g(t)+s(t)+h(t)+\epsilon_t$$
$g(t)$ deals with the trends I mentioned above. This is exactly what you would think it is, and accounts for non-periodic features of the data. It takes the form of a piecewise linear or logistic function.
$s(t)$ accounts for seasonality - in other words these are periodic changes in our timeseries (maybe an increase of sunscreen purchases in the summer). As this is periodic, the natural way to model this term is with Fourier decomposition to identify important frequencies in the signal.
$h(t)$ deals with predictable changes in the timeseries but is for events like holidays (this can happen at different times year to year so this is not necessarily periodic, think Easter). The user provides a list of events and how they want to account for it.
$\epsilon_t$ is just an error term to deal with anything that cannot be addressed with the rest of the model.
This page has a great explanation if you want more detail. I highly recommend you check it out because it is very cool!


text:
How can I be sure that the final model, trained on all data, is correct?
The 'by the book' method of delivering final machine learning models is to include all data in the final training (including validation and test sets). To check robustness of my model I use randomly chosen population for training and validation sets with each training (no set random seed). The results on validation and then test sets are pretty satisfactory for my case however they are always different each time, precision spans between 0.7 and 0.9. This is due to fact that each time different data points fall to set with which model is trained.
My question is: how do I know that final training will also generate good model and how to estimate its precision when I do not have anymore unseen data?

I'm not sure if I understand your question correctly, but PAC model (probably approximately correct) offers some probabilistic performance guarantees on the trained models from the training accuracy and so on. Maybe this is what you're looking for: https://en.wikipedia.org/wiki/Probably_approximately_correct_learning

The purpose of the test set is to test your model before deploying, otherwise, you would not need the test set in the first place. If you retrain your model by also including the validation and test datasets, of course, you cannot test your model anymore. You need to leave the test dataset separate and not use it for retraining, unless you have more data for testing.

But the final model, deployed on production should be trained on all data. I just wonder how to be sure that such model will work well in future.

@Makintosz As I say in my answer, I don't know where you read this, but, no, the final model should not be trained on all data (as far as I am concerned), unless you are willing not to be able to test the model you will deploy. There's no way to test your model without data. In the end, your model is a machine learning model that learns from and is evaluated on data.

First source I could find: https://machinelearningmastery.com/train-final-machine-learning-model/ it says: "Your model will likely perform better when trained on all of the available data than just the subset used to estimate the performance of the model.
This is why we prefer to train the final model on all available data."

@Makintosz Well, it's true that the model will likely perform better given more data, but then you cannot empirically evaluate it anymore if you don't have more data. However, I've just seen that someone suggested looking into stuff like PAC learning and error bounds. Maybe that serves your purpose.

We usually divide the dataset into multiple subsets namely (training, validation and test sets). During training, we validate the model against the validation set. And during testing, we use the test dataset to obtain metrics for the model. We should make sure the subsets are taken from the same sample. Once you've tested it against the test subset, there's nothing really we can do.
You can also increase your dataset, by using multiple data sources if the problem statement allows you to.


text:
Is there a clustering algorithm that can make n clusters and the n+1 "others" cluster?
As far as I know all clustering algorithms assume that all delivered data points have to find its cluster.
My question is, is there an algorithm that could focus only on n clusters (number stated by user) and try to dismiss the rest of the points that (according to algorithm) do not belong to n clusters, like in the picture shown below? Where we know that there are for example 2 classes that we need to cluster (red and green) and the rest (blue) we do not need in any cluster and therefore algorithm does not try to assign them to any cluster?
For example if we would have 1 000 pictures of animals, of which 200 are dogs, 200 are cats and the rest are all other animals known to men and we want to make 1 cluster for cats, 1 for dogs and maybe another for collectively all others that do not match dogs or cats.

So, I've prepared some data that resembles your sketch:
n , u = np.random.normal , np.random.uniform
x = np.concatenate([ n(1.0,0.2,100), n(3.0,0.3,100), u(0,10.0,100)])
y = np.concatenate([ n(7.0,0.4,100), n(5.0,0.3,100), u(0,10.0,100)])
# lets shuffle it a bit
idx = np.arange(x.shape[0])
np.random.shuffle(idx)
data = np.array([x,y])[:,idx]

And then I just tried using sklearn.mixture.GaussianMixture with n+1 = 3 components and default parameters:
gmm = GaussianMixture(n_components=3).fit(data.T)
cls = gmm.predict_proba(data.T).argmax(axis=1)

# Plotting
color = [['r','k','g'][i] for i in cls]
scatter(data[0],data[1],c=color, marker='.')
scatter(gmm.means_[:,0],gmm.means_[:,1],c='b',marker='o')

That seems awfully lot like the thing you've sketched.

Nice demonstration of what Gaussian  mixture models can be used for!


text:
Can a model, retrained on images classified previously by itself, increase its accuracy?
Let's assume I have a CNN model trained to categorize some objects on the images. By using this model I find more categorized images. If I now retrain this model on data set that consists old set and newly categorised images is there a chance that such new model will have higher accuracy? Or maybe because new data posses only information that could be found on initial set, model will have similar/lower accuracy?
Please let me know if something unclear.

The most likely outcome of this approach is wasted time and very little effect on accuracy.
There will be changes to the model. Some will be beneficial and improve the model, but some will backfire making it worse.
For instance:

The model predicts with probability 0.4 that an image is in a certain class. It is the highest prediction, and actually true. It will be added to the training dataset with a "ground truth" of probability 1.0, so on balance more and better data has been added to the data set. This will improve generalisation, as whatever caused the relatively low 0.4 value initially - e.g. a pose or lighting variation - will now be covered correctly in the training set.
The model predicts with probability 0.4 that an image is in a certain class. It is the highest prediction, and actually false. It will be added to the dataset with "ground truth" of probability 1.0 for the wrong class. This will weaken associations to the correct class for similar input images, meaning for exaple that a certain pose or lighting difference that is already causing problems for the model will be used to incorrectly classify images in future.

These two scenarios will occur, on average, at a rate determined by the model's current test accuracy. So if your current model is 90% accurate, 1 in 10 images in your new training data will be mislabelled. This will  "lock in" the current errors at the same rate on average as they already occur.
The effect may be a drift up or down in accuracy as the model will definitely change due to the new training data. However you have little to no control over how this drift effect goes if you are not willing or able to oversee the automatic classifications generated on new data by the model.
There are a few ways to get some improvement unsupervised from new data. For instance:

Build an autoencoder from the early convolutional layers of your model and train it to re-generate all inputs as outputs. This should help it learn important features of the variations in data that you are using. Once this training is done, discard the decoder part of the auto-encoder and add your classifier back in to fine tune it. This may help if you have only a small amount of labelled data, but a lot of unlabelled data.
Use a model that has better accuracy than yours to auto-label the data. This might seem a little chicken-and-egg, but you may be able to create such a model using ensemble techniques. The ensemble model could be too awkward to use in production, but may still be used in an auto-labeling pipeline to improve your training data.

Note you may get even better results simply ignoring the extra unlabeled data, and instead fine tuning a high quality ImageNet-trained model on the labeled data you already have - saving yourself a lot of effort. Depends on the nature of the images, and how much labeled data you are already working with.

So, there is a chance, that after classifing new images with given accuracy(ie. 90%) and then manually correcting the classes to 100%, the model will improve slightly? I mean correcting manually is faster than labeling from the begining.

Yes. Provided your manual labelling remains high quality (you find and correct all mistakes), then this kind of "assisted" labelling is no different in quality to obtaining more raw data in the first place. Having more data is not guaranteed to improve performance, but the amount of data available is often a bottleneck.


text:
What is the difference between active learning and online learning?
The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? Is one a subset of the other?

Active learning (AL) is a weakly supervised learning (WSL) technique where you can have both labelled and unlabelled data [1]. The main idea behind AL is that the learner (or learning algorithm) can query an "oracle" (e.g. a human) to label some unlabelled instances. AL is similar to semi-supervised learning (SSL), which is also a WSL technique, given that both deal with unlabelled and labeled data, but do that differently (i.e. SSL does not use an oracle).
Online learning are machine learning techniques that update the models as new data is collected or arrives sequentially, as opposed to batch learning (or offline learning), where you first collect a dataset of multiple instances and then you train a model once (although you can later update it when you update your dataset). Batch learning is currently the common way of training machine learning models, given that it avoids problems like the known catastrophic interference (aka catastrophic forgetting) problem, which can occur if you learn online. For example, neural networks are known to face this problem when learning online. There are incremental learning (aka lifelong learning) algorithms that attempt to address this catastrophic interference problem.

Ah, I had misunderstood what online learning was previously. If you are training on streaming data, is online learning the typical approach?

@David Yes, I would say that you need online learning if you want to learn from a stream of data. Note that you should probably look into incremental learning algorithms, which are algorithms that somehow try to avoid the problems that naive online learning algorithms can incur.

What do you think about online algorithms compared to boosting algorithms? It seems that boosting algorithms can also be used with streaming data as well.

@David To be honest, I am not familiar with boosting algorithms, though I've heard of them many times. I just didn't yet have the opportunity to learn about them. I suggest that you ask a question on the site. Maybe someone can give you a more precise answer. Try to ask a specific question.

As it is referred in the survey paper "Active Learning Literature Survey":

The key idea behind active learning is that a machine learning algorithm can
achieve greater accuracy with fewer training labels if it is allowed to choose the
data from which it learns. An active learner may pose queries, usually in the form
of unlabeled data instances to be labeled by an oracle (e.g., a human annotator).
Active learning is well-motivated in many modern machine learning problems,
where unlabeled data may be abundant or easily obtained, but labels are difficult,
time-consuming, or expensive to obtain.

Online learning uses data which become available in a sequential order. It's main goal is to update the best predictor for future data at each step.
So, online learning is a more general method of machine learning that is opposed to offline learning, or batch learning, where the whole dataset has already been generated and used for training / updating the model's parameters. Moreover, a common technique for training Machine Learning models is to first perform online learning, in order to acquire an adequate data size, and then perform offline learning on the whole dataset and finaly compare the results generated by the two learning processes.
On the other hand, active learning can be performed both with online learning[1] and offline learning, in order to reduce manual annotation effort during the annotation of training data for machine learning classifiers. That is, independently of how data have been generated and with what order, active learning should make the least queries, to an Oracle, needed for annotation of a subset of the data.


text:
How to update Loss Function parameter after compilation
I used following custom loss function.
def custom_loss(epo):

  def loss(y_true,y_pred):
      m=K.binary_crossentropy(y_true, y_pred)
      x=math.log10(epo)
      y=x*x
      y=(math.sqrt(y)/100)
      l=(m*(y))

      return K.mean(l, axis=-1)
  return loss

and this is my discriminator model
def Discriminator():

  inputs = Input(shape=img_shape)

  x=Conv2D(32, kernel_size=3, strides=2, padding="same")(inputs)
  x=LeakyReLU(alpha=0.2)(x)
  x=Dropout(0.25)(x, training=True)

  x=Conv2D(64, kernel_size=3, strides=2, padding="same")(x)
  x=ZeroPadding2D(padding=((0, 1), (0, 1)))(x)
  x=BatchNormalization(momentum=0.8)(x)
  x=LeakyReLU(alpha=0.2)(x)

  x=Dropout(0.25)(x, training=True)
  x=Conv2D(128, kernel_size=3, strides=2, padding="same")(x)
  x=BatchNormalization(momentum=0.8)(x)
  x=LeakyReLU(alpha=0.2)(x)

  x=Dropout(0.25)(x, training=True)
  x=Conv2D(256, kernel_size=3, strides=1, padding="same")(x)
  x=BatchNormalization(momentum=0.8)(x)
  x=LeakyReLU(alpha=0.2)(x)

  x=Dropout(0.25)(x, training=True)
  x=Flatten()(x)
  outputs=Dense(1, activation='sigmoid')(x)
  model = Model(inputs, outputs)
  #model.summary()
  img = Input(shape=img_shape)
  validity = model(img)
  return Model(img, validity)

and initialize discriminator here
D = Discriminator()
epoch=0
D.compile(loss=custom_loss(epoch), optimizer=optimizer, metrics=
['accuracy'])
G = Generator()
z = Input(shape=(100,))
img = G(z)
D.trainable = False
valid = D(img)

i want to update epo value of loss function after each epoch in the following code
for epoch in range(epochs):

  for batch in range(batches):
      ............
     d_loss_real = D.train_on_batch(imgs, valid)
     d_loss_fake = D.train_on_batch(gen_batch, fake)
     d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)
     g_loss = combined.train_on_batch(noise_batch, valid)

Are there any way for updating loss function without effecting training after compiling the model?


text:
If I am interested in theoretical computer science, is AI a bad choice?
I hope this question is ok here, but since I have found a tag which deals with these issues (profession), I'll ask away. I also hope this may be useful to other people with similar doubts, since I am failing to find valuable information on this topic online.

I am interested in the theoretical side of CS, such as computability, logic, complexity theory and formal methods. At the same time, I am deeply fascinated by Artificial Intelligence and the questions it poses to our understanding of the notion of intelligence and what does it mean to be a human being.
In general, is AI a more "applied"/engineeristic field, or are there theoretical aspects to research in?
In short: If I prefer formal/theoretical compsci, is AI a bad career choice?
(note: I am asking this because I am a CS undergrad considering getting into a AI MSc).

One big problem is AI is buzzword-of-the-year, and everything that can be (and much that should not) is being labelled "AI". As a result, some AI courses may be theory-heavy, some will not be, and some might be data science with a new trendy name. Do you have a synopsis of your proposed MSc course?

Yes. https://www.uva.nl/en/programmes/masters/artificial-intelligence/artificial-intelligence.html?1549639118410=

There are other MScs which are even more "theoretical" (such as Utrecht's course) but they seem too keen on psychology, cognitive science and philosophy for my interests. I have chosen UvA because it looks like the most compsci-heavy course out there. But I am afraid this comes at the cost of a too much "applied" inclination

For context, do you have any idea of any course - on any subject - that does offer your preferred mix of theory vs applied work? I know that might be hard to assess for you, but it is also hard for someone answering to try and predict what counts as enough theory or too much applied work for you (and not themselves).

This is hard to answer as I am young and unexperienced, but I think that my goal would be the following: a course that would enable me to conduct research in the field of theoretical computer science (or at least, prepare me for a phd to in the right direction), but at the same times that would land me a job in case I fail to enter in research. And of course, which deals with AI. maybe I am asking too much :-)

There are certainly results in theoretical computer science / pure math with deep implications for AI. But to my knowledge these results typically aren't labeled as results of artificial intelligence, but as something more congruent in that particular field (For example in CS, we might say "agent with unbounded computational power"; in math might say some statement is "decidable/undecidable" with respect to some system). Of course they still matter in the field of AI, but you need to know what you are looking for.
See my question What are some implications of Gödel's theorems on AI research? for some examples. Or you can look up MIRI's research guide for a better idea of what existing work is out there that links formal math / CS to AI research.
Another point to raise is that there is no good definition of AI in fields outside of normal discourse (or even within, perhaps), so its difficult to decide what discussions pertains to the study of AI. Questions like whether ZFC with/without choice is expressive enough might not be on the mind of most AI researchers, but could still have some implications.
So to answer you question more directly, there is certainly a field of study regarding theoretical AI. Regarding whether or not its a good choice is something for you to decide, but it is (in my humble and not-very-well-educated opinion) very difficult field that isn't very popular, and has not seen major progress in many years.

AI has not seen major progress and isn't very popular?

Sorry I should be more clear: The specific brand of "pure AI" construed as unbounded agents / decidability results / incompleteness theorems is largely unpopular relative to the engineering aspects of AI (neural nets, probabilistic inference)

But in another sense, yes: AI (more specifically AGI) has not seen major conceptual breakthroughs since the days of Simon and Newell. (imo) Recent progress in NNs and probabilistic methods are very powerful but also very task specific. But very popular indeed.

Any "serious" AI program is full of theoretical and mathematical foundations (you will study plenty of statistics and optimisation methods) anyway, but I would say that much useful AI today is an applied or engineering area. Anyhow, you will need to be comfortable with a lot of mathematical details (especially, linear algebra and calculus). If you're more interested in statistics, optimisation or robotics, you should go for AI.
If you study "pure" computer science, you should also have one or two courses related to AI (at least, one ML course). If you are more interested in traditional CS algorithms, data structures, software engineering, operating systems, compilers, theory of computation, computer networking, programming languages and/or databases, then you should go for CS.
However, before enrolling in a master's program, you should really have a look at the details of the courses they offer. Furthermore, you might also take into account that, during your studies, you might change idea regarding one subject.

This answer contains some guidelines to choose among AI or CS solely based on the reader's interests. It completely ignores future job opportunities.


text:
Formulation of a sentence using FOL
I want to formulate the following sentence using FOL; I want also to know whether there any contradictions in it, or if it is consistent.
Assuming human relations are binary:

All Human Relations are Utilitarian (Human relations are Utilitarian
  only when people in those relations are selfish and calculative).
  I am a human. People are Humans. Unselfish people are nice. Therefore, I am nice.

There's a contradiction: either all relations are utilitarian, or they are not; but not both. The second sentence thus contradicts the first, unless all people are selfish and calculative. And in the remaining sentences you establish that there is at least one person that is not selfish.

∀ r HumanRelations(r) ->  Utilitarian(r)
∃ r Utilitarian(r)-> Selfish(People(p,r)) ∧ Calculative(People(p,r))
Human(I)
∃ p People(p) - > Human(p)
~Selfish(p) -> Nice(p)
Nice(I) , can someone verify?


text:
DQN Agent not learning anymore - what can I do to fix this?
I am trying to use Deep-Q-Learning to learn an ANN which controls a 7-DOF robotic arm. The robotic arm must avoid an obstacle and reach a target.
I have implemented a number of state-of-art techinques to try to improve the ANN performance. Such techniques are: PER, Double DQN, adaptive discount factor, sparse reward. I have also tried Dueling DQN but it performed poorly. I have also tried a number of ANN architectures and it looks like that 2 hidden layers with 128 neurons is the best one so far. My input layer is 12 neurons, the output 10 neurons.
However, as you can see from the image down here, at a certain point the DQN stops learning and gets stuck at around 80% of success rate. I don't understand why it gets stuck, because in my opinion we could reach an higher success rate, 90% at least, but I just can't get out of that "local minimum".
so, my question is: What are some techniques I can try to unstuck a DQN from something that looks like a local minimum?
figure:

note: the success rate in this picture is computed as the number of successes in the last 100 runs.

Is your success rate measured during training runs, or a separate assessment using greedy actions only? What is your exploration technique - if $\epsilon$-greedy, what is $\epsilon$?

Hi @Neil Slater. It is measured during training. My exploration rule is epsilon greedy with espilon starting at 1 and decaying to 0.01 with a decay factor of 0.9995

Thanks for that detail. It is not possible to say what variation caused by a 0.01 exploration rate would make to your environment. Could you please test one later (mid-80% ) success rate agents, using $\epsilon = 0$, with, say 1000 runs, and report the success rate? If it is close to 80%, then the environment and/or agent are robust to mistakes. If it is notably higher, then this is a better/true measure of the performance of your agent after training

yes already tested that with 10k runs: 78% success rate with eps = 0. I'm sorry but I didn't understand the rest of the comment: could you explain again?

I meant that the measurement is necessary, because even a small exploration rate can adversely affect success during training. This is very dependent on the nature of the environment. A "robust" (my term, not standard) environment here is one where one or two mistakes don't matter much and can be corrected. Which in turn means that your graph during training is more usable to help rack possible fixes


text:
Can AI research lead to new findings in general cognitive science?
Can AI be used as a tool to investigate our minds?
To be more precise, what am I specifically asking for here are examples of discoveries on artificial intelligence (so algorithms, programs and computers that try to implement intelligent systems) that brought to light facts about intelligence and cognition in general. Have this ever happened? Is it frequent? How influential and important were these discoveries, if any?
A possible example of what I mean could be the PSSH, which states that a formal system is sufficient to simulate general intelligent behaviour. I believe that this is relevant to Cognitive Science in general because it entails our understanding of this phenomena. (Of course, this is just an hypotesis, but I believe that its importance in the AI debate makes it a really compelling result).

This is about hard AI and soft AI: proponents of hard AI work on systems that simulate the way human cognition works, with the eventual (hypothetical) goal of replicating it. This presupposes that you know how cognition works, and presumably you will learn about it as you attempt to replicate it.
Soft AI, on the other hand, tries to emulate the outcomes only. For example, Weizenbaum's ELIZA is clearly on this side, as it uses simple pattern matching, and does not 'understand' anything about the conversations it is having.
Obviously, we don't even know fully what it means to 'understand' something, and building working systems is not really possible with a hard approach. Hence, soft AI is more common, as researchers are usually measured by their outcomes rather than their ideas. As far as I am aware, the hard AI approach has been all but abandoned long ago.
As current AI seems to be dominated by statistical approaches, I doubt that we can find out many useful things about cognition this way.
One interesting side-note: it seems to me that the capabilities of modern AI systems have developed away from human capabilities. A three-year-old can do some things that a sophisticated AI system cannot do, but in some areas (chess, translation, ...) the capabilities of AI systems surpass what humans are capable of. Maybe imitation is indeed not the right way to approach AI.


text:
DQN Agent helped by a controller: on which action should I perform backprop?
Background
I am working on a robotic arm controlled by a DQN + a python script I wrote.
The DQN receives the 5 joint states, the coordinates of a target, the coordinates of the obstacle and outputs the best action to take (in terms of joint rotations).
The python script checks if the action suggested by the DQN is safe. If it is, it performs it. Otherwise, it performs the second highest-ranking action from the DQN; and so on. If no action is possible, collision: we fail.
During training, this python functionality wasn't present: the arm learned how to behave without anything else to correct his behaviour. With this addition on the already-trained network, the performance raised from 78 to 95%.
Now my advisor (bachelor's thesis) asked me to leave the external controller on during training to check whether this improves learning.
Question
Here's what happens during training; at each step:

the ANN selects an action
if it is legal, the python script executes it, otherwise it chooses another one.

Now... On which action should I perform backprop? The one proposed by the arm or the one which was really executed? (So, which action should I perform backprop on?)
I am really confused. On the one hand, the arm did choose an action so my idea was that we should, in fact, learn on THAT action. On the other hand, during the exploration phase ($\epsilon$ greedy), we backprop on the action which was randomly selected and executed, with no interest on what was the output of the arm. So, it would be rational too, in this case, to perform backprop on the action really executed; so the one chosen by the script.
What is the right thing to do here?. (Bonus question: is it reasonable to train with this functionality on? Wouldn't it be better to train the Network by itself, and then later, enhance its performance with this functionality?)

Q-learning - which DQN is based on - is an off-policy reinforcement learning (RL) method. That means it can learn a target policy of optimal control whilst acting using a different behaviour policy. In addition, provided you use single step learning (as opposed to n-step or Q($\lambda$) variants), you don't need to worry much about the details of the behaviour policy. It is more efficient to learn from behaviour policies closer to the current best guess at optimal, but possible to learn from almost anything, including random behaviour.
So it doesn't really matter too much if you change the behaviour during training.
In your case, the script is actually doing more than just changing the behaviour. It is consistently filtering out state/action pairs that you have decided should never be taken, as a domain expert. This has two major consequences:

It reduces the search space by whatever fraction of state/actions are now denied by your safety script.
It prevents the agent from ever learning about certain state/action pairs, as they are never experienced.

The first point means that your learning should in theory be more efficient. As for how much, you will have to try and see. It might only be a small amount if the problem states and actions are unlikely to be reached during exploration from near-optimal behaviour.
The second point means that your agent will never learn by itself to avoid the problem state/action combinations. So you will always need to use the safety script.
In fact you can view the safety script as a modification to the environment (as opposed to a modification to the agent), if its decisions are strict and consistent. Filtering available actions is a standard mechanism in RL when action space may vary depending on state.

On which action should I perform backprop?

In DQN, you don't "perform backprop" on an action. Instead you either use directly or store some observed data about the step: $s, a, r, s'$ where $s$ is the start state, $a$ the action taken, $r$ the immediate reward, and $s'$ the resulting state. You then update the the current action value estimate(s) based on a TD target $\delta = r + \gamma \text{max}_{a'} Q(s,a')$ either online or from the experience table.
When Q-learning learns, it updates the estimate for the action value $Q(s, a)$ - and $a$ is taken from the actual behaviour (otherwise the agent will update its estimate of an action that it didn't take). So your answer here is to use - or more likely store in the experience table - the action actually taken. If the action recommended as optimal at that time is different, ignore the difference, what matters is the observed experience.

I don't know if it is appropriate to ask about this here in the comments, but what do you think about using Duelling architecture in this situation? I thought that maybe decoupling state-evaluation and action-evaluation may help, since we have an external controller. what do you think?

@NetHacker: I don't know about the dueling architecture, so cannot answer here. I think anyway that it is a different enough topic that you should ask a new question. The general rule is that comments are for clarifying posts or suggesting corrections to them. Follow-up questions with significant new details should ideally be new questions on the site


text:
Is there any paper, article or book that analyzes the feasibility of acheiving AGI through brain-simulation?
In my understanding, the mind arises from a physical system, the brain. I see that there is a big research under the topic of simulating physical systems efficiently (especially in quantum computing). Hence, in theory, we could achieve AGI by simulating the physical brain.
Is there any research I should look into regarding this topic? I would like to hear if it is possible, why, why not, what are the limitaions, how far we are from achieving this, and anything else, really. Also, I would like to read something about my first assumption ("the mind arises from a physical system, the brain").
I have searched AI.SE but I've found only related questions, so I don't think this is duplicate. For reference:

Why are we asking, “How can we simulate the brain?”
How powerful a computer is required to simulate the human brain?
Are there any artificial neuromorphic systems which can mimic the
brain?
Is it possible to build human-brain-level artificial intelligence based on neuromorphic chips and neural networks? (this one is pretty similar but I think it's slighly different, and it doesn't answer my question).

Note: I am not asking for the possibility NOW, but in general, so telling me that "we don't know the brain enough" is not on topic.

The Title Question

Is there any paper, article or book that analyzes the feasibility of achieving AGI through brain-simulation?

Yes. There are various analyses that have been published. We have some early work like Some Philosophical Problems From the Standpoint of Artificial Intelligence, John McCarthy and Patrick J. Hayes, Stanford University, 1969. And there is more recent and optimistic work like Essentials of General Intelligence: The Direct Path to Artificial General Intelligence, Peter Voss, 2007. There are refutations of the concept of General Intelligence, artificial or not, in some posts here (listed below) and quite a bit of academic scrutiny of the assumptions behind the AGI concept and the offspring of it: The Singularity. Concern about The Singularity is a social phenomenon, not a scientific one. It began with the depiction of the emergence of a super-intelligence as an inevitability in science fiction and in sensational media.
But mathematically terse proofs of AGI feasibility are not found in the media or in the Terminator franchise started by the wonderful imagination of James Cameron. This is not to say that world domination cannot fall into the hands of a new species created by humans. In fact, Jaques Ellul, in his Techological Society points out that in many ways humans are already subservient to technology — that the creation of humankind is already autonomous and dominant. His 449 page heap of evidence is witty and quite convincing.
Academic publications either

(a) Assume feasibility and discuss approaches to its design or

(b) Illuminate caveats in the idea of a universally intelligent system

Where is the Mind?
Whether the entirety of the mind arises from a physical system is a question that dates back to René Descartes and Gottfried Leibniz. Current conceptions of intelligence tend to deny the interaction and interdependence of the human brain, the rest of the human, and the biosphere into which the human fits without extreme expenditure to produce a floating, underwater, or underground simulation of the biosphere. The mind may center in the brain, but any brain disconnected from the biosphere and the circulatory system for even a short time is completely useless. A think tank is not a bunch of smart people in a sensory deprivation tank. Quite the opposite. Early experimentation with think tanks revealed the importance of mental health of the participants through diversion and physical exercise so that the various forms of neurological stasis could be maintained.
It is quite questionable whether computers without bodies will ever learn as much of general use (that can be put to use in the biosphere) without moving about and interaction with the biosphere. It is highly likely that the mind is more like Carl Jung sees it, as in the brain and elsewhere. Ludwik Fleck, in a somewhat Jungian fashion, speaks of science as the product of a collective in his Genesis and Development of a Scientific Fact, a good read that throws considerable light on the folly of most people's naive conception of facts.
Quantum Computing Buzz
When Intel's founder, Robert Noyce, brought in Gordon Moore in 1975, it was because of various statistical observations he had made, including what is now known as Moore's Law, a curve that on semi-log paper in 1965 looked linear. Since then, the business assumption in Silicon Valley has been that compactness and speed would double frequently and consistently. In the last few years, speed has stopped doubling and compactness seems to have reached its limit around 7 nm. The transistors in this VLSI technology have only a handful of atoms to represent the semiconductor topologies that permit gating, the fundamental activity of digital circuitry.
In essence, to continue the Silicon Valley boon, it has to become Particle Valley, using carbon allotropes or other nano-tech to build computers where Moore's Law is not overturned by the limitations of the structure of the universe. Whether this can be done requires its own feasibility study. Already, SSD (solid state drive) technology relies heavily on probability. Errors are occurring in great proportions and error correcting techniques are used to keep them manageable. That's because, as we reach for designs closer to the quantum level, Brownian motion or the light electromagnetic interference can upset the digital operation intended.
Challenges Versus Show Stoppers
As with many trends in the history of human endeavors, determining what is a challenge and what is a show stopper is not clear until viewing in hindsight. That is why people are asking the various questions in this Stack Exchange listed in the question.
Stating that mentioning our partial knowledge of brain operation is off topic for this question might have been appropriate if we were trying to invent something that we had not already seen in operation in nature. However, that is not the case. We call it artificial intelligence because we think that we are intelligent and want to extend that facility we find in ourselves. The following fields, in addition to quantum physics and machine learning, are thus necessarily relevant to the feasibility of a more generally usable fabricated intelligent system.

Neuro-chemistry in general
Genetics as it applies to neurological network structuring and cell activity
Correlation between DNA coding and human mental performance and stability
Applied psychology
Addiction science
Cognitive science
Mathematics in general and probability and statistics specifically
Control theory
Philosophy, especially in regard to whether intelligence actually exists and the ideas of culturally determined knowledge and belief
Language and linguistics
Symbology
Conceptions of a thought collective

Not only do we not know enough in these areas, but we know very little. There is also considerable evidence that humanity tends to forget some of the things it once knew. Take away our petroleum and we may find out just how much we forgot.
Whether any of the things we have lost from our general knowledge are prerequisites to solving the challenges of universally intelligent system design is unknown. A simple example is that many computing science graduates do not know anything about Kurt Gödel's incompletness theorems and how they led to Alan Turing's completeness theorem. Many use back-propagation in their Python code but don't have any conception that it is a corrective feedback distribution scheme.
On the flip side, we have no strong argument to draw the conclusion that there is some theoretical limitation as solid as the second law of thermodynamics, one of the apparently insurmountable hard stops built into our physical reality on and off planet. We have no proof that intelligence does not exist and is rather an anthrocentric fantasy. We have no proof that whatever limitations humans have cannot be overcome by copying the intellectual life of human beings into human creations or using intellect to create greater intellect in machines.
What we know, as smart as we think we are, is greatly outweighed by what we don't know. Anyone in real research work will agree.

Refutations of AGI Philosophy
Some of the many refutations of AGI that appear throughout academia. Some articles include these.

What is AGI?, Luke Muehlhauser, 2013

Philosophy will be the key that unlocks artificial intelligence, David Deutsch, 2012, The Guardian

On Artificial Intelligence, Rodney Brooks, 2018, The Library of Economics and Liberty

These are some posts here that question various assumptions prevalent in popular AI media.

Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?


text:
How can we find find the input image which maximizes the class-probability for an ANN?
Let's assume we have an ANN which takes a vector $x\in R^D$, representing an image, and classifies it over two classes. The output is a vector of probabilities $N(x)=(p(x\in C_1), p(x\in C_2))^T$ and we pick $C_1$ iff $p(x\in C_1) \geq 0.5$. Let the two classes be $C_1= \texttt{cat}$ and $C_2= \texttt{dog}$. Now imagine we want to extract this ANN's idea of ideal cat by finding $x^* = argmax_x N(x)_1$. How would we proceed? I was thinking about solving $\nabla_xN(x)_1=0$, but I don't know if this makes sense or if it is solvable.
In short, how do I compute the input which maximizes a class-probability?

In deep networks there is actually a wide variety of solutions to the problem, but if you need to find one, any easy way to do this is just through normal optimization schemes
$$\hat x = argmin_x \ L(y,x)$$
where $L(y,x)$ is your loss function. Since ANN's are generally differentiable you can optimize this iteratively with some form gradient descent scheme:
$$x^{i+1} \leftarrow x^{i} - \lambda \nabla_{x^i}L(y,x^i)$$
where $\lambda$ is your learning rate.

Probably the simplest way to search for an image with the highest probability of being a cat is to use a technique similar to Deep Dream:

Load the network for training, but freeze all the network weights
Create a random input image, and connect it to the network as a "variable" i.e. data that can be changed through training
Set a loss function based on maximising the pre-sigmoid value in the last layer (this is easier to handle than working with 0.999 etc probability)
Train using backpropagation, but instead of using gradients to change the weights, back propagate all the way to the input layer and use gradients to change the input image.
Typically you will also want to normalise the input image between iterations.

There is a good chance that the ideal input you find which triggers "maximum catness" will be a very noisy jumbled mess of cat-related features. You may be able to encourage something more visually apppealing, or at least less noisy, by adding a little movement - e.g. minor blurring, or a slight zoom (then crop) between each iteration. At that point, it becomes a more an artistic endeavour than mathematical.
Here is something I produced using some TensorFlow Deep Dream code plus zooming and blurring to encourage larger scale features to dominate:

Technically the above maximises a single internal feature map of a CNN, not a class probability, but it is the same thing conceptually.


text:
What is the relationship between fuzzy logic and objective bayesian probability?
I understand fuzzy logic is a variant of formal logic where, instead of just 0 or 1, a given sentence may have a truth value in the [0..1] interval. Also, I understand that logical probability (objective bayesian) understands probability as an extension of logic, where uncertainity is taken into account. To me they sound rather similar (they both extend formal logic by modelling truth as a continuos interval between 0 and 1).
My question is, what is the relationship between these two concepts?. What is the difference, and what are the differences in AI approaches based upon these two formal systems?

This was a somewhat hotly debated question in the 1980s. The debate was more-or-less ended with papers like Cheeseman's In Defense of Probability.
The short answer is that Fuzzy Logic does not just assign a continuous value to sentences, what it does is assign degrees of membership in different fuzzy sets. These degrees of membership range between 0 and 1.
In contrast, probability says: Among the set of values this variable could take on, what fraction of them are in a certain set? This fraction also ranges between 0 and 1.
This might seem like a semantic distinction, but it has deep implications if you try to use these systems to reason about uncertainty.
For example, consider the question "Will it rain tomorrow?". A probabilistic approach would try to determine the fraction of days like tomorrow that have had rain. A fuzzy logic approach will try to determine whether tomorrow is like a rainy day. The distinction becomes obvious if we then ask whether it will not rain tomorrow. The probabilistic approach will try to find the fraction of days like tomorrow where it did not rain. The chance of it raining or not raining will sum to 1. The fuzzy logic approach will try to determine whether tomorrow is like a non-rainy day. Note that tomorrow may be both like a stereotypical rainy day and not like one. There is no firm requirement that these sets be disjoint. This reflects Cheeseman's critique of Fuzzy Sets: they implicitly reject the additive axiom of probability theory, which (as denoted by the name axiom) is something that is fairly unreasonable to reject.
While there are various approaches to make memberships in fuzzy sets more probabilistic in nature, that's the root distinction.
I think the dominant modern view is that fuzzy sets are a great tool when you need to reason about membership in a fuzzy concept. Whether rice is "done" or not isn't a question of something being true or false. It's a degree of membership in the set "done". On the other hand, whether the object in front of a self-driving car is a person or not is either true or false. It is not necessarily a good idea to reason about this in terms of whether the object is partially in the person set or plastic bag set.

Do all fuzzy operators reject the additive axiom? Or do the ones based on the product respect additivity? Related question:  https://ai.stackexchange.com/questions/38119/does-this-property-in-product-fuzzy-logic-have-a-name-and-any-consequences
Maybe the answer to the linked question is "additivity"?