text:
Is it possible to vectorise a CNN?
I am trying to write a CNN from scratch and am wondering if it is possible to vectorize the convolution step.
For example, if I had a dataset of 500 RGB images of size 32x32x3, and wanted the first convolutional layer to have 64 filters, how would I go about the vectorization of this layer?
Currently, I am running through all 500 images in a for loop, convoluting individually. I do this for all the images up to the flattening stage (where it essentially becomes a normal NN again), at which point I can implement the normal vectorisation approach to get to my output, etc.
A holistic overview of the process would be appreciated, as I am struggling to get my head around it and am struggling to find and information on the matter online.

related: https://medium.com/analytics-vidhya/implementing-convolution-without-for-loops-in-numpy-ce111322a7cd

Yes you can vectorize a CNN. See this github file for details: https://github.com/parasdahal/deepnet/blob/master/deepnet/layers.py
After looking through it it basically transposes the input to some dimension and apply matrix multiplication to the weight with some other kind of transfromation. Pls refer to the github repository for details.
Hope this can help you and have a nice day!


text:
Who is working on explaining the knowledge encoded into machine learning models?
The thing about machine learning (ML) that worries me is that "knowledge" acquired in ML is hidden: we usually can't explain the criteria or methods used by the machine to provide an answer when we ask it a question.
It's as if we asked an expert financial analyst for advice and he/she replied, "Invest in X"; then when we asked "Why?", the analyst answered, "Because I have a feeling that's the right thing for you to do."  It makes us dependent on the analyst.
Surely there are some researchers trying to find ways for ML systems to encapsulate and refine their "knowledge" into a form that can then be taught to a human or encoded into a much simpler machine.  Who, if any, are working on that?

Please, next time when you ask a question, put your question also on the title. I think that this helps to attract more people because they can have an idea of what your post is about. Btw, you're looking for "explainable artificial intelligence".

I have one concept, which allows to structure knowledge gathered by a ML system in both categorical and algorithms-like structure.
The key idea here is that we contain in our minds some network-like structure in our minds, which helping us not only to classify some data, but also to form an abstract meaning about text or message.
Other than that, in opposite to our minds, it looks like this solution can produce these structures in readable for us forms.

I think you are looking for the field known as explainable artificial intelligence. The book Interpretable Machine Learning: A Guide for Making Black Box Models Explainable will surely help you to understand the issues and existing techniques. See also the following question Which explainable artificial intelligence techniques are there?.

We actually can explain how the machine answers our questions. For example, we know how the ML works for the case of image recognition. See the following paper by Chris Olah:
https://distill.pub/2017/feature-visualization/
and also
https://distill.pub/2018/building-blocks/
By feature visualization, we know that the machine sees at each layer and how it decides using given information. As the images are visual it is easy to "visualize" the features but of course, interpreting inner workings of a Neural Network for other kinds of data (such as financial data, for example) would be much more difficult I presume. But still, we can understand how the initial data is weighted or how the intermediate features are calculated.
Given that, I highly doubt the reasoning of a machine can be taught to a human. It is far too complex to be learned. We can easily understand one or two dimensional functions, but the data we use in machine learning problems usually have hundreds of dimensions.

When human engineers design a system, say a moon rocket, of course computers are used to speed the calculations, but the engineers could "do it by hand".  A moon rocket can contain hundreds of thousands of components.  Because the design is hierarchical, it is possible to train an engineer to do any "local" or "global" part of the design process, and understand what he's doing and why it it necessary.  A human being could be trained to do any small enough part of what a machine learning system does, but unless the ML system is hierarchical, it's hard for humans to understand the "why".

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Has machine learning been combined with logical reasoning (for example, PROLOG)?
There are mainly two different areas of AI at the moment. There is the "learning from experience" based approach of neural networks. And there is the "higher logical reasoning" approach, with languages like LISP and PROLOG.
Has there been much overlap between these? I can't find much!
As a simple example, one could express some games in PROLOG and then use neural networks to try to play the game.
As a more complicated example, one would perhaps have a set of PROLOG rules which could be combined in various ways, and a neural network to evaluate the usefulness of the rules (by simulation). Or even create new PROLOG rules.  (Neural networks have been used for language generation of a sort, so why not the generation of PROLOG rules, which could then be evaluated for usefulness by another neural network?)
As another example, a machine with PROLOG rules might be able to use a neural network to be able to encode these rules into some language that could be in turn decoded by another machine. And so express instructions to another machine.
I think, such a combined system that could use PROLOG rules, combine them, generate new ones, and evaluate them, could be highly intelligent. As it would have access to higher-order logic. And have some similarity to "thinking".

This seems to be a duplicate of this, although you focus on logical reasoning (which is only one method of GOFAI: there's also rule-based and maybe other subdivisions).

In reference to your exact question, there is published research that attempts to bring these two areas together.
For example, HolStep: A Machine Learning Dataset for Higher-order Logic Theorem Proving (2017) by Cezary Kaliszyk, François Chollet, Christian Szegedy. This group also has other published work related to the subject.
Regardless of their results, they list several areas of logical systems that are highly suited to machine learning methods (section 3.1, p. 4):

Predicting whether a statement is useful in the proof of a given conjecture

Predicting the dependencies of a proof statement (premise selection)

Predicting whether a statement is an important one (human named)

Predicting which conjecture a particular intermediate statement originates from

Predicting the name given to a statement

Generating intermediate statements useful in the proof of a given conjecture

Generating the conjecture the current proof will lead to

It's tough to know whether or not you can combine Higher Order Logic and Machine Learning in an effective way without needing to create a general AI. This is equivalent to wondering if an effective merging of the two areas is an AI-complete / AI-hard problem.
There are active attempts at general AI by researchers such as Ben Goertzel (many others as well but just to give a popular name for googling). Research into general AI would give you an idea of whether or not other pieces of the puzzle are needed in order to create something "highly intelligent".

Another example where machine learning has been combined with symbolic AI is in the context of knowledge graphs (which can be viewed as a graphical/visual representation of a knowledge base), where people have been proposing ways to learn embeddings of the entities and relations of the graphs (known as knowledge graph embeddings), in order to be able to perform tasks like triple classification (i.e. given a triple $\langle s, r, o\rangle$ with a subject $s$, relation $r$ and object $o$, is this a real fact?).

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is the difference between the ant system and the max-min ant system?
I'm studying ant colony optimization. I'm trying to understand the difference between the ant system (AS) and the max-min ant system (MMAS) approaches. As far as I found out, the main difference between these 2 is that in AS the pheromone trail is updated after all ants have finished the tour (it means all ants participate in this update), but in MMAS, only the best ant updates this value. Am I right? Is there any other significant difference?

If the answer below answers your original question (I don't know, to be honest), you should click the "check" symbol and accept it, then, please, flag this comment as "no longer needed", so that it's deleted.

Let's start by making clear that both AS and MMAS use only global pheromone update. Now, the MMAS has two main differences regarding AS:

In AS, all ants that completed a solution are used for the update, while in MMAS only the best ant with a complete solution is used for update (as you had pointed out).

In AS, the pheromone values are not explicitly bounded. In MMAS, the pheromones are enforced to lie within a pre-set interval $\tau_{min} \leq \tau_{ij} \leq \tau_{max}$ (which gives its name to the algorithm). To ensure this condition, the pheromone update is done by means of the formula

$$
\tau_{ij} \leftarrow \left[ (1-\rho) \cdot \tau_{ij} + \Delta \tau_{ij}^{best} \right]_{\tau_{min}}^{\tau_{max}},
$$
with the operator $[x]_a^b$ defined as
$$
[x]_a^b=
\begin{cases}
    a \qquad \mathrm{if} \quad x > a\\
    b \qquad \mathrm{if} \quad x < b\\
    x \qquad \mathrm{otherwise}
\end{cases}
$$
Reference: Ant Colony Optimization. Artificial Ants as a Computational Intelligence Technique.


text:
How is the rollout from the MCTS implemented in both of the AlphaGo Zero and the AlphaZero algorithms?
In the vanilla Monte Carlo tree search (MCTS) implementation, the rollout is usually implemented following a uniform random policy, that is, it takes random actions until the game is finished and only then the information gathered is backed up.
I have read the AlphaZero paper (and the AlphaGo Zero too) and I didn't find any information on how the rollout is implemented (maybe I missed it).
How is the rollout from the MCTS implemented in both the AlphaGo Zero and the AlphaZero algorithms?

AlphaZero/Go don't have rollout. Rollout replaced with neural network value estimation

@mi: I thought that original Alpha Go did have a rollout policy, but that one of the simplifying changes in AlphaZero was to completely remove it?

@NeilSlater That's correct. The question seems to be about AlphaGo Zero and AlphaZero though, not about Alpha Go :) AlphaGo: uses a fast rollout policy (trained like the policy network, but is not a large DNN, just a single layer + softmax I believe). AlphaGo Zero: no rollouts. AlphaZero: no rollouts.

@mirror2image even though it was only a line, you answered the question -- why not make it slightly more verbose and make it an answer

Thanks for the answers. However, I'm still confused. I understand that they use the V value returned from the network to simulate who won, but how do they know which value of V a player won or lost? Because as far as I understand, in the weight update (in the loss function), they will compare the probability distribution gathered from the self-plays and the difference between the real winner (from MCTS) and the value V.

In short, I'm confused about the V value range returned from the network. Will it be in the [-1,1] range?

@ihavenoidea  I got the same question have you figured it out?


text:
Are there well-established ways of mixing different inputs (e.g. image and numbers)?
I am interested in the possibility of having extra input along with the main data. For instance, a medical application that would rely mostly on an image: how could one also account for sex, age, etc.?
It is certainly possible to put the output of a CNN and other data into, say, a densely connected network; but it seems inefficient. Are there well-established ways of doing something like this?

A more efficient way would be creating a multi input model, with something like this:
___________    _____________
|__Image__|    |Other input|
_____|_____     _____|_____
|___CNN___|     |__Dense___|
_____|______    _____|______
|_Features1_|   |_Features2_|
         __|_____|__
         |__Merge___|
         _____|______
         |___Dense__|
         _____|_____
         |__Output__|

However, you could also combine the unstructured data to the image, as stated in the quora answer:

The out-of-the-box method:
If you want to just take your CNN library and use it without much
  thought, there's an easy way to do it.
Your image has “channels”: red blue and green channels, for example.
  Just add another channel for each unstructured feature. Those channels
  will just be 2D-arrays whose entries are all the same value: the value
  of your outside feature.
It means more memory and more parameters though. If you have a lot of
  unstructured data, this can become prohibitively expensive.
The more efficient method (and still not hard):
You use one or more deconvolutional filters to bring the unstructured
  data up to the size of the structured data, concatenate them along the
  channel dimension, and keep going as if nothing happened.

Source: How can l train a CNN with extra features other than the pixels? (Quora)

authors: انور محمدمهدي سعد النفيعي


text:
How could I locate certain words or numbers in a financial statement?
I would like to code a script that could locate a specific word or number in a financial statement. Financial statements roughly contain the same information, they are however not identical and organized in the same way. My thought is that by using Tensorflow I could train a neural network to locate the specific words or numbers for me. I am thinking that if I label different text and numbers in 1000 financial statements and use them to train the neural network, it will then be able to identify these numbers or words in all financial statements. For example, tell it in all 1000 training statements which number that is the profit of the company. Then when I give it an unseen financial statement, it should be able to identify which number that is the profit.
Is this doable? I have been working with coding in python for a couple of months and so far I've built some web scrapers and integrated them with twitter, slack and google sheets. However, this would be my first AI related project. I would be very grateful for all your thoughts on this project and if anyone could steer me in the right direction by sharing relevant tutorials.
Thanks a lot!

Hi and welcome to this community! To clarify, you say "For example, tell it in all 1000 training statements which number that is the profit of the company". Is this the only task you have, that is, you want to predict the profit of a certain company given financial statements, or you want to solve other tasks too?

Thanks a lot! I don't want it to predict anything. I just want it to identify specific numbers or words (names) in the financial statements. The goal is to have the key information in the financial statements automatically organized in a spreadsheet without having to type it in manually.

Well, in that case, you don't really need neural networks or deep learning. You can just write a program that searches for the keywords you're looking for. However, which key information are you talking about? Do you know what these keywords are?

Well I do think that I need neural networks because the name of the ceo or the amount of profit for example varies every time. Their place in the financial statement also varies. I need the neural network to figure out the right name or the right number by looking at the context, just like a human does.

@Lohant00 "Well I do think that I need neural networks because the name of the ceo or the amount of profit for example varies every time", then maybe you want to predict something. I think you should edit your post and clarify these points (you should explain in your post what you've just written in the comments).


text:
Is there anything theoretically revolutionary about Deep Neural Networks?
In recent years, we have seen quite a lot of impressive display of Deep Neural Network (DNN), as demonstrated most famously by AlphaGo and its cousin programs.
But if I understand correctly, deep neural network is just a normal neural network with a lot of layers. We know about the principles of the neural network since the 1970s (?), and a deep neural network is just the generalization of a one-layer neural network to many.
From here, it doesn't seem like the recent explosion of DNN has anything to do with a theoretical breakthrough, such as some new revolutionary learning algorithms or particular topologies that have been theoretically proven effective. It seems like DNN successes can be entirely (or mostly) attributed to better hardware and more data, and not to any new theoretical insights or better algorithms.
I would go even as far as saying that there are no new theoretical insights/algorithms that contribute significantly to the DNN's recent successes; that the most important (if not all) theoretical underpinnings of DNNs were done in the 1970s or prior.
Am I right on this? How much weight (if any) do theoretical advancements have in contributing to the recent successes of DNNs?

Related: https://datascience.stackexchange.com/questions/14352/how-are-deep-learning-nns-different-now-2016-from-the-ones-i-studied-just-4-ye/14387#14387

@NeilSlater, not quite the same. As I am asking about how much weight  theoretical advancement has ( or is any) in contributing to DNN .

What are you willing to count as "theoretical advancement"? It is possible to claim everything is just book-keeping since the invention of calculus and the chain rule (in 17th century). Backpropagation did not invent this theory, just applied it. Optimisation through gradient descent is also a very old theoretically. Neural networks seen in that light are primarily an invention of applied maths/engineering, not theoretical constructs . . . so I would like to understand your threshold for what you consider as a theory?

@Graviton, Several items in the link above can be considered as theoretical advancements. Using ReLU instead of smooth functions, or the theory section above discussing local vs global minima in higher dimensional functions are theoretical; and the introduction of dropout layer is also in the list. But are they revolutionary? That is a different question.

@serali I agree with you that the question whether those advancements are revolutionary is different from the linked SE question, and hence this is my question here

@NeilSlater my threshold is the same with the scientific community out there . Is there anything revolutionary in terms of theoretical enhancement of DNN as far as the scientific community is concerned?

That just moves the goalposts. There isn't a "scientific community out there" that has consensus views on how major a theoretical result has to be, before it counts as a significant contribution to the field. It is a matter of personal opinion, unless you set something more objective as a threshold. You could go with papers published with theory in neural networks? In which case there have likely been hundreds since 1970s (including some of the early work defining back-propagation which started in 1970s, but went into the 1980s)

@Graviton To make this question more specific (and less open to subjective interpretation), I suggest that you be more specific. For example, you could say that some (or all) of the info in this answer https://datascience.stackexchange.com/questions/14352/how-are-deep-learning-nns-different-now-2016-from-the-ones-i-studied-just-4-ye/14387#14387 does not list "theoretical breakthroughs", for example, you may say that you do not consider the use of ReLU or dropout (or whatever) really a big theoretical breakthrough and that these are just "tricks".

@Graviton Then maybe you can explain what a "breakthrough" is for you with an example. Otherwise, it's not immediately clear what you're really looking for.

The first neural network machine was the stochastic neural analog reinforcement calculator (SNARC), built in the 1950s. As you can see, it's pretty old. After that, there were several advances regarding backpropagation and the vanishing gradient problem. However, the ideas itself are not novel. Simply put, we have the data and processing power today that we did not have back then.
You could look at the Wikipedia timeline.


text:
Why can't LSTMs tell a long story?
There is a recent trend in people using LSTMs to write novels. I haven’t attempted this myself. From what I’m hearing, they can tell a story, but it seems they lose the context of the story rather quickly. After which they begin constructing new, but not necessarily related constructs.
Can they construct a plot in the long term?

An LSTM has trouble remembering over very long time frames. It also, as nbro said, can't really learn common-sense knowledge. It means it can't truly determine what information is important to remember (like a character dying for example) consistently. It very easily loses context when you let it run for extended periods of time (especially if it's longer then the input it was trained on), so it can't realistically write a complete story coherently. Even if you just wanted a short story (~200 words), it is very likely it would lose some important contextual knowledge.

The long-short term memory (LSTM) is a type of recurrent neural network, which is only suited for sequence modelling, that is, to keep track of statistical dependencies between elements of a sequence.
The LSTM prediction capabilities are limited to the training data that is used to train it, the inductive bias (in the case of LSTMs, the inductive bias particularly refers to the fact that elements of a sequence are dependent on each other) and the available computation resources. However, storytelling often assumes the existence of a common-sense knowledge between the storyteller and the listener (but LSTM completely ignores this) and requires the true understanding of language, which is believed to be an AI-complete problem (in simple words, it is a very complex task, which probably cannot be exactly solved with statistical models).
Furthermore, even though LSTM partially addresses the vanishing gradient problem (and they were specifically created to partially solve this issue), they can still suffer from the exploding gradient problem. Moreover, although LSTMs and, in general, neural networks are universal function approximators, in practice, functions may not be continuous, which is an assumption made in the universal approximation theorems.

Just a minor comment: it is probably worth noting that exploding gradients can be fixed easily with gradient clipping in most cases.

@MathiasMüller Thanks for your feedback and thanks for contributing to the site! I've actually not worked much with LSTMs (even though I am quite familiar with the theory), so your feedback is definitely valuable (especially, if you had hands-on experience with LSTMs). Clearly, I am aware of gradient clipping, but I am not sure how much it can help in this specific case, because I am not sure I've ever used it (in this case).

Do you mean, this specific task of telling a story in particular? I don't know, I don't work on story generation. Some technical reading about exploding gradients in RNNs that you might not be aware of. The authors argue that LSTMs fix vanishing gradients to some extent, but they (= LSTMs) do not fix exploding gradients. The authors propose clipping the gradient norm.

@MathiasMüller No, I didn't mean storytelling in particular. I had already come across this paper, but I think I've never fully read it. Thanks for pointing this out. If I have some time, I will have a deeper look at it ;)


text:
Do algorithms like OpenAI's "think up strategies"?
I was discussing with a friend whether current AI does anything remotely similar to 'thinking' and he argued that AIs that play games must think up strategies.
While thinking may not be precisely defined, my understanding of algorithms like OpenAI was that they just minimize a very non-convex objective, but still play the game based on examples, and not by coming up with intentional strategies. Is my understanding incorrect?

The key here is think up strategies. If we define this as examining, creating a hypothesis, and testing it as strategizing then yes AI has the ability to strategize. It can examine other users' games, quantifies actions that correlated with victory then test if it gains victory by doing those actions.
Strategy by definition is: a plan of action or policy designed to achieve a major or overall aim.
AI can not classically plan a series of actions designed to achieve a major victory. Instead, it learns the right strategy by testing simulated scenarios, like someone who thinks about the consequences before doing them, but the AI actually has the opportunity to play the game hundreds of times, in order to learn the correct strategy. Similar to Bill Murray in the movie Groundhog Day, learning the ideal day to live. The AI can strategize by experiencing the game over and over until it fine-tunes what an ideal game should be and has seen enough examples of games to not be outwitted by gimmicky strategies.
To summarize, AI can strategize, just in a way fairly different than people.

Thanks you for your insight. I guess my point of contention to 'thinking up strategies' can then better be described as: The AI does not come up with a strategy by means of contemplation, but by randomly trying (simulating scenarios) and choosing only the winning versions... Right?

Yes, although a bot could technically also train on pushing the game toward states that exist in wins and variables in the game associated with winning too. So the bot examines and assimilates win data and meta win data as well. Maybe during part of a lost game the team was winning then incorrect actions caused a loss, a good bot like a good person should be able to ‘learn’ from loss games and win games because not every decision or action in a lost game is incorrect.


text:
What evaluation metric are used for sequence-to-sequence prediction problems?
I am solving many sequence-to-sequence prediction problems using RNN/LSTM.
What type of evaluation metrics can be used for sequence prediction problems?
One metric is the mean squared error (MSE) that we can give as a parameter during the training model. Currently, the accuracy of my sequence-to-sequence problems is very low.
What are other ways through which we can compare the performance of our models?

It depends on what type of data you're having and what kind of results you're interested in. For example you could use root mean square error (RMSE) which will greatly penalize large errors. If you want your results to be easily interpreted you could use mean absolute percentage error.

This really depends on your data. MSE and its variant, the RMSE, are good for regression problems. In other words, when you want to produce a real number as an output, for example, in a time series forecasting situation. The MAPE is good and can be interpreted as a percentage, but it does not work well if you have zeros in your dataset. As the other answer mentions, BLEU is good too, but only if your model is doing machine translation, or working with some other type of categorical data; however, you would not use BLEU if you are trying to predict house prices, for example.
To find the best evaluation metric you have to consider your data and the problem you are trying to solve. Its good to go over a list of potential loss functions and see what fits your problem best.

I would recommend taking a look at Bilingual Evaluation Understudy(BELU) score which is commonly used in evaluating machine translation results by sequence to sequence model. Here is the reference https://en.wikipedia.org/wiki/BLEU

Also consider the soft-dtw  loss https://tslearn.readthedocs.io/en/stable/gen_modules/metrics/tslearn.metrics.soft_dtw.html#r26a17581b9fd-1


text:
How to measure object size from the disparity map using CNN?
I am a student learning about image processing using CNN. I want to learn how to measure the object size from the disparity map obtained from left and right stereo images.


text:
What needs to be done to make a fair algorithm?
What needs to be done to make a fair algorithm (supervised and unsupervised)?
In this context, there is no consensus on the definition of fairness, so you can use the definition you find most appropriate.

The paper Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges argues that ensuring fairness is not a trivial task and that the current statistical formalizations of fairness lead to a long list of criteria that are each flawed (or even harmful) in different contexts, that is, there are trade-offs between the proposed formalizations. Therefore, fairness constraints in algorithms have to be specific to the domains to which the algorithms are applied. To achieve that, there is a need for collaboration with domain experts and explainable artificial intelligence.
The major obstacle towards fair machine learning algorithms is the presence of algorithmic bias, which can be subdivided into the following main categories:

the bias in the data, and
the inductive bias (the implicit or explicit assumptions behind the algorithm or model) or, in general, any bias introduced during the development of the algorithm or model (for example, a certain choice of a subset of features can change the outcome of the model).

The bias in the data can be due to different factors, such as a biased choice of the collected (or labelled) data or measurement errors (which can make the data not representative of the population). Causal inference can be used to understand the causal relationships in the data, thus it can be used to find the source of bias in the data. To avoid unfairness due to bias in the data, there is a need to analyze and understand the data, so that to improve its quality (for example, by increasing the diversity of the data). However, the bias in the data is not always easily reducible, given that certain outcomes of an experiment may rarely occur or are hard to produce in practice, so unbiased data may not always be easily collectible.
In the paper The selective labels problem: Evaluating algorithmic predictions in the presence of unobservables (2017), the authors address the selective labels problem (that is, the judgments of decision-makers determine which instances are labeled in the data, which can thus introduce bias in the data) and develop an approach called contraction, which can be used to compare the performance of predictive models and human decision-makers (even in the presence of unobservables). There are also works that are based on Bayesian or causal inference (for example, risk-adjusted regression)
The sample bias (the data is not representative of the overall population due to a systematic intentional or unintentional error in data collection, a measurement error, which can also be due to social prejudices) is also a form of bias in the data. In the paper Residual unfairness in fair machine learning from prejudiced data (2018), the authors address this problem in the context of and of police stop-and-frisk (where a biased police behavior leads to over-proportional stopping of a racial minority group). Nathan Kallus and Angela Zhou show that adjusting the classifier for fairness does not solve the sample bias problem.
In the paper Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges, the author argues that any attempt to reduce the bias introduced during the development of the algorithms or models, if it does not take into account the specific social and moral context where they are supposed to be applied, can still lead to algorithmic bias. Furthermore, algorithms should undergo frequent reevaluations, given that, for example, the underlying population or the application context may change.

I will eventually reword this answer to make it clearer and add more details.


text:
Imposing physical constraints (previous knowledge) in a neural network for regression
I'm trying to train a neural network to do a multiple non-linear regression $y=f(x_i), i=1,2…N$. So far it works good (low MSE), but some predictions $y$ are “non-physical”, for instance for our application it is known  from first principles that when $x_2$ increases, then $y$ also has to increase ($dy/dx_2>0$), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that $y + x_5 + x_7$ should be less than a constant $K$
I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a "harder" way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)

About the second one, is it x5+x7 <K or f(x5)+f(x7)<K? The former is not a constraint on the output but and error in the dataset. You should be able to weed them out without machine learning.

Hi serali, I had a typo in that equation, I meant the constaint y + x_5 + x_7 < K

Some ideas out the top of my head:

In the case of $dy/dx_2>0$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds
In the case of $y + x_5 + x_7 < K$ you could use a clipping function on the output layer?


text:
Are humans superior to machines in chess?
A friend of mine, who is an International Master at chess, told me that humans were superior to machines provided you didn't impose the time constraints that exist in competitive chess (40 moves in 2 hours) since very often games were lost, to another human or a machine, when a bad move is made under time pressure.
So, with no time constraints and access to a library of games, the human mind remains superior to the machine is my friend's contention.  I'm an indifferent chess player and don't really know what to make of this.  I was wondering if any research had been made that could back up that claim or rebut it.

He's assuming the computer isn't under time pressure. It is. Removing time limits is not giving an advantage to the human player, it's giving even more advantage to the computer. Of course, unless the game simply goes on forever :)

The problem seems under-specified to me, because if you stipulate "no time constraints," then surely a computer could in principle search the entire game tree of chess and only play optimal moves (even though you wouldn't even be 1% done by the heat death of the universe). You have to have some time constraints to make the question meaningful.

@DavidZhang  I guess what I have in mind is time constraints that are so lax that Zeitnot is not an issue at all, like, for example, 40 moves in 40 weeks or 40 months.  Enough time at any rate to make it possible for the human mind and its creativity to seriously challenge the computing capacity of the machine and maybe defeat it.

Losing games to computers because of mistakes made under time pressure was probably a thing about 20 years ago, when Kasparov lost to DeepBlue after such a mistake(correction: it was Kramnik with the blunder, not Kasparov (see edit 2)). But after Kramnik's loss in early 2000s, no world champion ever tried to play against a computer (to my knowledge). Nowadays, there are computer only tournaments among programs with ratings well above 3300 (for comparison, Carlsen's peak rating was around 2880), and it is not uncommon for computers to make moves with no apparent meaning to humans.
No time limit for humans also mean no time limit for computers so I doubt any human can win a single game against a computer. Older models like Stockfish 8 depend on their computational power as it can look at several millions of position per second, Google AlphaZero  managed to beat Stockfish with 80000 positions per second: they don't seem to depend on brute force calculations any more. Keep in mind that this is without any prior knowledge of openings etc, they are trained using reinforcement learning, starting from the rules of the game only. From there, they can develop their own strategies and implement them without making any mistakes. They create their own openings from scratch, so existing libraries is not going to be very useful.
I am not aware of any research on this but lack of a challenge from humans is probably enough evidence. Also, Grand Masters regularly use chess engines in their training routine to analyze positions, so there is that.
A few years ago there was a game between Stockfish against GM Nakamura + Rybka, which Stockfish won. It is possible that human GM + Stockfish might have better chances against AlphaZero in correspondence without any time limits, but we probably will never know.
Here is an interview with Carlsen after a game, very interesting to show what he thinks about AlphaZero.
Both Kramnik and Kasparov made serious mistakes in their matches against computers. Kasparov resigned in a drawn position and missed a knight sacrifice, and Kramnik missed a mate in 1.

Comments are not for extended discussion; this conversation has been moved to chat.


text:
Is it possible to create a decompiler using AI?
I would like to decompile a compiled file to source code.
Is it possible to use any AI technique to perform decompilation? Is there any research on this topic? If yes, can you briefly explain one of the existing approaches (just to get some insight)? I would also appreciate links to research papers on this topic.

Here you have a similar question.


text:
What technology do people use to create bots for games like LOL or Runescape?
I was curious about how people make AI to play games. Does anyone know of the AI used to play these games? What allows the AI to see/click the screen in real-time? Even just direction on what libraries for such tasks would be helpful. I can't imagine game developers make an API for creating bots in their games like browsers use with selenium.

Bot development is more about 'hacking' than AI in a way that in the very first place you need to read and (over) write game data which you are not supposed to (and thereby potentially violating the Terms and Conditions - so be aware of that). The AI part is fairly simple for most hack/bot applications that I can think of.
Read data
To read game data you can for example:

use an assembly code debugger like Ollydbg to locate relevant data in the memory, e.g. amount of gold
Observe graphical objects being rendered, e.g. a unit being drawn
Intercept network packages containing all kinds of game information, e.g. a unit appearing on your screen

Write data
In a similar way there are multiple ways to write data:

overwrite game data in memory
Use the Windows API SendInput() function to emulate keyboard inputs
Use the Windows API SendMessage() function to send messages to the game
Manipulate network traffic

These lists are not comprehensive but to give you an idea of how it is being done.
AI-wise an A* algorithm for example can be deployed to do path finding.
If you are interested in the topic I suggest to read 'Game Hacking' by Nick Cano. The book provides a good introduction.


text:
creating your own dataset similar to cityscapes format
I'm trying to train a neural network with my own dataset. The neural network can accept the cityscape format.
Is there any application that can give mask/segmented image, instance image, label IDs images and JSON file, similar to cityscape dataset format?
Basically, I want to create my own dataset similar to the cityscape dataset format.

This part "Is there any application that can give mask/segmented image, instance image, label IDs images and JSON file" is still not clear. Which application are you talking about? Where do these "mask/segmented image, instance image, label IDs images and JSON file" come from? In "Is there any application that can give", can give to what? To the neural network? As far as I understand, you want to create a dataset similar to cityscape, but from raw data. Is this what you want? If yes, explain more about your raw data and where it comes from.


text:
Unsupervised learning to optimize a function of the input
I am looking to build a neural network that takes an input vector $\mathbf{x}$ and outputs a vector $\mathbf{y}$ such at $f(\mathbf{x}, \mathbf{y})$ is minimized, where $f$ is some function. The network will see many different $\mathbf{x}$ during training to adjust its weights and biases; then I will test the network by using the test set $\{\mathbf{x}_1, \dots, \mathbf{x}_n \}$ to calculate $\sum(f(\mathbf{x}_1, \mathbf{y}), \dots, f(\mathbf{x}_n, \mathbf{y}))$ to see if this sum is minimized.
However, I have no labels for the output $\mathbf{y}$. The loss function I am trying to minimize is based on the input and output, instead of the output and label.
I tried many standard Keras and TensorFlow loss functions, but they are unable to do the job. Any thoughts on how this might be achieved?

Can you give the specific task you're working on to add clarity?

According to your description, you already know your function $f$ to be optimized. So you should use it directly instead of the standard loss functions. In this other post there is an explanation of how to use $f$ as a custom loss function in Keras.

From your question, it sounds like your only training data is {1,…,} and the network has to magically come up with values {y1,…,y} such that an unknown function is minimized. How do you plan to give feedback to the network during training?
Your situation appears to be something like this:
X-->Model-->Y-->f(X,Y)
where X is being copied from the first layer to the last layer using a non-sequential architecture.
The solution to this problem would be to add an extra layer to your network that implements f(X, Y). However, this will only be trainable using standard methods like gradient descent if f(X, Y) is differentiable. If f(X, Y) is not differentiable, then you may need to use a different optimization method for learning the weights of the model and it may be more difficult. Particle swarm optimization is one possibility here.


text:
Is there any existing attempt to create a deep learning model which extracts vector paths from bitmaps?
I need an algorithm to trace simple bitmaps, which only contain paths with a given stroke width.
Is there any existing attempt to create a deep learning model which extracts vector paths from bitmaps?
It is obviously very easy to generate bitmaps from vector paths, so creating data for a machine learning algorithm is simple.  The model could be trained by giving both the vector and bitmap representation. Once trained, it would be able to generate the vector paths from the given bitmap.
This seems simple, but I could not find any work on this particular task. So, I suppose this problem is not fitted for current deep learning architectures, why?
The goal is to trace this kind of image, which would be drawn by hand with a thick felt pen and scanned:

So, is there a deep learning architecture fitted for this problem?
I believe this question could help me understand what is possible to do with deep learning and what is not, and why. Tracing bitmaps is a perfect example of converting sparse data to a dense abstract representation; I have the intuition one can learn a lot from this problem.

If we seek proven working source code to plug into a GPLv2-licence compatible solution, we should at least consider autotrace. Its source code is open for review. It can be tested against the example images we have and, if it works fine, called by our GPLv2 software. We can even use the calling code in Inkscape's plug-in image tracing implementation as a good starting point for design and implementation of our calling program, whether it be C, C++, Java, Python, or ECMA (JS).
The trace algorithm in Adobe Illustrator is comparable but is not open source.
If we seek theory, there are several academic publications discussing some of the theory, the last being most aligned with machine learning ideology. I would not dismiss earlier work simply because it doesn't connect with the current machine learning idioms. Investigating what is fully implemented and successfully used by many follows a wise old business proverb: The bird in the hand is worth two in the bush.

Potrace: a polygon-based tracing algorithm, Peter Selinger, 2003
Vector Representation of Binary Images Containing Halftone Dots, Kei Kawamura, Hiroshi Watanabe, Hideyoshi Tominaga, 2004
Testing AutoTrace:
A Machine-learning Approach to Automated
Tongue Contour Data Extraction, Jae-Hyun Sung, Jeff Berry, Marissa Cooper, Gustave Hahn-Powell, and Diana Archangeli, 2013

Many of the online drawing programs collect data.  It would not be surprising if, behind the gracious give-away of online bandwidth, they are establishing a continuously improving data set for training a new breed of autotracers.  None have published AI designs admitting as much, but they would not be legally obligated to do so because a single input example is indeterminable from the autotrace service that could resulting from the training.

Thanks a lot! My question contains two questions: 1. How to trace bitmaps (and you answered very well), and 2. Is there a deep learning architecture fitted for this problem? I am very curious about this second question since it could help me understand what is possible to do with deep learning and what is not, and why. Tracing bitmaps is a perfect example of converting sparse data to a dense abstract representation ; I have the intuition one can learn a lot from this problem.

any luck update in this. I fully understand
And today, I can tell u that my research has led me here, and that bitmap tracing for simple lines is so so so trash still after decades,
And I have been a graphic designer for bout 20 years now.
So anyway.im also a low level- mid level developer u could say,
And for so many years I have been thinking of different ways to help solve this problem.
And I dont know
But can some1 please advise me if this method could work
Though it's incomplete I'm hoping I'm on the right path and someone could add to it
Would it be effective to
Train a model on a dataset of bitmaps and it's corresponding SVG that will contain the max amount of points
And % similarity average to the the bitmap
(This idea stems around the thought of training a diffuser or something)
Then
Train another model(or something)
On the same bitmaps dataset
Though the corresponding SVG in this data set will be simplified to the least amount of paths and still within the same % similarity average to the bitmap
So that should simplify the paths and still look like the original
(Obviously not theoretically because I'm not actually sure)
So now when you run this against a bitmap u want to vectorize
I think maybe
Tiling the bitmap
Scan all the tiles
Reproduce that tile bitmap and corresponding SVG overlay
with the trained model that produces the most points
And then stich the tiles back together
And apply a final simplify using the
Model trained with the least points
OK so as muddled as that all is really..
I'm hoping someone can understand that.
And either tell me to shut up.
Or any help to develop this idea further or,
If you need a visual representation of what I'm trying to say, in order to understand it
I can draw up something and send it
But really this has been a life long goal..
I would honestly design the entire dataset
Just to shape the model to spec.
Or am I just way off the mark here

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.


text:
How can max-pooling be applied to find features in words?
I'm reading about max-pooling in a dynamic CNN paper. I can see how it can help find features in images, given that the pixel with the highest density gets pooled, but how does it help to find features in words?

Could you please provide the link to the paper you were reading?

In an image you are pooling usually over some (n x n) set of positions which lets you maintain spatial correlation but on the other hand most 1D CNNs used for language modeling pool over the temporal axis completely annihilating all form of temporal correlation within the resulting feature vector. I will take it this difference is what confuses you.
For simplicity sake Ill keep my explanation to max-pooling but it can extend to any of the others.
MaxPooling as the name suggests just takes a maximum value from a set of candidates. In images you commonly see maxpooling over neighborhoods of pixels. This is because if we assume high activation is correlated to some feature being more prevalent (this assumption is not inherent, its more learned than anything else by construction of the architecture/training), then doing this we keep the most prevalent features, maintain spatial correlation, and reduce the dimensionality.
When using 1D CNNs for LMs rather than RNNs the idea is that each kernel is searching for a feature, and so you use an exorbitant number of the kernels at various sizes to search for a lot of features. Pooling over the temporal dimension then just tells you at each neuron how prevalent its corresponding feature is in that time series input. That is why your output is usually a feature vector of size equal to the number of kernels you used.
A point I want to emphasize is that pooling takes multiple inputs and returns one based on some condition/function but that is vague, and it's usage in different scenarios can be for different goals (to an extent).

see that makes sense for images, but I saw a paper using it for words which didn't make sense to me

I gave the example across whole temporal dim but maybe imagine also using a finite region— which maintains tempora correlations and fits the idea like images


text:
How can I improve the performance of a model trained to detect vehicle poses?
I'm looking for some suggestions on how to improve our vehicle image recognition. We have an online marketplace where customers submit photos of their vehicles. The photos need to meet certain requirements before the advert can be approved.
Customers are required to submit the following vehicle photos: front, back, left-side, right-side, engine (similar to the front photo but with the hood open) and instrument panel cluster.  The vehicle must be well framed in the photo, in other words, it must not be too small or so big that the edges touch the frame of the photograph. It also needs to be one of the mentioned types and the camera must be facing the vehicle directly with only small angle variations (a front photo can't include a large piece of the side of a car).
Another developer had a go and built a CNN with Keras which does alleviate some manual grind (about 20,000 photos were used for training - no annotations). The accuracy sits at around 75% for the vehicle photos but only 55% for the engine and instrument cluster. Each photo is still manually checked, but it is a case of agreeing or disagreeing with what was recognised.
I was wondering if it wouldn't be better to detect a vehicle in the image using an existing pre-trained model like ImageAI. Use the bounding box of the vehicle to determine it is correctly placed in the frame of the photograph and within acceptable dimensions. There may be multiple vehicles in the picture so work with the most prominent one.
At that point would it be worth trying to develop something to workout the pose of the vehicle (idea: https://github.com/johnberroa/CORY) or just do some transfer learning with whatever pre-existing trained model was used and spend some time annotating the images?


text:
How to deal with images of different sizes, which need to be passed to a model of fixed input size, without losing details and spatial information?
I have the following problem while using convolutional neural networks to detect forgeries:
Resizing the image to fit the required input size may not be a good way because the forgery detection largely relies on the details of images, for example, the noise. Thus the resizing process may change/hurt the details.
Existing methods mainly use image patches (obtained from cropping) that have the same size. This way, however, will drop the spatial information.
I'm looking for some suggestions on how to deal with this problem (input size inconsistency) without leaving out the spatial information.

What exactly is the spatial information that you fear to lose? How would it be an input to the forgery detection? If I understood correctly, the detection depends mostly on local features, so cropping seems to be a valid step.

Would you share the average size of your image? Have you tried with large input tensor?

I don't think that the input size inconsistency won't leave out spatial information in the Convolutional Neural Network. The image resizing would loose the characteristics of the object on the image.
It looks like that you don't want to crop your input image, which looks like being fabricated. I like to suggest these preprocessing before the Convolutional Neural Network:
  (1) Find an original image or a picture of the real object
  (2) Perform image registration between a suspicious image and the original image (the registration result should be fine)
  (3) Calculate color difference in each pixel position
  (4) Generate new image with these differences
  (5) Feed to your Convolutional Neural Network for the anomaly detection

When using convolutional neural networks for forgery detection, resizing images can indeed be problematic because crucial details, such as noise patterns, may get altered, which can impact the model’s performance. At the same time, cropping images into patches can result in losing important spatial information.
One solution to this problem is to use letterboxing, a technique commonly used in object detection models like YOLO and SSD. With letterboxing, the image is resized while maintaining its aspect ratio by adding padding. This way, the image’s original details, including the noise, are preserved, and you avoid distortions or loss of spatial data. like some classification models like yolov8 also use letterboxing in their training and inferece of the yolov8 classification model
However, you’ll need to ensure that the model doesn't falsely learn from the padded areas, possibly by using techniques like masking to ignore the padded regions during training.
This approach should help maintain both the detailed features and spatial consistency of the images while feeding them into the neural network at a fixed input size.


text:
How does deepfake technology work with multiple people in a single frame?
I was watching this video from corridor crew, according to them, they have used deepfake technology to create this video. I myself have never made a deepfake videos, but I have enough knowledge in the underlying technology to know that it's hard to swap a face with multiple people existing simultaneously in a frame and let alone swapping faces of multiple people in a single frame. But corridor crews video showed that multiple deepfakes can be done, and that's why I am sceptical about the video of using deepfake technology.
If the video is indeed made with deepfake technology, then what is the mechanism behind this? My own guess is that they might have masked other people and concentrated on one in a frame. Then they have used this masked frame to generate the deepfake, which then concatenated to the original frame. Do you think this is possible? Is there a research article or blog post which explains this process?


text:
Which algorithm can I use to solve a problem with multiple objectives and constraints?
Consider a problem with many objectives. In my case, these are school grades for different courses (or subjects). To be more concrete, suppose that my current grade for the math course is $12/20$ and for the philosophy course is $8/20$. My objective is to get $16/20$ for the math course and $15/20$ for the philosophy course.
I have the possibility to take different courses, but I need to decide which ones. These courses can have a different impact depending on the subject. Let's say that the impact factor is in the range $[0, 1]$, where $0$ means no impact and $1$ means a big impact. Then the math course could have a big impact (e.g. $0.9$) on the grade, while maybe a philosophy course may not have such a big impact.
The overall goal is to increase all the grades as much as possible while taking into account the impact of their associated course. In my case, I can have more than two courses and subjects.
So, which algorithms can I use to solve this problem?

I have edited your post to simplify and hopefully clarify your explanation. Please, make sure that the current version is still consistent with your initial problem. Also, if you have found the solution to your problem, feel free to write an answer below or to accept an existing answer (if it solved your problem).

From your description, the problem you want to solve is a linear optimization problem: suppose we use the indices $i$ and $j$ to denote the $i-$th class and the $j-$th grade. Also, let us call $y_j$ the current value of the $j-th$ grade, $g_j$ the goal value in grade $j$, $c_{ij}$ the impact factor of taking class $i$ in grade $j$, and $x_i$ the binary variable that indicates if you go to class $i$ or not. Now, suppose you decided to take certain classes (which is equivalent to choosing the values $x_i$ for each $i$) and measure the new grades $\tilde g_j$. If your impact factors are accurate, then the new grades should be $\tilde g_j = \sum_i(x_ic_{ij}+y_j)$. Clearly, what you would like is for this new values to be as close as possible to the goal values. So, a possible way to express your optimization problem is that you want to minimize the addition of all these differences:
$$\min_{x_0,\cdots,x_n} \sum_j g_j - \tilde g_j = \sum_j g_j - \sum_i(x_ic_{ij}+y_j), \\ \text{s.t. } x_i\in\{0,1\}, \forall\hspace 2pt i.$$
Most probably, you will have certain time restrictions. For example, maybe each class takes $t_i$ hours from your time and your total available time is just $T$ hours. Or maybe you have a limit of classes $N$ you can take, irrespective of their time duration. With these two constraints, your problem would look like this:
$$\min_{x_0,\cdots,x_n} \sum_j g_j - \sum_i(x_ic_{ij}+y_j), \\ \text{s.t. } x_i\in\{0,1\}, \forall\hspace 2pt i,\\ \sum_ix_it_i\leq T,\\ \sum_i x_i\leq N.$$
Apart from the constraint that $x_i$ should be 0 or 1, this is a linear optimization problem with linear constraints. Integer programming is an area of optimization that studies this type of problems so probably it's a good direction for your research.

I may have missed something but I am actually quite surprised about how the impact factors are taken into account here. Could you define them in a more explicit way?

I just interpreted the "impact factor" of a class i in grade j as the value improvement that can be obtained in this grade j by taking class i. So, if x_i is 0, then there is no improvement, but if it is 1, the improvement is precisely the value of the impact factor. This is a simple model, but it's the one I understood was proposed in the question.

the impact factor (as I have imagined in start) is a value between 0 and 1, with 0 = no impact and 1 = big impact. What I do not know is for how much the grade will be improved if I used the impact factor 1... This is something I am actually making some research... Probably will need some machine learning algorithm to test afterwards the impact of a class on the grades... to improve this impact factor value...

You should consider using probabilistic graphical models if the impact factors (i.e., a linear model) turn out to be poor descriptors of the effect of taking one class in certain grade. Basically, they would allow you to model how certain interaction of classes affects the distribution of your grades. The benefit of this method is that it's easier to answer questions later of how does taking a class i affect the grade j, in comparison with deep learning methods.

Of course, in that case the optimization problem would not be linear and Integer Programming would not be helpful. Non-linear optimization methods like stochastic gradient descent, which means neural networks too, could be helpful. However, I'm not sure how to assure in that case that your x's are either 0 or 1.

Thank you for all those informations. I need to go deeper in my research to solve this problem.


text:
Are there any good ways of simultaneously incorporating object detection with speech recognition?
Are there any good ways of simultaneously incorporating object detection with speech recognition? For example, if you want to identify whether an animal is a dog or cat, we can obviously use visual features (e.g. YOLO, CNNs, etc.). But how would you incorporate speech and sound in this model?

Check out this paper. It deals with the problem of mixing input modalities! Here's the abstract.

This paper presents a novel model for multimodal learning based on gated neural networks. The Gated Multimodal Unit (GMU) model is intended to be used as an internal unit in a neural network architecture whose purpose is to find an intermediate representation based on a combination of data from different modalities. The GMU learns to decide how modalities influence the activation of the unit using multiplicative gates. It was evaluated on a multilabel scenario for genre classification of movies using the plot and the poster. The GMU improved the macro f-score performance of single-modality approaches and outperformed other fusion strategies, including mixture of experts models. Along with this work, the MM-IMDb dataset is released which, to the best of our knowledge, is the largest publicly available multimodal dataset for genre prediction on movies.


text:
Counterexamples to the reward hypothesis
On Sutton and Barto's RL book, the reward hypothesis is stated as

that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)

Are there examples of tasks where the goals and purposes cannot be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal?
All I can think of are tasks with subjective rewards, like "writing good music", but I am not convinced because maybe this is actually definable (perhaps by some super-intelligent alien) and we just aren't smart enough yet. Thus, I'm especially interested in counterexamples that logically or provably fail the hypothesis.

Well, "all of what we mean by goals and purposes" has some wriggle room which is open to interpretation and stretching the meanings of things. Subjective rewards are easy problem to fix, you just need to go to the source of subjective measurement - for example just ask a large enough cross-selection of people whether the output was "good music" and take an average. Or even just count downloads of the latest automated hit single. That is hard to do in practice, and slow perhaps, but the hypothesis says nothing about how convenient the reward signal needs to be.

@Bananin Are you looking for specific examples in the context of reinforcement learning or any example we can think of?

@Bananin you mention subjective rewards are counter examples but they are not, theyre just difficult to model. Like Neil said, you have to find the source of disambiguity: in his examples he offers a commonly used heuristic which is to find the expectation over some set of subjective representatives to define a more "objective" measure-- but regardless subjectivity can be made objective by conditioning on the subjective factor "good vs what I believe to be good" in which case you can still use the above concept.

By the way, the reward hypothesis was suggested by Michael Littman, according to Barto & Sutton's book (2nd edition, p. 70).

What if a scalar reward is insufficient, or its unclear on how to collapse a multi-dimensional reward to a single dimension. Example, for someone eating a burger, both taste and cost are important. Agents may prioritize taste and cost differently, so its not clear on how to aggregate the two. It is also not clear on how a subjective categorical taste value can be combined with a numerical cost.

I like your perspective! However, I think the issue wouldn't be the different prioritization across agents (the reward hypothesis would fit them individually), but the impossibility of such prioritization. Example: COVID lockdowns. They keep people from getting infected but hurt the economy. We want to optimize for economy and health, but how do we add them up?

it being /unclear/ as to how it will collapse into a single dimension value itself does not nessicate that there isn't one. or, perhaps, one can imagine there being a very complex function that considers pretty much everything imaginable into account

What you are describing is addressed in a field of research known as multi-objective reinforcement learning (MORL), which uses vector rewards. I'd suggest you read the following survey paper for an overview of MORL (disclaimer - I was a co-author on this, but I genuinely think it is a useful introduction to this area) Roijers, D. M., Vamplew, P., Whiteson, S., & Dazeley, R. (2013). A survey of multi-objective sequential decision-making. Journal of Artificial Intelligence Research, 48, 67-113. https://www.jair.org/index.php/jair/article/view

I don't think that this is a counterexample to the reward hypothesis. It doesn't matter whether or not we know the optimal way of combining the objectives to achieve the optimal multi-objective or whether the combination of multiple objectives to form the scalar-based one leads to our desired behaviour. The RH just states that we can maximize a scalar-based objective to achieve some goal, but it says nothing about how to design that objective.

I believe that there is no clear answer to your question. It essentially boils down to whether you are a reductionist – whether you believe that quantitative measurements can truly give justice to the complexity of the real world, and that a framework such as expectation maximization can losslessly capture what we care about as humans in the performing of tasks.
From a non-reductionist perspective, one would be aware that almost any mathematical representation of complex real-world goals will necessarily be a proxy rather than the true goal (as many goals are not mathematically formalizable, such as what we perceive as "good music" or "meaning"), and thus the reward hypothesis is at best an approximation. Based on this, a non-reductionist's reward hypothesis could be rephrased as:

that all of what we mean by goals and purposes can be well thought of approximately operationalized (albeit at a certain domain-dependent loss) as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)

Clearly the original (stricter) version of the reward hypothesis does apply to some cases, such as purely-quantitative domains (e.g. maximizing $ earned on the stock market, or maximizing score in a video game), but as soon as the problem involves enough "complexity" (e.g. humans, or wherever you think the boundary should be), a non-reductionist would say that mathematics is clearly not fit to the task to truly capture the intended goal.
More info on the reward hypothesis (as presented by Michael Littman himself) is here. I would have added it as a comment to the question but do not have enough reputation.

I think there's a typo in "Clearly this doesn't apply to all cases, such as purely-quantitative domains", i.e. why wouldn't it clearly apply to "purely-quantitative"?

It's actually not a typo – what I meant is that if your goal is purely defined within a quantitative system (the score in a game, the number of dollars in your bank account), then a scalar reward might in fact not be a loss-full reduction of your goal, but your goal itself. However, if one investigates even these cases closely, usually these goals turn out to also be simplifications of more fuzzy goals ("maximize the $ in my bank account while being ethical").

Maybe you didn't get my point or I don't fully get what you wrote (though I think I do). You're saying that the reward hypothesis (RH) doesn't apply to all cases, such as purely quantitative domains, for example, maximizing the score of a game, but that is one of the examples where the RH usually applies (assuming that the goal is not fuzzy).

Thanks for the comment – edited my original post for clarity

The closest counterexamples I can think of are cases where reward shaping is required to learn a good policy but ends up having unintended consequences.
Reward shaping is usually used in cases we want to encourage a particular behavior or when the reward is sparse or when capturing exactly what you want is not straightforward or infeasible. But it is not a good practice to rely too much on it as it can have unintended consequences. A simple example of this is described here https://openai.com/blog/faulty-reward-functions/.

I don't understand how reward shaping contradicts the reward hypothesis. Can you clarify that?

I will try directly to answer the question you asked here. As you said in the comment any task can be modeled with rewards with some assumptions. But whether the modeling leads to the intended original task getting solved is not guaranteed. I consider this as a failure of reward modeling and hence these tasks might not fall under the reward maximization assumption.

When I think about it, my point is actually targeting the cases where the original reward modeling doesn't work well with the existing training techniques (because of sparse reward or reward shaping). You can argue both ways I think.

The reward hypothesis says that a goal can be thought of as the maximization of the reward. Either if you find a maximum or not of the objective function doesn't imply that you cannot model a problem as the maximization of the reward. So, I really don't think that reward shaping goes against the reward hypothesis.

The book sets this hypothesis up by laying out a few assumptions:

In reinforcement learning, the purpose or goal of the agent is formulated in terms of a special signal called the reward, passing from the environment to the agent. At each time step, the reward is a simple number.

We could think about what counterexamples to those assumptions might be:

The reward signal originates internally, instead of originating from the environment. (e.g. meditation, or abstract introspection)
The signal is not received every time step, or isn't necessarily expected to be received at all. (e.g. seeking of transcendent experiences)

What might be common for these counterexamples is that the reinforcement learning mechanism itself undergoes spontaneous change. A signal that would have been positive before the spontaneous change might now be negative. The reward landscape itself might be completely different. From the agent's perspective, it might be impossible to evaluate what changed. The agent might have a 'subconscious' secondary algorithm that introduces changes in the learning algorithm itself, in a way that's decoupled from any reward-defined behavior.

Why would point 1 contradict the reward hypothesis? It should not matter where the reward originates from. Your second point seems to be about sparse rewards, but that still doesn't contradict the reward hypothesis (I think).

It seems like @Bananin was looking for some examples that are hard to logically navigate through in the way traditional reward accumulation is. Both points come down to an inability to define or accumulate rewards. If rewards are spontaneous, or if the agent changes its accumulation mechanism between rewards, it makes things difficult to extrapolate and model.


text:
How could decision tree learning algorithms cope with imbalanced classes?
Decision trees and random forests may or not be more suited to solve supervised learning problems with imbalanced labels (or classes) in datasets. For example, see the article Using Random Forest to Learn Imbalanced Data, this Stats SE question and this Medium post. The information across these sources does not seem to be consistent.
How could decision tree learning algorithms cope with imbalanced classes?

Decision Tree learners, on their own, are not a good way to deal with imbalanced data. The most commonly used algorithms, by default, make no attempt to address this problem.
If you look carefully at the three sources you post, you will find that they actually all agree on this point.
Two of the sources actually propose methods of addressing this shortcoming, by making adjustments to the decision tree learning algorithms. The proposed adjustments are essentially standard solutions to these problems, being applied to decision trees.
An example technique, discussed in the first paper you reference, is changing the weightings of the classes. An inefficient/approximate way to do this is to increase the number of examples from the minority class. For example, if you had an 80/20 split, you could add 3 new copies of each minority class example to move to an 80/80 = 50/50 split. Of course, if you add new data points, your algorithm may take longer to run. Instead, you can just modify the weightings of the classes in your optimization function. This approach is algorithm-specific, and will depend on your loss function, but achieves the same effect, just without needing to increase the number of points you use.


text:
Are there any general tips for troubleshooting a VAE when apparently it is not learning?
I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the output) are a 12D vector. I tried several sizes for the latent space, but, for some reason, it's not training. From the beginning, the KL loss in almost zero (around 1e-10), while the reconstruction loss (MSE for Gaussian distribution) is around 1, and they basically vary around these values without learning anything further.
Are there any general tips for troubleshooting a VAE (I never trained one before)?
I am pretty sure that the code is right and the data for sure has a background and signal (the ratio is 10:1), so I am not really sure what I am doing wrong.

Hello bill welcome to AI:SE!  Maybe try checking on the gradients to see if they are vanishing? If you ' adjusted the input and output to match what I need' without adjusting the quantity of layers and ensuring that the gradients don't vanish or explode anywhere in the middle then that could be the issue.

@MichaelHearn thank you for the idea! I will look into it. However, my NN is 6-7 layers deep, so I doubt there is a major gradient problem (I trained a lot deeper ones without BatchNorm or anything special, without any problem).


text:
How is the general return-based off-policy equation derived?
I'm wondering how is the general return-based off-policy equation in Safe and efficient off-policy reinforcement learning derived
$$\mathcal{R} Q(x, a):=Q(x, a)+\mathbb{E}_{\mu}\left[\sum_{t \geq 0} \gamma^{t}\left(\prod_{s=1}^{t} c_{s}\right)\left(r_{t}+\gamma \mathbb{E}_{\pi} Q\left(x_{t+1}, \cdot\right)-Q\left(x_{t}, a_{t}\right)\right)\right]$$
If it is applied to TD($\lambda$), is this equation the forward view of TD($\lambda$)?
What is the difference between trace $c_s$ and eligibility trace?


text:
Choosing neural network output for prediction (regression) of a dynamical system
I’m trying to train a neural network to approximate the output of a dynamical system $dy/dt=f\left(y(t), u(t) \right)$, namely, given $y(0)$ and $u(t_i), i=1,2...N$ I want the network to predict $y(t_i), i=1,2...N$. So far I’ve thought of several approaches, namely

Predict the derivative $dy/dt (t_{i+1}) = f_1 \left(y(t_i), u(t_i) \right)$ and then compute $y(t_{i+1}) = dy/dt (t_{i+1}) \cdot dt + y(t_{i})$
Predict the increment $\Delta y (t_{i+1})= f_2 \left(y(t_i), u(t_i), \Delta t \right)$ and then compute $y(t_{i+1}) = \Delta y (t_{i+1}) + y(t_{i})$
Directly predict the next value $y(t_{i+1}) = f_3 \left(y(t_i), u(t_i), \Delta t \right)$

Which option is recommended?

authors: Серёга Петров


text:
What's the difference between RMSE and Euclidean distance, and when to use a custom loss?
I'm searching for a loss function that fits my project. Actually, I have two questions, but they are in the same direction. I take a look at the definition of the root mean squared error (RMSE) and the Euclidean distance and they look the same to me. That's why I want to know the difference between the two. What would be the difference if I use RMSE as a loss function or the euclidean distance?
The second question is how to search for a loss function. I mean I know it depends on the problem and common things are MSE for regression and cross-entropy for classification, but let's say I have a specific problem, how do I search for a loss function?
I also saw that some people use a custom loss function and most of the deep learning frameworks allow us to define a custom loss function, but why would I want to use a custom one? How do I get the intuition that I need a custom loss function?
To be more specific, I'm doing a project where I need to reduce the GPS error of a vehicle. I have some vehicle data and my neural network will try to predict the longitude and latitude, so it's a regression problem. That's why I thought that the Euclidean distance would make sense as a loss function, right? Now, somehow MSE also makes sense to me because it is getting the difference between prediction and ground truth. Does this make sense to you as a professional ML engineer or data scientist? And if there would be a custom loss function that you can use, what would you suggest and why?

For the first question, RMSE and Euclidean distance have no difference, not that i know of.
For the second question, you only need the common loss function for normal tasks.
MSE is a common loss function used in linear regression tasks as well as loss function similar in nature like the RMSE. For classification tasks, Cross Entropy Loss is preferred. For logistic regression, use Binary Cross Entropy loss. See this for details:

Cross-entropy loss, or log loss, measure the performance of a
  classification model whose output is a probability value between 0 and
  1. It is preferred for classification, while mean squared error (MSE) is one of the best choices for regression. This comes directly from
  the statement of your problems itself. In classification you work with
  a very particular set of possible output values thus MSE is badly
  defined.
To better understand the phenomena it is good to follow and understand
  the relations between
Cross-entropy
Logistic regression (binary cross-entropy)
Linear regression (MSE)
You will notice that both can be seen as a maximum likelihood
  estimator (MLE), simply with different assumptions about the dependent
  variable.
When you derive the cost function from the aspect of probability and
  distribution, you can observe that MSE happens when you assume the
  error follows Normal Distribution and cross-entropy when you assume
  binomial distribution. It means that implicitly when you use MSE, you
  are doing regression (estimation) and when you use CE, you are doing
  classification.

Source: https://intellipaat.com/community/2015/why-is-the-cross-entropy-method-preferred-over-mean-squared-error-in-what-cases-does-this-doesnt-hold-up
For a custom loss function, there is examples like the triplet loss and cases where you need to optimize two loss at once, then you need a custom loss function. For the case of triplet loss, it is used in one-shot learning tasks like face recognition. There, face images are fed through a CNN to get embeddings about description of the face. In the training phase there is three images per batch. One anchor, one positive (which is the same person as anchor
) and one negative (different person from anchor). The loss function, triplet loss, maximizes the distance between the anchor embedding and the negative embedding. In reverse, it also minimize the distance between the anchor embedding and the positive embedding. The loss function is like this:

As you can see, there is two components here, and they are subtracted to maximize the second component and minimize the first. The alpha value is to make the loss positive as the gradient descent optimizes the loss to be as near to zero as possible.
The case of style transfer also introduces a custom loss function. Here is the loss function:

Style transfer basically alters the input image to be styled like the style image of the system.
The loss function consist of style loss and content loss. The content loss decreases the distance of the output image and the style loss decreases the distance of the output image and the style in an artistic sense. The second loss makes the output image imitate the style of the the style image.
In both scenario a custom loss function is used. In both cases, the custom loss function is used to optimize two losses together, either positively or negatively. A weight can also be introduced to weight the two losses as shown in the second case. However the second case introduce a custom loss function. That leads to the second use of custom loss function. They can be used for specific tasks and optimize a specific goal. The first use for custom loss function can be explored but the second requires careful engineering and research.
So in short, custom loss function are used for either combining two loss or a specific task.
Hope I can help you.

thnx for your answer but with all respect I don't know how does that answer my questions. I mean I'm not a beginner with NNs,I already know the differences between mse/rmse and log loss and why is mse used for regression and cross entropy for classification. I also wrote in the question that I knew that. My question is about difference between rmse and euclidean direction and why would someone need to use custom loss if the standard losses works fine and are differentiable, why or when should I use a custom loss or how I get the intuition of using custom loss. that was my question

Sorry didn't add the final part. Wait a second ediiting my question. Thank you for your reminder @basilisk

I have changed the answer. Sorry for didn't adding it at first. Hope it can help you @basilisk

thanks for the update. it surely helped to understand a use case for it, but although it does make sense to me, I can't understand a generalized idea of it. I understood from your answer that it is helpful to use that triple loss for image recognition and that can be used when I have 2 losses to optimize but  in the third paragraph I gave an explanation about my project which is a multi regression task with FFNNs or RNNs and my NN should predict two values that mean I ll have two losses I think, I don't think if the triple loss is useful for me, also my NN should minimize both errors

so it ll not try to maximize one loss and reduce the other. I think probably a normal rmse loss can be applicable in this case, and that's  what actually the question is about. I think your answer is good and helpful but it point to a specific problem which is image recognition but if we talked in general, how or when and why does an ML Engineer decide to use a custom loss. let's say you are an ML engineer at work and you receive some project or task, how do you decide to tell your team for example hey people we should use a custom loss for this problem? I'm just curious about this

you can also read the last paragraph of the question, I explained what is my project about and it's a research so no one have made this before and I read many research papers but there is nothing about it. My NN will give two outputs but it is not necessary that I use two losses because I want to minimize both those output losses so my case is not like the triple loss case you mentioned.

So if a custom loss is used it is used for optimizing two different goals , not only optimize two things. The two goals maybe one positive one negative, or weighted loss. In your case of two output, you can just use a conventional loss function and optimize the output using the target as the two output have the same "goal" like the objective is to minimize the distance between the output and label, not one minimize one maximize. You don't need a custom loss function for this.

thanks for the answer, I also thought so I just wanted to here another point of view from the community experienced people

Also since your problem is to predict the longitude and latitude, both outputs are the same kind, so no weighted loss is required.

@basilisk I just wanted to add an example of a custom loss function. Ordinal Categorical Classification; meaning when order of the classes matter: given classes 1 to 5, identifying 1 as 2 is not as bad as identifying in as 5. There is no loss function for such a problem in Keras, so one has to use a custom function, or use regular classifier which do not use all the information available.


text:
Is it possible to train a CNN to predict the dimensions of primitive objects from point clouds?
Is it possible to train a convolutional neural network (CNN) to predict the dimensions of primitive objects such as (spheres, cylinders, cuboids, etc.) from point clouds?
The input to the CNN will be the point cloud of a single object and the output will be the dimensions of the object (for example, radius and height of the cylinder). The training data will be the point cloud of the object with the ground truth dimensions in a regression final layer?
I think it is possible for images since it will similar to a bounding box detection, but I am not sure with point clouds.

How exactly do the points relate to the object? Are they merely all contained within the object? Are they only around the surface of the object? Do they define the surface of the object? Are they random/approximate, or exact?

I'm working on a similar problem. I'm using a 2D point cloud of an object, for example, X and Y coordinates for height, and with that more simple data set I will train a regression model (currently working on that). In my opinion, this approach with dissecting complex point cloud into cross sections that contain wanted dimension and feeding that to the model will be more simple and easier for the training.

I know that this doesn't answers your question directly, but I hope it helps and I don't have enough points to comment on your question.


text:
Are Relational DBs and SQL used in Expert Systems?
In the book Prolog Programming for Artificial Intelligence, a large and intricate chapter (chapter 14) is dedicated to Expert Systems. In these systems, a  knowledge-database is represented through facts and rules in a declarative manner, and then we use the PROLOG inference engine to derive statements and decisions.
I was wondering: are there any examples of expert systems that represent knowledge through a standard Relational Database approach and then extract facts through SQL queries? Is there any research in this area? If not, why is a rule-based approach preferred?

A recent research example is the "Grind" system. Take a look at the paper Computing FO-Rewritings in $\mathcal{E} \mathcal{L}$ in Practice: from Atomic to Conjunctive Queries (2018) by Peter Hansen and Carsten Lutz. Here's the abstract.

A prominent approach to implementing ontology-mediated queries (OMQs) is to rewrite into a first-order query, which is then executed using a conventional SQL database system. We consider the case where the ontology is formulated in the description logic $\mathcal{E} \mathcal{L}$ and the actual query is a conjunctive query and show that rewritings of such OMQs can be efficiently computed in practice, in a sound and complete way. Our approach combines a reduction with a decomposed backwards chaining algorithm for OMQs that are based on the simpler atomic queries, also illuminating the relationship between first-order rewritings of OMQs based on conjunctive and on atomic queries. Experiments with real-world ontologies show promising results.


text:
If loss reduction means model improvement, why doesn't accuracy increase?
Problem Statement
I've built a classifier to classify a dataset consisting of n samples and four classes of data. To this end, I've used pre-trained VGG-19, pre-trained Alexnet and even LeNet (with cross-entropy loss). However, I just changed the softmax layer's architecture and placed just four neurons for that (because my dataset includes just four classes). Since the dataset classes have a striking resemblance to each other, this classifier was unable to classify them and I was forced to use other methods.
During the training section, after some epochs, loss decreased from approximately 7 to approximately 1.2, but there were no changes in accuracy and it was frozen on 25% (random precision). In the best epochs, the accuracy just reached near 27% but it was completely unstable.
Question
How is it justifiable? If loss reduction means model improvement, why doesn't accuracy increase? How is it possible to the loss decreases near 6 points (approximately from 7 to 1) but nothing happens to accuracy at all?

Loss reduction means model improvement, it does not in the wrong setup, wher random choise produces least loss. So it is some critical setup error. What classes do you have?
I got also thet recently experimenting with an encoder with too narrow coding layer - it just EQUILIZES the output with average values cause this state has minimum loss.

It's important to remember what exactly the loss is measuring, and have some typical values in mind.
The cross-entropy loss is $-\mathbb{E}_{x,y\sim p}\left[\log q(y|x)\right]$, where $p$ is the data distribution and $q$ is the model distribution.
A couple of points about the loss:

It's nonnegative, in the range $[0, \infty)$.
Predicting randomly (assuming balanced classes) gives loss equal to $\log k$.

In your case with four classes, the loss for a random classifier is $\log 4 \approx 1.39$. So the story for what happened with your model is probably that initially (due to initialization, etc) it predicted high but wrong confidence, such as giving 99% probabilities to certain classes. This gives very high loss, but then after training for a while it reduces its loss to just under the random loss by predicting 25% on all examples.

In your definition of cross-entropy, the CE will always be negative, given that log of a number between 0 and 1 (excluded) is always negative, but, in practice, the implementation of cross-entropy may use the negative of your formulation, so that you can minimize it (i.e. you want to conceptually minimize a "loss") rather than maximizing (i.e. the CE is defined in terms of the likelihood, so you want to maximise the likelihood).

Yep that's a good point!


text:
What is the use of softmax function in a CNN?
What is the use of softmax function? Why was it used at the end of fully connected layer in convolution neural network?

The main purpose of the softmax function is to transform the (unnormalised) output of $K$ units (which is e.g. represented as a vector of $K$ elements) of a fully-connected layer to a probability distribution (a normalised output), which is often represented as a vector of $K$ elements, each of which is between $0$ and $1$ (a probability) and the sum of all these elements is $1$ (a probability distribution).
In the case of a classification task, the $i$th element of the vector produced by the softmax function corresponds to the probability of the input of the network of belonging to the $i$th class (e.g. a dog).

but can it be replaced to argmax function (taking element with higher value as 1, everythin else is 0)

@user8426627 You could do that, but you might lose the probabilistic interpretation of the results (classification). At the end, you will have to make a decision, so you will choose one (or more) of those outputs (anyway). The most obvious decision is to choose the class with the highest probability, but this might not always be the case.

i just want to figure out, does softmax gains benefit when packpropagating from it compare to just use linear difference

@user8426627 I'm not sure what you mean by "linear difference". If you're asking for the advantages of the softmax function (compared to other functions), then maybe this is another question.

the square los, what has a derivative  expected - output. It is related to question, since it is about "What is the use of softmax function" if we have other one

@user8426627 I think that the usage of the softmax is due to the functionality it provides. For example, suppose you're trying to classify animals that are, in certain ways, very similar to each other: e.g. cheetahs and leopards. By having a probability distribution over the possible classes of a given input, you can understand if your model is certain or not regarding the class of the input. If your model tells you that there is a 40% probability that the input is a cheetah and 39% that it is a leopard, then you (and the model) will be a bit unsure regarding the actual class of the input.

In short the softmax function helps with multi-classification i.e output of more than one of two possibilities.
It works well this the categorical cross entropy.


text:
How do I know if the assumption of a static environment is made?
An important property of a reinforcement learning problem is whether the environment of the agent is static, which means that nothing changes if the agent remains inactive. Different learning methods assume in varying degrees that the environment is static.
How can I check if and (if so) where in the Monte Carlo algorithm, temporal difference learning (TD(0)), the Dyna-Q architecture, and R-Max a static environment is implicitly assumed?
How could I modify the relevant learning methods so that they can in principle adapt to changing environments? (It can be assumed that $\epsilon$ is sufficiently large.)

Hi and welcome to this community! Can you please tell us where you saw this definition of a "static environment". There are the notions of a stochastic and deterministic environments, which do not seem to be what you're refering to.

It seems to me that the default assumption is static, because, even if learning based on real-time, real-world data, the parameters are static. Thus, while the environment might change, the model being utilized is static..