text:
What's an appropriate algorithm for classification with categorical features?
My input data consists of a series of 8 integers. Each integer is a discrete token, rather than a relative numeric value (i.e. '1' and '2' are as distinct as are '1' and '100').  The output is a single binary value indicating success or fail.  For example:
fail,12,35,60,82,98,111,142,161
success,23,46,59,87,102,121,145,161
fail,13,35,65,83,100,102,122,161

I have say 500,000 of these entries.
Success or failure is determined by the combination of the eight tokens that go to make up the input. I am certain that no single token will dictate success or failure, but there may be particular tokens or combinations of tokens which are significant in determining success or failure, I don't know, but would like to know.
My question is, what kind of machine learning algorithm should I implement to answer the question of which tokens and combinations of tokens are most likely to lead to success?
In case it's relevant or useful, a few more notes on the input data:
There is a limited range of tokens (and thus integers) in each slot. So with this data input:
success,A,B,C,D,E,F,G,H

A is always say one of 1, 2, 3, 4 or 5.  B is always one of 6, 7 or 8.  C is always one of 9, 10, 11 or 12.  So in the general case, possible values for A are never possible values for the other slots and there are between 2 and 12 values for each slot.  No idea if that makes a different to the answer but wanted to include it for completeness.

What you have is called a classification problem with categorical features. That is, the features can be represented numerically, but the numbers have no relative meaning.
Algorithms that rely on smooth function approximation will probably not work well here. These would include classic approaches to regression, and also function approximation via a neural network. That's because the data are anything but smooth!
In contrast, classic classification algorithms like Quinlan's C4.5 decision tree learner, (implemented in the Weka Toolkit as J48, and possibly in SciKitLearn as DecisionTreeClassifier, though the documentation is less clear), are ideal for this: they actually work by splitting up numeric values into discrete categories anyway, so there's no issue at all for them. Most versions also support a way to pre-tag features as categorical, and the algorithms rely on the cross-entropy of each feature's categories, without making assumptions of smoothness.


text:
Can I combine two classifiers that make different kinds of errors to get a better classifier?
I have a dataset with 2,23,586 samples out of which i used 60% for training and 40% for testing. I used 5 classifiers individually, SVM, LR, decision tree, random forest and boosted decision trees. SVM and LR performed well with close to 0.9 accuracy and recall also 0.9 but tree based classifiers reported an accuracy of 0.6. After a careful observation, I found out that SVM and LR did not predict the labels of 20,357 samples identically. So Can I apply voting and resolve this conflict wrt prediction outcome? Can this conflict be due to an imbalanced dataset?

welcome to AI SE. Your question is ok, but the title could be more descriptive to your problem. Could you please clarify it a little bit?

How many of the 20357 cases that SVM and LR fails has been correctly classified by the remainder methods?

Your question leaves a lot of questions, but will try to assist in what l understood. when gathering your data please try by all means to make sure that there are equal samples per channels and monitor your model for label leaking

Yes, you can. There are a lot of different techniques, usually called Ensemble Methods.
A better approach might be to use something like AdaBoost along with a cheaper method like the decision trees you looked at. AdaBoost explicitly tries to train classifiers to correctly handle different parts of the data, rather than hoping that different methods turn out to do so by chance.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
How can one use Genetic Algorithms to solve the "15 Puzzle" (Mystic Square)?
How would one go about solving the 15 squares puzzle using a Genetic Algorithms approach?  In particular, I'd like to understand how you would
represent the "chromosome" in the evolving system.  That is, what's the relationship between the (artificial) "genes" and some sort of phenotypical expression w/r/t the problem.  It seems like genes would somehow represent moves or sequences of moves but I'm not entirely clear how this would work.

Yes, you are right. Sequences of moves, i.e. algorithm. I suggest you to read this site about GA, if you haven't already.

There are a few ways of handling this within GA's, but most of them actually amount to using some kind of Genetic Programming instead.
The simplest way, and most similar to what you've proposed is called linear genetic programming. In this representation, you break the genome into a set of equal-width pieces. Each piece is interpreted as a machine-language instruction for a virtual machine. In your case, plausible instructions might be "move left" or "move right". Most versions use variable-length genomes, so your program's length depends only on how many instructions there are.
Another approach is to use the standard LISP-like genetic programming system, which Koza documents in his book for other simple problems, like the Santa Fe Trail.
More complex encodings are also possible. Grammatical evolution is another one that is similar to a GA in spirit, but interprets a genome according to a CFG instead of as instructions to a virtual machine.

This is an old question and answer, and, certainly, GP may solve the actual problem, but maybe you should have explained how exactly one could solve the specific problem. For example, how could one encode a solution/individual in the case of tree-based GP? In any case, of course, if you don't have time, you don't have to spend it on this old question (that probably everyone forgot about, apart from me, of course :P)

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How would an AI understand grids?
OK, now I think an AI must view grids in a different way to computers.
For example a computer would represent a grid like this:
cells = [[1,2,3],[4,5,6],[7,8,9]] = [row1,row2,row3]

That is a grid is 3 rows of 3 cells.
But... that's not how a human sees it. A human sees a grid as made of 3 rows and 3 collumns somehow intersecting.
If an AI is built on some mathematical logic like set theory, it's like a set of rows which in turn is a set of cells.
So what would be a way to represent a grid in a computer that is more "human". And doesn't favor either rows or columns? Or is there some mathematical or programmatical description of a grid that treats rows and columns as equivalent?

If I understand you correclty, you are wondering if there is a way of describing a 2-dimensional surface without choosing which dimension is the first to be indexed? So you don't want to pick (x, y) vs (y, x) but have some other way of defining position in a grid?

Although it is common to represent a grid as two dimensional array in a computer program, this is not the only way to represent one. You could, for example, use a generalized graph structure made of linked nodes, with 4 links each. Many other representations are possible. Even if you use a 2d array, some languages would index the columns first rather than the rows (FORTRAN for instance).
That issue is really an aside though, because it conflates representation with reasoning. By choosing to represent the world in certain data structures, we can made it easier or harder for different AI algorithms to reason about them, but the reasoning can often proceed even if the representation is inefficient. That is, an AI algorithm can ask questions about the columns in the grid, even if the columns of the grid are not represented in a way that makes it programmitcally easy to group them together.


text:
Prove that in such cases, it is possible to ﬁnd an ERM hypothesis for $H_n$ in the unrealizable case in time $O(mnm^{O(n)})$
Let $H_1$ , $H_2$ ,... be a sequence of hypothesis classes for binary classiﬁcation.
Assume that there is a learning algorithm that implements the ERM rule in the realizable case such that the output hypothesis of the algorithm for each class $H_n$ only depends on $O(n)$ examples out of the training set. Furthermore, assume that such a hypothesis can be calculated given these $O(n)$ examples in time $O(n)$, and that the empirical risk of each such hypothesis can be evaluated in time $O(mn)$.
For example, if $H_n$ is the class of axis aligned rectangles in $R^n$ , we saw that it is possible to ﬁnd an ERM hypothesis in the realizable case that is deﬁned by at most $2n$ examples.
Prove that in such cases, it is possible to ﬁnd an ERM hypothesis for $H_n$ in the unrealizable case in time $O(mnm^{O(n)})$.

I've added the "homework" tag to this problem because it looks like it is a homework problem from the book "Understanding Machine Learning" by Shalev-Shwartz & Ben-David. I'm guessing you would like a hint for this problem, rather than a fully worked answer?

@JohnDoucette Not absolutely. I'm just reading the book. I came across the exercise, and I also have the solution to the exercise, but I cannot understand the solution. Hope you can help me!

Here's a rough proof sketch that might be clearer, but is probably less precise, than the solution you already have.

We have a sequence of hypotheses classes, but the problem actually only asks us to consider a single class. A hypothesis class is a set of possible models for classification that differ only in a set of parameter values.
We are told that the algorithm for selecting a hypothesis from $H_n$ uses only $O(n)$ samples from the training set, and that using those samples, it takes the algorithm $O(n)$ to find the hypothesis that minimizes the empirical risk for the class, provided that the ERM minimizing hypothesis has an empirical risk of 0 (i.e. it is realizable).
Further, if we pick some hypothesis at random from the class, we are told that the cost of determining its exact empirical risk is $O(nm)$, where $m$ is the total size of the training set, and $n$ is just a parameter associated with this particular hypothesis class $H_n$.
Observer that, in the unrealizable case, we cannot use the algorithm to find the risk minimizing hypothesis (which, by 2, runs in O(n)). No other algorithm is mentioned, but we are also told (in 2) that the algorithm can construct a hypothesis from $O(n)$ datapoints in $O(n)$ time, even if no feasible hypothesis can be found for those $O(n)$ points.
Suppose we wanted to find the hypothesis with minimal risk via brute force. We could compute every subset of the training dataset that is of the size $O(n)$ that the algorithm may be able to use to construct a hypothesis in $O(n)$ time. There are $O(m^{O(n)})$ such subsets (e.g. $m*(m-1)/2$ possible pairs, $m*(m-1)*(m-2)/6$ possible triplets, etc.).
Since there are $O(m^{O(n)})$ possible subsets of size $O(n)$, and our algorithm can compute a candidate hypothesis in $O(n)$ time, it costs us $O(nm^{O(n)})$ to find all possible hypotheses that our algorithm can produce for this dataset.
Since it costs us $O(nm)$ time to compute the emperical risk of a hypothesis generated by this algorithm, we can minimize risk by computing risk for each of the $O(m^{O(n)})$ hypotheses we made in step 6, and then selecting the one with lowest empirical risk. This costs us $O(mnm^{O(n)})$ time total. Our total runtime is then the sum of the cost to generate the models in step 6, and the cost to evaluate them here in step 7: $O(mnm^{O(n)})+O(nm^{O(n)})$.
Since $O(mnm^{O(n)})+O(nm^{O(n)}) \in O(mnm^{O(n)})$, we have proven the desired bound on runtime.


text:
What will change when workstations will have ARM Machine Learning Processors onboard?
lately we read that many manufacturers are forcing ARM architectures to be used on future workstations. One of ARM's recent announcements is a machine learning processor. What will change in terms of computing performance if ARM architectures become new standard, and these kinds of ML-focused chips are found in most devices?

It is not clear to me that much will change.

ARM makes many devices, mostly designed to consume as little power as possible. My guess is that most workstations will not contain ARM's ML processor, even if they contain an ARM CPU.
ARM's ML processor can do machine learning. It is specifically optimized for training convolutional deep neural networks, and it is a bit faster that most existing non-purpose built chips, especially for its price. In practice, chips like this one should make ML a bit cheaper and faster, but it's not a breakthrough product. This just looks like the next iteration of Moore's law.


text:
Can I constrain my neurons in a neural network in according to the orders of the input?
I'm working with data that is ranked. So the inputs are 1,2,3 etc. This means the smaller numbers (ranks) are preferred to the larger ones. Hence the order is important. I want to estimate a number using regression; however, with the constraint that the order of the numbers must be monotonic non-linear.
Imagine the following input table:
1, 2
2, 3
3, 1

For instance, if the output is 1000 for each input, then the estimation could be:
1 * (800) + 2 * (100) = 1000
2 * (300) + 3 * (60) = 780
3 * (150) + 1 * (400) = 850

Evidently, the estimated Xs are monotonic non-linear decreasing.
For the first column: 1 -> 800, 2 -> 600 (2x300), 3 -> 450 (3x150)
for the second columns: 1 -> 400, 2 -> 200 (2x100), 3-> 180 (2x60)
So here's the question. Can I ensure my model (neural network) enforces the given constraint? I am using Keras.

hi, do you mean that you want a control on the output of the network w.r.t the order of the inputs you put in?

Do you think you could pre-process the inputs so that they come in order?

No. I mean to put a constraint when finding the weights (or preferably neuron) that the less value the input neuron has the more values should be found to solve the regression problem. Pre-processing is not a bad idea, but the problem is that I have thousands of features that all describe an order for an input row.


text:
Is there any AI system for finding the best way to schedule university classes?
I was wondering whether there is an AI system which could be used to resolve the class clashes problem which mostly happens in universities. In almost every university students face this problem, where two or more courses that many students want to take together get scheduled at the same time. Does anyone know about a system which resolves this issue or someone who works on this problem?

In terms of constraint satisfaction, you may want to look into how Latin squares have been used in scheduling problems. The most basic scheduling applications are trivial, but here's a paper on a more complex form: MALS: multiple access scheduling based on Latin squares

Welcome to AI.SE @Israr Ali.
The problem of scheduling a timetable is an example of a constraint satisfaction problem, a topic long studied in AI.
There are many possible techniques to apply to this kind of problem. They can be organized into three broad categories:

Global search algorithms, like backtracking search can be used to try and find an assignment of times to classes that results in no conflicts at all. With proper heuristics, these can be quite fast in practice, even though their worst-case runtimes are poor. Russell & Norvig's AI: A Modern Approach has a very good overview of these approaches in Chapter 6.
Local search algorithms, like hill climbing search start by assigning every class a randomly selected time, and then making small adjustments that monotonically decrease the number of conflicts (e.g. swapping the times of two classes). These are often faster than backtracking approaches and are especially so if you are willing to accept a "pretty good" schedule that might still have fewer conflicts than an optimal one. Chapter 4 of AI: A Modern Approach provides a good introduction.
Planning approaches, particularly non-classical algorithms like GraphPlan can also be used for these domains. They make use of the structure of planning or scheduling problems in particular to re-frame the search problems addressed by Backtracking techniques. By using this domain-specific representation, they are able to achieve very high-quality solutions very quickly. Chapter 11 of AI: A Modern Approach covers Graph Plan in some detail, and could be a good introduction to these more specialized techniques.


text:
How do I predict the occurrence of rare events?
I am trying to predict crime. I have data with factors: location, keyword description of the crime, time crime occurred and so on. This is for crimes that occurred in the past.
I would like to treat the prediction of crimes as a binary classification problem. In this model, the data I have collected would form the "positive" examples: they are all examples of a crime happening. However, I am unsure what to use for the negative examples.
Obviously, most of the time there is no crime at the location, but can I use this as negative data? For example, if I know there was a crime at 7pm at location X, and no other crimes there, should I generate new negative data points for every hour except 7pm?
Ideally, I want to create probabilities of crime based on a set of factors.

It might be more informative to:

Label each combination of location, type, and time of crime with a crime rate. For example, theft, in Crystal City, at 11pm at night, occurs 20 times per year, or 0.4 times per resident per year.
Predict the crime rate, rather than individual events.

This avoids the need to have explicit examples of "non-crime", and lets you instead directly learn something related to the probabilities of crimes being committed (the rate).

That makes sense. Do not know if I have enough data for that. Many thanks.

This is hardly useful. That rate will be 0 most of the time, which corresponds to having explicit examples of "non-crime".

@jan-glx That will depend to a great degree on the timescale you look at. Using, say, the yearly rate, will yield non-zero statistics for most kinds of crime for the size of areas these surveys usually cover (neighborhoods; police districts). I have used this technique myself, and seen others use it specifically with crime data, with good effect.

You are asking to bin the data in bins that are so wide that most of bins are non-zero. Possible, sure, but then one won't be able to make use of all the features: keyword description of the time, time of the crime, location they span a space so large, there won't possibly be as many crimes. the approach you describe is useful if you want to predict crime based on features of the area but not if you want to use features of the crime.

I would go so far as to say that unless the training examples include predicate data- that is, data about conditions leading up to a crime or non-crime-, then you cannot have enough information to predict the occurence of a crime from conditions or events that happen in advance of a potential crime not yet committed.

This doesn't seem highly likely, and in fact, I have seen people use this kind of data to predict the probability that a crime will occur. In fact, it is likely you already do this yourself. For instance, where would you rather be at 3am: An affluent suburb, or the "bad part" of your town?

Are you saying that the training data can just include spatial data and other such information that is only available at the immediate time of the crimes?  I suppose that's true.  A crime map of a city and no other information would give a valid basis for estimating future likelihood of crimes at various places in the city; and adding the times that the crimes occurred would make it possible to predict, e.g., that it is risky to walk alone at 3am Sunday morning in a particular neighborhood.  So I agree.  I won't delete my answer because it plus these comments might be useful to someone.


text:
How do I compute log-likelihood for training set in supervised learning?
I am building a supervised learning model and I wish to compute the log-likelihood for the training set at the point of the minimum validation error.
Initially, I was computing the sum of all the probabilities with maximum value obtained after applying softmax for each example in the training set at the point of minimum validation error but that doesn't look correct.
What is the correct formula for the log-likelihood?

The log-likelihood function for the training set (in general, not for deep learning in particular) will depend on your choice of loss function.
I'm guessing you're using something like a quadratic loss function for a binary classification problem, since this is a common approach. In that case, the log-likelihood is the sum of logs of the squared differences between the target label and the softmax value produced by your model. If you want to compute the log likelihood under a particular set of parameters (say, those that minimize validation error), then you just use those parameters in the model when generating softmax values.

I too thought on the similar lines. I thought of using negative loss likelihood loss as given here. But this gives us negative log-likelihood loss whereas we are interested in calculating only log-likelihood. Could you please clarify my doubt?

I suspect that multiplying by -1 will give you the log likelihood then. Or you could minimize instead of maximizing.


text:
Can a crossover result in a node with no outgoing connections?
I'm currently implementing the original NEAT algorithm in Swift.
Looking at figure 4 in Stanley's original paper, it seems to me there is a chance that node 5 will have no (enabled) outgoing connection if parent 1 is assumed the fittest parent and the connection is randomly picked from parent 2.
Is my understanding of the crossover function correct and can it indeed result in a node with no outgoing connections?

The disabled/enabled bit in a connection gene indicates whether or not it should be expressed in the calculation of the network.
Here's an example:

This is a neural network and its corresponding connection genes which represent the layout of the network among other things.
The top connection gene going from 1 -> 3 that has a weight of 0.1 is expressed in the calculation of the network.
The bottom connection gene going from 2 -> 3 that has a weight of 0.4 is not expressed in the calculation of the network.
Calculating the network:

1633 * 0.1 = 163.3
Given the weights in the example the output of this network is 163.3

Hypothetically, if both connection genes had their enabled bit set to true (which can happen in the future, to quote the paper: "The disabled genes may become enabled again in future generations") then the output of the network would be:

(1633 * 0.1) + (10 * 0.4) = 167.3

To answer your question, the connection between two nodes still exists in the network regardless of whether the bit in the gene is enabled or disabled, but, if the bit in the gene is disabled it is not used when the output of the network is being calculated as I've shown above.


text:
Transposition table is only used for roughly 17% of the nodes - is this expected?
I'm making a Connect Four game using the typical minimax + alpha-beta pruning algorithms. I just implemented a Transposition Table, but my tests tell me the TT only helps 17% of the time. By this I mean that 17% of the positions my engine comes across in its calculations can be automatically given a value (due to the position being calculated previously via a different move order).
For most games, is this figure expected? To me it seems very low, and I was optimistically hoping for the TT to speed up my engine by around 50%. It should be noted though that on each turn in the game, I reset my TT (since the evaluation previously assigned to each position is inaccurate due to lower depth back then).
I know that the effectiveness of TT's are largely dependent on the game they're being used for, but any ballparks of how much they speed up common games (chess, go, etc) would be helpful.
EDIT - After running some more tests and adjusting my code, I found that the TT sped up my engine to about 133% (so it took 75% as much time to calculate). This means those 17% nodes were probably fairly high up in the tree, since not having to calculate the evaluation of these 17% sped up things by 33%. This is definitely better, but my question still remains on whether this is roughly expected performance of a typical TT.

I don't think that's necessarily a strange number. It's impossible for anyone to really tell you whether that 17% is "correct" or not without reproducing it, which would require much more info (basically would have to know every single tiny detail of your implementation to be able to reproduce).
Some things to consider:

The size of your transposition table / the number of bits you use for indexing into the TT. If you have a relatively small TT, meaning you use relatively few bits for indexing, you'll have bigger probabilities of collisions. That means you will have to replace existing entries more often, which means they might no longer be in the table anymore by the time you encounter transpositions during the search.
Where in the search tree are the nodes located that are recognized as transpositions already in the table? If you detect transpositions very high up in the search tree, you save a lot more search time than if you detect a transposition somewhere deep down in the search tree; once you detect a transposition that has already been searched sufficiently deep for the value stored in the table to be valid, you can cut off the complete subtree below that node from the search. This becomes more valuable as it happens closer to the root. So, just the number "17% of nodes" doesn't really tell us much.
Are you using iterative deepening? Since you mentioned only minimax + alpha-beta pruning in the question, I suspect you're not using iterative deepening. TTs become significantly more valuable once you do use iterative deepening, because then almost every state encountered becomes a "transposition". You'll already have seen all those states in a previous iteration with a lower search depth limit. Now, it is important to note with this combo of ID + TTs, that you can no longer completely cut off searches for all recognized transpositions. If an entry in the table holds a value that was computed with a search depth of $d$, that value will no longer be valid when performing a subsequent iteration of ID with a max search depth of $d + 1$ for example. However, that "outdated" value stored in the TT can still be used for move ordering, which can lead to significantly more prunings from alpha-beta pruning.
How efficient is the remainder of your engine? A TT is not 100% free, it takes a bit of additional time too (for example to compute the hash values for your game states). If the rest of your engine is relatively slow (i.e. inefficient implementation for playing moves, copying game states, etc.), the computational overhead of the TT won't matter much and even a low number of recognized transpositions will still be valuable. If the rest of your engine is very fast, it'll be more important to have a high number of transpositions for the TT to be really valuable.

As an "educated guess", I'd say the number of 17% you describe is not necessarily strange. Especially given your edit to the question, where you indeed mention that it is likely that transpositions are found high up in the tree (close to the root). When this happens, you immediately remove the probability of recognizing all those states deeper down in the tree of getting recognized as transpositions yourself. So, the pool of states that could "potentially" be found in the TT is much less than 100% of the states stored in the TT.
It's really just that though, just an educated guess. It's going to be very difficult for anyone to give a conclusive "yes" or "no".


text:
Does Monte Carlo tree search qualify as machine learning?
To the best of my understanding, the Monte Carlo tree search (MCTS) algorithm is an alternative to minimax for searching a tree of nodes. It works by choosing a move (generally, the one with the highest chance of being the best), and then performing a random playout on the move to see what the result is. This process continues for the amount of time allotted.
This doesn't sound like machine learning, but rather a way to traverse a tree. However, I've heard that AlphaZero uses MCTS, so I'm confused. If AlphaZero uses MCTS, then why does AlphaZero learn? Or did AlphaZero do some kind of machine learning before it played any matches, and then use the intuition it gained from machine learning to know which moves to spend more time playing out with MCTS?

Monte Carlo Tree Search is not usually thought of as a machine learning technique, but as a search technique. There are parallels (MCTS does try to learn general patterns from data, in a sense, but the patterns are not very general), but really MCTS is not a suitable algorithm for most learning problems.
AlphaZero was a combination of several algorithms. One was MCTS, but MCTS needs a function to tell it how good different states of the game might be (or else, it needs to simulate entire games). One way to handle this function in a game like chess or Go is to approximate it by training a neural network, which is what the Deep Mind researchers did. This is the learning component of AlphaZero.

John's answer is correct in that MCTS is traditionally not viewed as a Machine Learning approach, but as a tree search algorithm, and that AlphaZero combines this with Machine Learning techniques (Deep Neural Networks and Reinforcement Learning).
However, there are some interesting similarities between MCTS itself and Machine Learning. In some sense, MCTS attempts to "learn" the value of nodes from experience generated through those nodes. This is very similar to how Reinforcement Learning (RL) works (which itself is typically described as a subset of Machine Learning).
Some researchers have also experimented with replacements for the traditional Backpropagation phase of MCTS (which, from an RL point-of-view, can be described as implementing a Monte-Carlo backups) based on other RL methods (e.g., Temporal-Difference backups). A comprehensive paper describing these sorts of similarities between MCTS and RL is: On Monte Carlo Tree Search and Reinforcement Learning.
Also note that the Selection phase of MCTS is typically treated as a sequence of small Multi-Armed Bandit problems, and those problems also have strong connections with RL.

TL;DR: MCTS is not normally viewed as a Machine Learning technique, but if you inspect it closely, you can find lots of similarities with ML (in particular, Reinforcement Learning).

Welcome to the mine-field of semantic definitions within AI! According to Encyclopedia Britannica ML is a “discipline concerned with the implementation of computer software that can learn autonomously.” There are a bunch of other definitions for ML but generally they are all this vague, saying something about “learning”, “experience”, “autonomous”, etc. in varying order. There is no well-known benchmark definition that most people use, so unless one wants to propose one, whatever one posts on this needs to be backed up by references.
According to Encyclopedia Britannica’s definition the case for calling MCTS part of ML is pretty strong (Chaslot, Coulom’s et al. work from 2006-8 is used for the MCTS reference). There are two policies used in MCTS, a tree-policy and a simulation-policy. At decision time the tree-policy updates action-values by expanding the tree structure and backing up values from whatever it finds from search. There is no hard-coding on which nodes should be selected/expanded; it all comes from maximizing rewards from statistics. The nodes closer to the root appear more and more intelligent as they “learn” to mimic distributions/state and/or action-values from the corresponding ones from reality. Whether this can be called “autonomous” is an equally difficult question because in the end it’s humans who wrote the formulas/theory MCTS uses. 50 years from now it may not be called autonomous, or ML, but today it would probably at least "qualify".


text:
How to find a cost function for human data
Currently, I am interested in how NNs or any other AI models can be used for composing music.
But there are many other interesting applications too, like language processing.
I am wondering that: NNs generally need a cost function for learning. But for example, for composing music, what would be an appropriate cost function? I mean,  algorithms can't (yet) really 'calculate' how good music is, right?

You've hit upon the central conundrum of supervised learning: if you want a machine to learn to do something, you need to know how to explain what that something is.
In the case of music, there are several possible approaches:

Make one set of "bad" songs, and one set of "good" songs. Develop a measure of how similar two songs are (maybe euclidian distance between their discrete Fourier transforms is a good starting place?). Your cost function is then based minimizing the average distance to "good" songs, and maximizing the average distance to "bad" songs. This may not work well though, because good and bad songs might differ only in the occasional misplaced notes.
Move to a reinforcement learning paradigm. Listen to each song proposed by your network. Give is a score based on your subjective enjoyment. Your cost function is based on maximizing this score. This might work well, but again, it might not. Music is tricky.
Use unsupervised approaches. Reward your network just for making something that resembles music (perhaps using the Fourier transform approach above), without labelling good and bad. The advantage is that you don't need to decide what is good or bad, and so you can use a lot more music in your dataset. The drawback is music as a whole might be too diverse to learn easily from examples.
Treat your music as a sequence of notes, and train a generative model to predict future notes on the basis of past notes. You can then generate new music by starting the model with a set of notes and letting it generate new ones for a long time.

@NeilSlater Indeed! I should have thought of that. I've updated the answer.

About your last point: If I understand you correctly, an option would be to plug a rnn output into its input again, so that the predicted note of the previous timestep is the new input. But how would I train it then? Would I just let it follow existing music, and the error is the difference between the predicted next note and the actual next note in that piece of music?

@Ben That's the essential idea. There are specific kinds of neural networks that are intended to train on sequences in this way. The link in my last point is to an easy-to-use package for training LSTMs, which might be helpful if you want to go that route.


text:
How to create Partially Connected NNs with prespecified connections using Tensorflow?
I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).
I am a complete beginner in neural networks. I have been recently working with tensorflow & keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...
I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections.
I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.

So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!

In case that you want to connect a placeholder to a new layer you can do as below
x = tf.placeholder(shape=[None, 784], dtype=tf.float32) # mnist for example

x = tf.reshape(x, (-1, 784, 1)) # change to new shape (784 ,1)

x = tf.unstack(self._input,axis=1) # getting a list with 784 elements of 1

con = tf.concat(x[1,2,3,4,5]) # for instance you want only from 1 to 5 inputs/neurons

you can feed your new layer with above con which is only corresponds from the input 1 to 5.
you can apply same technique to a layer instead of tf.placeholder


text:
What is the best approach for writing a program to identify objects in a picture then crop them a specific way?
My works quality control department is responsible for taking pictures of our products at various phases through our QC process and currently the process goes:

Take picture of product
Crop the picture down to only the product
Name the cropped picture to whatever the part is and some other relevant data

Depending on the type of product the pictures will be cropped a certain way. So my initial thought would be to use a reference to an object identifier and then once the object is identified it will use a cropping method specific to that product. There will also be QR codes within the pictures being taken for naming via OCR in the future so I can probably identify the parts that way if this proves slow or problematic.
The part I am unsure about is how to get the program to know how to crop based on a part. For example I would like to present the program with a couple before crop and after crop photos of product X then make a specific cropping formula for product X based on those two inputs.
Also if it makes any difference my code is in C#

@pasabaporaqui I don't think the question is unclear at all. The title of the question doesn't just "ask for object identifaction", it already specifically asks for the sequence "identify objects, then crop them". That's exactly what's asked in the main body of the question as well.

@pasabaporaqui The question is 100% clear, we can clearly understand what the person wants; he wants to 1) identify objects because afterwards he wants to 2) crop the images, where different cropping methods should be used for different object types. There is absolutely no unclarity there. You might disagree that that approach is correct, you might argue that it might be better to try directly classifying "cropping methods" rather than doing so indirectly through a classification of "object types". That doesn't mean the question is unclear. It is advise which should go into a comment or answer

@pasabaporaqui That sounds like Machine Learning to me. Present the program with examples of how I want it to crop, then it should learn how to crop new situations by itself. This is where he says he already has identified the object type and therefore knows the cropping method to use, he'll presumably have different example input+output pairs for different cropping methods. That's just a rough example he has in mind though. He explicitly states that there's the part he's unsure about, so a different solution might be better.

Depending on kind and amount of data you posess, there are few approaches that you might consider.

Marking target objects on dataset and training CNN that returns coordinates of target object. In this case, remember that it is usually faster when training data ROIs have their coordinates relative to image size.
Use some kind of focus mechanism, like spatial transformer network:

https://arxiv.org/abs/1506.02025

This kind of network component is able to learn image transformation (including crop) that maximazes target metric for main classifier. This tutorial on pytorch:

https://pytorch.org/tutorials/intermediate/spatial_transformer_tutorial.html

shows some nice visualizations of STN results. Good thing about this kind of network is that, given enough data, it might learn proper transformation from image classification data (photo -> class). One does not need to explicitly mark target objects on image!
Object detection networks, like YOLO, Faster-RCNN. There are many tutorials on that matter, eg:

https://www.datacamp.com/community/tutorials/object-detection-guide

Saliency extraction. Simple idea is to generate heatmap showing what parts of input image activates classifier the most. I guess you could try calculate bounding box basing on such heatmap. Example research paper:

https://arxiv.org/abs/1805.08249

Points 1 and 2 are probably easies to implement, so I would start with them.

@pasabaporaqui It might be the case in point 3 and 4, but point 2 directly shows example that even person not knowing much about NN's might understand. Your criticism is counterproductive.

Question by OP is pretty clear. The title might be misleading. Still his problem is clearly explained: how to approach cropping image knowing how does it look like. As I understood OP does not have much experience in AI so I provide naive approach together with research subjects that are currently applied to his problem.

Thanks @paffciu thats correct I have 0 experience in AI professionally or from school and neither does anyone I know so I didn't even know where to start with this problem, this is exactly what I was looking for.

This sounds like you have a supervised learning problem. Microsoft provides a C# library, but it may not be suitable for your problem.
There are many different algorithms you could try, most of which will be within the sub-area of computer vision. Probably some kind of deep neural network is the best bet these days, but the right choice will probably depend on the details of your problem. Goodfellow et al. have a recent book that might be a good resource for deciding what to use.
Maybe someone who works in computer vision can give you a more specific suggestion.


text:
Is a switch from R to Python worth it?
I just finished a 1-year Data Science master's program where we were taught R. I found that Python is more popular and has a larger community in AI.
What are the advantages that Python may have over R in terms of features applicable to the field of Data Science and AI (other than popularity and larger community)? What positions in Data Science and AI would be more Python-heavy than R-heavy (especially comparing industry, academic, and government job positions)? In short, is Python worthwhile in all job situations or can I get by with only R in some positions?

You can't switch yourself to Python. You are not talking about a project you already wrote in R and wants to port it to Python, you are simply asking about learning Python (not forgetting R). Is it worth learning Python? Nowdays it is almost impossible to not learn Python if you work with anything related to data handling with a computer...

It might also be worth mentioning that if you are writing papers or preparing presentations that you would have to compare the tools there. So e.g. in the case of R you would have Sweave or KnitR for static papers or (beamer class) presentations and in the case and Jupyter Notebook (formerly IPython Notebook) for dynamic notebooks (which you can also export to TeX).

I want to reframe your question.
Don't think about switching, think about adding.
In data science you'll be able to go very far with either python or r but you'll go farthest with both.
Python and r integrate very well, thanks to the reticulate package. I often tidy data in r because it is easier for me, train a model in python to benefit from superior speed and visualize the outcomes in r in beautiful ggplot all in one notebook!
If you already know r there is no sense in abandoning it, use it where sensible and easy to you. But it is 100% a good idea to add python for many uses.
Once you feel comfortable in both you'll have  a workflow that fits you best dominated by your favorite language.

Comments are not for extended discussion; this conversation has been moved to chat.

Of course, this type of questions will also lead to primarily opinion-based answers. Nonetheless, it is possible to enumerate the strengths and weakness of each language, with respect to machine learning, statistics, and data analysis tasks, which I will try to list below.
R
Strengths

R was designed and developed for statisticians and data analysts, so it provides, out-of-the-box (that is, they are part of the language itself), features and facilities for statisticians, which are not available in Python, unless you install a related package. For example, the data frame, which Python does not provide, unless you install the famous Python's pandas package. There are other examples like matrices, vectors, etc. In Python, there are also similar data structures, but they are more general, so not specifically targeted for statisticians.
There are a lot of statistical libraries.

Weakness

Given its purpose, R is mainly used to solve statistical or data analysis problems. However, it can also be used outside of this domain. See, for example, this Quora question: Is R used outside of statistics and data analysis?.

Python
Strengths

A lot of people and companies, including Google and Facebook, invest a lot in Python. For example, the main programming language of TensorFlow and PyTorch (two widely used machine learning frameworks) is Python. So, it is very unlikely that Python won't continue to be widely used in machine learning for at least 5-10 more years.
The Python community is likely a lot bigger than the R community. In fact, for example, if you look at Tiobe's index, Python is placed 3rd, while R is placed 20th.
Python is also widely used outside of the statistics or machine learning communities. For example, it is used for web development (see e.g. the Python frameworks Django or Flask).
There are a lot of machine learning libraries (e.g. TensorFlow and PyTorch).

Weakness

It does not provide, out-of-the-box, the statistical and data analysis functionalities that R provides, unless you install an appropriate package. This might be a weakness or a strength, depending on your philosophical point of view.

There are other possible advantages and disadvantages of these languages. For example, both languages are dynamic. However, this feature can both be an advantage and a disadvantage (and it is not strictly related to machine learning or statistics), so I did not list it above. I avoided mentioning opinionated language features, such as code readability and learning curve, for obvious reasons (e.g. not all people have the same programming experience).
Conclusion
Python is definitely worth learning if you are studying machine learning or statistics. However, it does not mean that you will not use R anymore. R might still be handier for certain tasks.

It seems like the "out of the box" feature set is irrelevant.  The relevant thing is the availability of packages that do what you want, no?

@DeanMacGregor If you do not have access to the internet, this feature is relevant! Furthermore, if a programming language already provides a feature out of the box, you do not have to lose time looking for it.

Considering Python is heavily infested on being 'batteries included', its weakness is not one you encounter often. Especially since there are Python installations in use which do have statistical packages included. For data science in particular, Anaconda is quite popular and solves your immediate concern.

I didn't have this choice because I was forced to move from R to Python:
It depends on your environment: When you are embedded in an engineer department, working technical group or something similar than Python is more feasible.
When you are surrounded by scientists and especially statisticians, stay with R.
PS: R offers keras and tensorflow as well though it is implemented under the hood of python. Only very advanced stuff will make you need Python.
Though I'm getting more and more used to Python, the synthax in R is easier. And though each package has its own, it is somehow consistent while Python is not..
And ggplot is so strong. Python has a clone (plotnine) but it lacks several (important) features. In principle you can do nearly as much as in R but especially visualization and data wrangling is much easier in R. Thus, the most famous Python library, pandas, is a clone of R.
PSS: Advanced statistics aims definitely at R. Python offers a lot of everyday tools and methods for a data scientist but it will never reach those >13,000 packages R provides. For example, I had to do an inverse regression and python doesn't offer this. In R you can choose between several confidence tests and whether it is linear or nonlinear.
The same goes to mixed models: It is implemented in python but it is so basic there I can't realize how this can be sufficient for someone.

I would say yes. Python is better than R for most tasks, but R has its niche and you would still want to use it in many circumstances.
Additionally, learning a second language will improve your programming skills.
My own perspective on the strengths of R vs Python is that I would prefer R for a small, single-purpose program involving tables or charts, or exploratory work in the same vein. I would prefer Python for everything else.

R is really good for table mashing. If most of what a particular program is going to do is smoosh some tables into different shapes, then R is the thing to pick. Python has tools for this, but R is designed for it and does it better.
It's worth switching to R whenever you need to make a chart, because ggplot2 is a masterpiece of API usability and matplotlib is a crawling horror.
Python is well designed for general purpose programming. It has a very well designed set of standard data structures, standard libraries, and control flow statements.
R is poorly suited for general purpose programming. It doesn't handle tree-structured or graph-structured data well. It has some rules (like being able to look into and modify your parent scope) which are immediately convenient, but when used lead to programs that do are hard to grow, modify, or compose.
R also has some straightforwardly bad things in it. These are mostly just historical leftovers like the three different object systems.

To elaborate more on the last point: computer programming done well is lego where you make your own bricks (functions and modules).
Programs are usually modified and repurposed past their original design. As you build them it is useful to think about which parts might be reused, and to build those part in a general way that will let them plug in to the other bricks.
R encourages you to melt all the bricks together.

As others have said, it's not a "switch". But is it worth adding Python to your arsenal? I would say certainly. In data science, Python is popular and becoming ever more popular, while R is receding somewhat. And in the fields of machine learning and neural networks, I'd say that Python is the main language now -- I don't think R really comes close here in terms of usage. The reason for all of this is generality. Python is intended as a general programming language, and allows you to easily script all kinds of tasks. If you're staying strictly within a neatly structured statistical world, R is great, but with AI you often end up having to do novel, miscellaneous things, and I don't think R can beat Python at that. And because of this, I think Python and its packages will be receiving more support and development when it comes to the more cutting-edge tech.

This is totally my personal opinion.
I read in my office (at a construction site) that "There is a right tool for every task."
I expect me to face a variety of tasks, as a programmer. I want as many tools as I can "buy or invest in", as possible. One day one tool will help me solve it, some other day some other tool. R (for statistics) and Python (for in general) are two tools I definitely want with me and I think it is worth investment for me.
As far as switch is concerned, I will use the most efficient tool I know (where efficiency is measured by client's requirement, time and cost investment and ease of coding) . The more tools I know, the merrier! Of course there is a practical limit to it.
All this is my personal opinion and not necessarily correct.

Person who chases two rabbits catches neither

And yes, Python is more popular. I work in both but, business speaking, it's easy to find a job on Python than in R.
So, you could:

Pick Python because it is more popular.   However, you must start from scratch.

Or

Stay with R, after all, you have one year worth of training with R.  But it is not popular.

The suggestion here that learning an additional programming language will somehow leave you worse off is nonsense. Learning additional programming languages, especially those that are unfamiliar, will always improve your skills as a programmer in any language.

It sounds like you have invested 1 year for data science with R, and embedded into R environment, but want to explore python for data science.
First learn the basics of the python like how lists and tuple works and how classes and objects work.
Then get your hands dirty with some libraries like numpy matplotlib pandas. Learn tensorflow or keras and then go for data science.

authors: قالط هايف المطيري


text:
What would be the best approach to teach an AI to learn how to "sing" along a beat?
I have heard and read about HyperGAN, LSTM and a few other techniques, but I have a hard time piecing the overall concept together.
End Goal
Being able to input an instrumental and get an output of how to sing to that instrumental.
My Dataset
I have extracted pitch points from thousands of actual acapellas from real songs.
My Theory
Feed the AI a pitch point PLUS say 19 thousand points of the original song instrumental.
Illustration

The red line (on top) is the pitch viewed vertically (lower pitch down, higher pitch up) of the voice sung by the singer over time viewed horizontally.
The bottom image is the song's frequency viewed vertically (lower freq down, higher freq up) viewed horizontally over time.
We take a point in time of the instrumental, say 0 minutes 30 seconds, and extract 19k points of the FFT spectrum vertically and call this a frame.
We also take the same point in time of the voice pitch, and also refer to this as a frame.
So now we have a frame which contains 20 thousand data points, one being the pitch of the voice, and the rest being the frequencies of the songs content.
QUESTION
What kind of model could be used to teach the AI the correlation of the voice and the instrumental?
And also, I have a hard time understand how, once the AI is trained, how could just an instrumental be fed to the AI to output pitch values of how one COULD sing along to the song.
Like, training we need to input 20 thousand values, but when we want the AI to sing for us using just an instrumental, would it not still expect voice pitch input?
At what layer would the instrumental be tapped into? At the outer most right layer?
EDIT
My mind has been working on this in the background throughout the day,
and I am wondering if instead of feeding 19k points of instrumental data each frame (which would be points from the frequency domain), one could just feed the instrumental frame points (which would be points from the time domain).
Maybe that would be better, but then maybe the AI would get less "resolution" to work with, but could be trained faster (less computing power needed).
Let's say the frequency domain is fed (higher resolution), the AI could potentially find correlations from low notes, mid notes and high notes, in any combination (more computing power needed).

You have to define a function between voice and a corresponding sound, but there is no a best way to do it, it depends on what kind od output you care, for example you could create an algorithm that receives music and siging as separate "files" and then that algorithm finds the relation between them now you can give that algorithm a new music only file and it will create the singing part with the function of the previous music-singing "system".
This is sing this music as the other one was sang.
At least that is the way i would do it.

I don’t believe your answer is the correct answer subjectively nor objectively, but it sure has an interesting concept and a step in the right direction. +1


text:
Integration of Sentiment analysis in CRM
What is the process for integrating sentiment analysis in a CRM? What I am searching for is a system which analyzes the customer comments or reviews using the CRM and finds out the customer sentiment on the services provided by the system or company or a product.
I have done a sentiment analyzer which takes text and shows the sentiment of the text. Now I want to integrate the above-mentioned sentiment analyzer to a CRM, how can I do that?

This is surely going to depend 100% on the implementation of the CRM. Integration work involves understanding the specific system you are integrating with, and there is very little practical (in the "how to do it") that could be said about it in general, especially in the context of AI. If the CRM is completely outsourced and has an API that you are allowed to use, then the API docs are worth a look. Otherwise, your first task is probably to meet with the technology owner of the CRM system - within your own IT department or 3rd-party as appropriate for your project.

I got one paper (M. R. Yaakub “Integration of Opinion Mining into Customer Analysis Model“) on the integration of opinion mining in CRM, which helps to get an idea about the process involved.


text:
How can we use data augmentation for creating data set for face recognition and will the inverted faces on augmented images detected?
I saw when browsing we can use data augmentation for creating a dataset for face recognition. The augmented images may include inverted, tilted or distorted faces. Do the model detect the face from the inverted image. When I tried my model cant able to detect any inverted or tilted faces.

Did you check your inverted image ? Look at them, be sure your data augmentation function is not producing garbage.

I used augmentor lib for augmentation

This is the purpose of rotational augmentation. In fact you should augment your images at different angle as well than just inverting it. This will make your model angle agnostic.

I have done than by rotating image in smaller angles. Now model can detect the face. But is there something available to detect faces in the inverted image.


text:
Behaviour of cost
In doing a project using neural networks with an input layer, 4 hidden layers and an output layer ,I used mini batch gradient descent. I noticed that the randomly initialised weights seemed to do a good performance and gave a low error. As the model started training after about 200 iterations there was large jump in error and then it came down slowly from there. I have also noticed that sometimes the cost just increases over a set of consecutive iterations.
Can anyone explain why these happen? It is not like there are outliers or a new distribution as every iteration exposes it to the entire dataset.
I used learning rate 0.01 and regularisation parameter 10. I also tried regularisation parameter 5 and also 1.
And by the cost  I mean, the sum of squared errors of all minibatches/2m plus regularisation term error.
Further if this happens and my cost after the say 10000th iteration is more than my cost when I initialised with random weights (lol) can I just take the initial value? As those weights seem to be doing better.
The large jumps are the most puzzling.
This is the code
Any help would be greatly appreciated. Thanks

I would say just don't go for using regularization at first, try with lower learning rate =  0.0001 see the behavior. Try to post entire architecture of your model so that one can better answer your problem.

Yeah. I had posted the code. Tried Xavier initialisation. Decreased cost down to 0.0000075 and it converges fine. So I took it down.

I figured out the problem after a bit of trial and error. This article may help too
First i set about pruning the dataset and removed outliers. Then i initialised the weights to better values using xavier initialisation.
These made the model slightly better but still the problem occured.
I then set the learning rate down to 0.000000075 and it converged after about 10000 iterations. I guess it was overshooting the minimum and reaching a point on the other side of the minimum that was farther than the previous point. This resulted in increased magnitude of gradient in the opposite direction and the cost explosively went up to the order of billions.

@DuttA I would hazard a guess that the OP has not normalised the inputs.


text:
Would you term Google's Captchas as Turing Test?
Quoting from Wikipedia page on Turing Test

The Turing test, developed by Alan Turing in 1950, is a test of a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human. Turing proposed that a human evaluator would judge natural language conversations between a human and a machine designed to generate human-like responses. The evaluator would be aware that one of the two partners in conversation is a machine, and all participants would be separated from one another. The conversation would be limited to a text-only channel such as a computer keyboard and screen...

The definition made me wonder if we would term the captchas that show up on Google here it gives textual instructions to the user, bot or not, asking them to select images containing a specific object out of the given set of images.

In the standard Turing test (or imitation game), the interrogator can ask multiple arbitrary questions, while, in the case of captchas, usually, there's only one question or problem. Additionally, in the Turing test, the interrogator interactively communicates with both the human and the machine. Furthermore, captchas do not test the conversational skills but only the visual ones of the AI. Therefore, the existing captchas are not an example of the standard Turing test.


text:
What exactly does "channel" refer to in tensorflow/lucid?
Tensorflow/Lucid is able to visualize what a "channel" of a layer of a neural network (image recognition, Inception-v1) responds to. Even after studying the tutorial, the source code, the three research papers on lucid and comments by the authors on Hacker News, I'm still not clear on how "channels" are supposed to be defined and individuated. Can somebody shed some light on this? Thank you.
https://github.com/tensorflow/lucid
https://news.ycombinator.com/item?id=15649456

I guess they're talking about filter. They optimize an image to make a high respond on the filter (channel)

(I don't read the papers, I don't see them on the github page. Moreover, they don't seem to talk about channel on the github page you've linked)

They talk about them in the tutorial and the linked articles. https://colab.research.google.com/github/tensorflow/lucid/blob/master/notebooks/tutorial.ipynb https://distill.pub/2018/building-blocks/ https://distill.pub/2017/feature-visualization/

Ohh, my bad, I didn't see there are two links ^^" (I edited your post to better see the two links)

The channel they are talking about is the depth of the layer L.

In this image, the channel is 5. There are 5 $3*2$ filters
They optimize an image from noise, to better respond to the filter F, in the layer L. Then you obtain this kind of image, and you can try to interpret the prupose of that filter (it learns to detect eyes, faces, wheels, and so on)
EDIT: To be more precise, you visualise the channel only if you optimise images for every filters, otherwise, you obtain a filter visualisation (if you do the optimisation one time, for one filter)
They call it "channel", because in a colored image, you have 3 dimensions : witdh, height, and channel (for color), as layers which have also 3 dimensions.


text:
How to measure the reasoning capabilities of neural networks
Which possibilities exist to evaluate the visual reasoning capabilities of neural networks in the field of image recognition?
Are there methods to measure the ability of machine reasoning?
Or something more specific: Is it possible to measure if a network understood the concept of a car / a cat / a human without using the classification accuracy.

I think it's a duplicate of : https://ai.stackexchange.com/questions/1479/do-scientists-know-what-is-happening-inside-artificial-neural-networks (at least related)

@JérémyBlain thanks for linking that related QA.  My sense is this is a slightly different question, focused on how to measure capability as opposed to understanding the internal reasoning process.

You must define "machine reasoning", so that an answer can be given.

What do you mean by "understood the concept of a car" ? A classification algorithm need not know that cars are used to transport humans or anything else it's used for. It only needs to know what combination of features we decide to label as a car.
Or maybe you meant we could use some other metric for understanding that is not classification accuracy?
This is a kind of Catch-22 situation.
When we want to classify a car we can reformulate the task to learning what set of a combination of features do us humans decide to call a car. What labels we want to assign a combination of features is arbitrary to a degree. It preferably has to be  consistent for inference but an NN can learn a dataset assigned completely random features to a 100% accuracy.
For instance, perhaps in an alternative universe we label the front-view of a car as 'B' and the rear view as 'A' and the notion/label of 'car' is nonexistent: even though we are ostensibly referencing the same object in physical space, we can assign it different labels.
It follows that to measure if a model has properly interpolated what combination of features we like to call car, we ourselves must be able list out such features; and if we were able to do that, we would not not need the process of learning/teaching algorithms in the first place!
A much more interesting subject is testing the extrapolative powers of neural networks- something much more proximate to what we canonically refer to as 'reasoning'.
It's quite hard to measure this in static environments such as the classification of images, but the ability to extrapolate is more apparent in reinforcement learning settings where the model requires us to train it for times order of magnitudes above the time for needed for humans.
The whole notion of pristine reasoning in machine learning breaks down when it comes to induction(see sufficient reason). Thus, we resort to probabilities that can be roughly translated to "this line of reasoning yielded the correct result 99.9% of the time, so it would make sense that we keep using it.
In fact, the idea of perfect-as-possible induction has been mathematically formulated except it isn't computable :( Solomonoff Induction


text:
Why is BERT/GPT capable of "for-all" generalization?
As shown in the figure:

Why does token prediction work when "Socrates" is replaced with "Plato"?
From the point of view of symbolic logic, the above example effectively performs the logic rule:
∀x. human(x) ⇒ mortal(x)

How might we explain this ability?  Moreover, how is this learned in just a few shots of examples?
I think this question is key to understanding the Transformer's logical reasoning ability.
Below are excerpts from 2 papers:

I feel that I must point out that "logical reasoning ability" isn't a good way to view any of these types of networks.  Transformers are interesting in how they provide attention through a sequence, but it is still just a series of mathematics transformations, not logical reasoning.

@DavidHoelzer Whether or not this is logical reasoning is a matter of philosophy, especially on a scale this large. After all, the human brain is ultimately emergent, as is the behaviour of such models.

@DavidHoelzer There is ample evidence that BERT/GPT is capable of logic reasoning, to a certain degree.

Can you please provide the link to the code that allows us to reproduce what you're claiming? Also, where did you take this image from? You also say "There is ample evidence that BERT/GPT is capable of logic reasoning, to a certain degree." , can you provide more examples (with references to the implementation or paper) of this? One such example does not convince anyone that it's capable of logical reasoning.

Having said that, in the example that you give, it's very likely that these models just replaced Socrates with Plato in both cases because, statistically, that seems to be a very common thing to do (from the training data). We cannot say it's logical reasoning, without more info about your specific example. If these models were just trained with data, then this is probably just a byproduct result of the statistics in the training data.

@nbro:  Yes, I can add some excerpts from papers to support that.

@YanKingYin Thanks. Can you also provide the links to or names of the papers?

@nbro: OK, 1 sec...


text:
How to implement RL model with increasing dimensions of state space and action space?
I've read in this discussion that "reinforcement learning is a way of finding the value function of a Markov Decision Process".
I want to implement an RL model, whose state space and action space dimensions would increase, as the MDP progresses. But I don't know how to define it it terms of e.g. Q-learning or some similar method.
Precisely, I want to create a model, that would generate boolean circuits. At each step, it could perform four different actions:

apply $AND$ gate on two wires,
apply $OR$ gate on two wires,
apply $NOT$ gate on one wire,
add new wire.

Each of the first three actions could be performed on any currently available wires (targets). Also, the number of wires will change over time. It might increase if we perform fourth action, or decrese after e.g. application of an $AND$ gate (taking as input two wires and outputting just one).

It's not fully clear to me what you mean by "targets" here. Can you clarify that?

Action $a_0$ might be for example an AND gate and targets would be specific circuits representing bits. So AND action would act on two targets - bits, while e.g. NOT action would act just on one target.

I still don't get what is a target in RL terms. What would a target correspond to in RL or MDPs?

@nbro: My understanding is that the possible targets are part of the state, and both the state space and action space dimensions may increase as the MDP progresses.

@NeilSlater That's exactly what I wanted to say, but I couldn't formulate it as clearly as you did. I edited my question.

@brzepkowski It may be a good idea to describe the exact problem you're trying to solve with RL. That will provide some more context and clarity, although the question is now is clearer.

I suggest keep the mention of actions that add "targets" and the need to specify which target to work with in each action. If there is any important constraint on targets - a maximum number allowed, or that they must be arranged in a graph or in a discrete space (like a grid) that may be worth mentioning.

I added the precise description of what I want to achieve.


text:
Training Keras Towards Or Against Analog Value?
For example, if I want to do a cat and mouse AI, the cat would wish to minimize the time taken for it to catch the mouse and the mouse would want to maximize that time. The time is analog and thus I cannot use a traditional Xy method but need another method that goes like this:
network.train_against_value(X, y, determinator)
Here, X is more like where the cat and mouse are. y is where the cat or mouse should move, and determinator is the time taken for the mouse to be caught, where the mouse wishes to maximize this value through its output of y and the cat wishes to minimize it. There is one Xy pair for each decision made by the cat and mouse, but one determinator throughout one game. Many games are played to train the AI.
Example: X: (300, 300, 200, 200) -> (mousex, mousey, catx, caty)
Y: (1,3) -> (xmove, ymove) direction, the numbers are then tuned by code for the actual movement to be always 1.
Determinator: 50 -> time for mouse to be caught in seconds
Where it would train so that with every X inputted it outputs a y so that determinator is minimum. Is there a method for train_towards_value as well? If there is no prebuilt method, how do I create one? What is the technical name for this kind of training?
I have two neural networks for the cat and mouse, where the cat is slower than the mouse but is larger and could eat the mouse. Just consider the mouse is difficult to control from the neural network because of inefficiencies so that it is possible for the cat to catch the mouse.

Don't you just want 2 different neural networks (1 for cat and 1 for mouse)? In this case you can just train to a minimum of time or a maximum, right?

Yes, so how do I do that?

Could you make it clearer what X represents for you? I'm guessing that it has something to do with what the cat or mouse observes. Would you describe them as agents or bots in some kind of game? Perhaps explain a bit more what you mean by a "cat and mouse AI" . . .use [edit] to add details to the question, don't leave it all in comments.

It may also help if you give some simplified concrete examples of how you expect train_against_value to work with actual values for X, y and determinator. I don't think it can be done like that, nor that is what you want, but seeing how you are thinking in detail will really help towards writing an answer.

What is the technical name for this kind of training?

The name for the problem is Sequential Decision Making or Optimal Control.
There are a few different approaches you can take when solving this kind of problem. However, I think that the way that you are describing your project, Reinforcement Learning (RL) would match your approach the best.

I cannot use a traditional Xy method but need another method that goes like this:
network.train_against_value(X, y, determinator)

Although this method signature could probably be made to work by re-structuring one or other RL frameworks, it is more usual when using RL with neural networks to treat the experience-gathering and scoring systems - the core ideas behind RL - as a data generator for either a supervised learning problem or in some cases directly producing the output layer gradients.
One approach you could use is called Deep Q Networks (DQN) which effectively generates mini-batches for supervised learning of a neural network, based on gathering experiences.
In brief, DQN trains a neural network to predict what you are calling the "determinator", but that RL would call the "value" of each action. So you may move the action choice (y) to the input of the neural network, or alternatively predict multiple values (one output for each possble action) - the RL theory is the same for each approach here, it is an implementation detail. The agent (cat or mouse) would then pick the action with the highest predicted value by default (with the cat's values being the negative of the mouse's), trying other actions at random whilst training so as to learn all values. Whenever the agent gained new experience, it would add that to the training data so it could improve its predictions.
RL is a complex subject in its own right, and to begin understanding it properly you would do well to start by tackling even simpler problems that don't require neural networks or running two agents against each other. There is a good introductory book on the subject with the option to read a free PDF version: Reinforcement Learning: An Introduction (Sutton & Barto)

So is there a Keras library or something where I can plug and play the RF?

@Aphrodite: There is https://github.com/keras-rl/keras-rl but it won't be plug and play from your current design, you will need to modify it to fit RL. It has examples and tutorials, so you might be able to figure out from those.


text:
What is the best way to integrate unchangeable ethics into a chatbot
I am building a generative model chatbot as a research and learning project. One of the most important parts of my project is to research ways in which I can make this chatbot work in a consistently ethical fashion.
This chatbot is simply a single Seq2Seq network running on my local machine. It can't be interacted with over the internet (yet), although I may end up creating a way to do that. It has no feedback loops of any kind as of right now, though reinforcement learning with a loop might be helpful.
The idea is that there would be some sort of unchanging knowledgebase for the chatbot to use that has hard-coded ethical statements and values that the bot has no ability to change. Whenever a question is asked of the chatbot,before being inputted to the network, the knowledgebase is searched and relevant facts will be appended to the input (separated from regular inptu by  tokens.
My question is, will this even be effective at allowing it to generate its own responses yet still be confined to the ethical standards given it?
My main concern is that it may begin to ignore these "facts" over time, and they may become irrelevant.
Another (possibly much better) approach might be to use deep reinforcement learning. However I may find it difficult to implement with my existing Sequence-to-Sequence network.
So which would likely be better? Or perhaps I should try a combination of the two?

You should probably give some details about the architecture of your chatbot, and how/whether user inputs are used in any feedback loop to train it. The failure of Microsoft's Tay bot was in part due to how it was designed to learn by interacting with the internet, and it was trolled until it broke. A more static data model may have less issues, but of course would not learn immediately from user input (you would probably want to somehow filter training inputs from an open-to-internet bot regardless though)

Alright, I updated my info. It is not accessible via the internet (other than if someone were to ssh into my computer)

I'm attempting to build something similar to Google Assistant or Siri, but with a generative model.

Is your seq2seq model trained with many examples that are augmented with the extra appended facts? I.e. are you augmenting the training data or trying to alter inputs to a model that has already been trained on a corpus without the extra tokens?

It has not been trained yet; I'm still working out the kinks of getting the system working that way. However I do plan to train it with the tokens.

Basic Fairness
If by ethics the question means that the system design is to foster basic ethics in the context of a conversation, then at least three attributes can be designed into the system.

Egalitarianism — no favor or disregard on the basis of gender, age, beliefs, financial status, ancestry, or culture, even if those features of the conversant are known and useful to know in personalizing the conversation
Temporal fairness — ensuring that the interaction with new correspondents and veteran correspondents provide these with equal access to relevant information, even if the establishment of trust is an evidence based process controlled by the system (recommended)
Sharing — balancing the objectives of the system to serve the interests of others as much as the interests of the system and its stakeholders — The Golden Rule: Treat others as you wish to be treated.

These attributes can be encoded in fuzzy rules, incorporated into the value function of an ongoing learning algorithm, or expressed in the loss function used during training, directing what is learned by a network.  All of these strategies require the tracking of metrics and the uses of them in adjusting behavior, a kind of social awareness.
There are obvious ways to do this in the mathematical expressions of values, assets, advantages, affirmations, attentions, losses, errors, pains, or other learning feedback approaches are assembled.  In generative network topologies, a discriminative network would need to be able to recognize unfairness and selfishness using the above principles.  Such places specific requirements on the designs of metrics and how fairness and the balance of interests are quantified.
A sum of squares across the dimensions (features) of potential inequity that can develop during a conversation is a possibility.  Using the fourth power instead of squaring would imbue permissiveness for smaller inequities but a drastic aversion to greater inequities.
$$\ell = \sum {(b_i - b_j)}^4 + \dots$$
The variable $\ell$ is the loss function result and $b$ is the benefit to conversation participants $i$ and $j$ where the two might be a human and the chatbot, a human and a stakeholder of the chatbot production server, or two humans that are in separate conversations with the chatbot, or two humans that are in the same conversation with the chatbot.  Note the ellipsis indicates more terms that affect loss as needed.
Cognitive Ethical Sophistication
If by ethics the question means the kind of ethics passed from families that value honor and truth to their children or taught at Harvard Law, the system design may have to wait until cognitive abilities are further developed in computing machinery.  Otherwise the question author will have to further AI to achieve the ethical sophistication.
Research into semantic acquisition, abstraction, automated assembly of causal models, application of these abstractions and models in planning and execution, and the integration of these with natural linguistics is ongoing.  Broad cognition and What some might call wisdom escapes the products of AI as of this writing.
A sufficiently deep network may approximate dialog that demonstrates what appears to be ethical awareness if trained with data that represents stories and conversational actions deemed to be ethical in those stories.  However, compacting learning the layers of comprehension that humans acquire through books, movies, other media, family life, and community events over a few decades (if they acquire it at all) into a computing project may be an overoptimistic objective at this time.
Although a collection of classic works might demonstrate much of a high ethical standard, it is not labeled data.  One would need to identify which character was transformed in a way that demonstrates the valuable ethical standards and then configure the learning process to learn from the character arcs of ethical characters in the stories.
Inherent Feedback

It has no feedback loops of any kind as of right now, though reinforcement learning with a loop might be helpful.

There is no way to adapt without feedback.  If the system is adapting, some feedback mechanism is correcting conditions that are maladaptive, even if the mechanism is opaque to the observer or identified by some other name.  In the case of GANs, the feedback creates a balance through the discriminative and generative network components.
$$\Bigg(\Big((G, E_1) \Rightarrow D\Big) \Rightarrow G, E_2\Bigg) \Rightarrow D \dots$$
That does not exclude the possibility that additional feedback paths may provide additional design advantages.
Hard Coding Ethics
Can judicious behavior be hard coded?

The idea is that there would be some sort of unchanging knowledge base for the chatbot to use that has hard-coded ethical statements and values that the bot has no ability to change.
Will this even be effective at allowing it to generate its own responses yet still be confined to the ethical standards given it?

It is not unchanging knowledge in the domain of ethics that leads to justice, but unchanging principles that guide the detection of what is not ethical that guides a continuous improvement of ethical standards.  That is why even constitutions have rules for amendment.  In probabilistic terms, without the balance of doubt and certainty, no ethical growth can occur.
No one would read a book or watch a movie where the protagonist has no character arc because it is unremarkable.  When a story is remarkable it is because ethics are adaptive and the adaptation must be a new kind that is not already covered many times in previous books or movies.  People like to be surprised by these adaptations.  They must occur against all fears and opposing external forces that are trapping the protagonist in a prison of mediocrity.
The story climax is when the protagonist, against all the building forces that keep them unremarkable, takes a path that discards all previous planning and forges a new radically different but fundamentally better one.
Excellent Objective in the Question
The intention to develop a system that is ethical is honorable and ethical in itself, so the question author may imbue some of those qualities into the system being designed.  In my opinion, such would be a triumph of greater value to the field of AI than all other progress up until now.
To make such an advancement, the framing of the problem will require thinking in new directions.  Consider the stated concern.

It may begin to ignore these "facts" over time, and they may become irrelevant.

Two principles may prove elucidating in regard to this concern.

Every language in common use has the equivalent of the English words should and ought, indicating a universal sense that something is awry.  This detection and identification of imbalance is central to the definition of heroism and honor, which is making at least an attempt to re-balance it.  Even the Oracle in the Matrix, after creating imbalance, sits on the park bench talking to Sati in the warmth of a new balance.  Identifying what needs disruption and re-balancing and constructing a strategy to do so is the one thing in common across all ethical behavior.  Ethics is remedial.  Some ethical choices require unlearning, and that is the most timeless feature of ethics.
Some of the most ancient principles, such as the Golden Rule or the idea of transparency – that right thought and right action are inescapably coupled – are more pertinent to this project than anything one can find in a book about machine learning.  The computer cannot hide a malicious intention and then converse ethically.  The objectives of the stakeholders of the system will need to also be ethical.

First Decision in Approach
The decision to make is the level of abstraction to apply.  Should this chatbot be ethical or be ethically minded?
The first would require the imbuing of ethical character into the bot.  The second would require the imbuing of the acquisition of ethical character into the bot.  As difficult as the more abstract second choice may seem, the first may be unachievable.  We don't have an example of it.  We do, however, have one proof of concept for the second choice: Good people.
The other sub-question to address is regarding selection of AI components.

So which would likely be better [generative or reinforcement]? Or perhaps I should try a combination of the two?

If you are serious about this objective beyond the simple tracking of basic fairness metrics, then you may have to use elements of both and possibly new things that extend the list of AI components and principles to use.
Take One Step Back
One thing to understand about generative topologies is that they are not really adversarial, even if that word is in the name of a seminal paper.  They are usually highly symbiotic and involve equilibria.  What would the discriminative network be without the generative one?  What would the generative one generate without the discriminative one?
The two collaborate to create a feedback system exhibiting a balanced equilibrium.  This is like biological stasis, which is the genius of it.  It is also like chemistry.  Salt in saturated suspension in water involves both dissolving and crystallization in a constant expression of opposing reactions at a molecular level.  Similarly, all ethical decisions are decisions are made within the context of at least one equilibrium, and the details matter.
Every court case where a judge or jury is to apply the public perception of ethics to the case has two sides presented.  They may appear to be adversaries, but the entire legal process is a sophisticated and symbiotic design to force collaboration in the interest of justice.
Every educated journalist understands the importance of avoiding sensational reporting.  Dramatically presenting only one side of the story sells news and is a constant temptation.  Journalistic integrity demands striking an equilibrium.  The result of the ethical choice determines whether the news item educates the public or adds polarization to a culture that may already be partially crippled by extremism.
Take Another Step Back
Reinforcement is an odd name to give what is essentially Agile planning via the probabilistic projection of expected advantages.  Should lean methods for strategic approaches to conversation be in this ethical chatbot?  If ethical behavior is defined to include telling stories that convey wisdom, then yes.  Roger Schank's work is of great interest in that case.  His work in story based reasoning may be a worthy direction of study.
Adapting the foundations of ethics to the ever changing story inherent in every significant conversation may require planning.  It is possibly unavoidably inherent every conversation when the intention of the participants is to do more than exchange pleasantries.

Thank you for this very informative answer! To clarify, my "knowledge base" (or at least the ethical side of it) would be made up of ethical principles rather than some sort of "fact". And my meaning was best described under your section "Cognitive Ethical Sophistication".