text:
Why is there tanh(x)*sigmoid(x) in a LSTM cell?
CONTEXT
I was wondering why there are sigmoid and tanh activation functions in an LSTM cell.

My intuition was based on the flow of tanh(x)*sigmoid(x)

and the derivative of tanh(x)*sigmoid(x)

It seems to me that authors wanted to choose such a combination of functions, the derivative would make possible big changes around the 0, since we can use normalized data and weights. Another thing is that the output would go to 1 for positive values and go to 0 for negative values which is convenient.
On the other hand, it seems natural that we use sigmoid in forget gate, since we want to have a better focus on the important data. I just don't understand why there cannot only be a sigmoid function in the input gate.
OTHER SOURCES
What I found on the web is this article where the author claims:

To overcome the vanishing gradient problem, we need a method whose second derivative can sustain >for a long range before going to zero. Tanh is a good function that has all the above properties.

However, he doesn't explain why this is the case.
Also, I found the opposite statement here, where the author says that the second derivative of the activation function should go to zero, however, there is no proof for that claim.
QUESTION
Summing up:

Why cannot we put a signal with just a sigmoid on the input gate?
Why there are tanh(x)*sigmoid(x) signals in the input and output gate?

The tanh functions within the cell represent cell output or cell state. These are the values that either get passed on to other layers, or within the layer to the next time step. In theory, other activation functions could be used here according to taste, similar to other feed-forward or RNN networks. However, the -1 to 1 output range of tanh is useful, and I expect tanh has been experimentally validated as a good general case activation function here.
The sigmoid functions are used as soft gates for manipulating the raw RNN values. Importantly for your analysis, there is no sigmoid that takes the same input as any tanh. Each of the green boxes in the cell diagram in your question has a separate learnable set of weights applied to the combined input+hidden_state vector.
That means that your analysis of tanh(x)*sigmoid(x) is moot. The function is effectively tanh(x)*sigmoid(y) because inputs to each activation function can be radically different.
The intuition is that the LSTM can learn relatively "hard" switches to classify when the sigmoid function should be 0 or 1 (depending on the gate function and input data). As the weights are independent on the gates and input value processing components, the gradients to the cell output and state components are not composed in a combined function, but simply multiplied by the current value of the relevant switch. A muliplying hard switch of 1 will allow the gradient to flow back directly from the output loss to the point at which the gate decision was made - depending on which gate was activated, this improved gradient signal will either be to the input processing weights or the hidden state procesing weights.
It is also possible for the input and cell state processing to be mixed in various combinations, and the gradient is not guaranteed strong. However, in situations requiring strong memory-like signals (such as using punctuation characters when processing text), it is possible to observe LSTM learning those signals, effectively classifying inputs with high confidence (close to either 0 or 1), thus creating toggle switches, counters etc, within the cell state vector.

I think a better way to understand LSTMs is by their purpose, instead of gradients and distributions.
If you analyze the interactions of each gate with the cell state, you'll realize that LSTMs essentially implement differentiable memory as a counter.
There are 2 important things to realize:

Each sigmoid/tanh is preceded by a linear projection, so the output of the sigmoid of the forget gate is different to that of the input gate.
Sigmoid outputs are mostly 0 or 1, while tanh outputs are mostly -1 or 1.

Following the numbering in the diagram:

The forget gate generates a binary mask which is multiplied with the previous cell state. Items with 0 in the mask are 'reset' to 0, while items with 1 in the mask are passed through unchanged.
The input gate generates 2 masks using sigmoid and tanh, whose product is added to the cell state. The tanh mask with, -1 or +1 outputs, determines whether to decrement or increment items in the cell state. The sigmoid mask determines whether an item should be updated at all, similarly to the forget gate.
The output gate determines what to expose to the subsequent layer and state using a similar logic. The tanh here acts as a 'binarizer' to simplify the cell state (which can theoretically be from -infinity to +infinity) to a binary -1 or +1 value, while the sigmoid masks out irrelevant items.

So while you could replace sigmoid and tanh with other activation functions, they make the most sense in this context and have the added benefit of well-defined derivatives. Remember that LSTM's were introduced in 1997 when tanh and sigmoid activation functions were still popular, and ReLU had not taken over yet (~2011).

The purpose of the tanh and sigmoid functions in an LSTM (Long Short-Term Memory) network is to control the flow of information through the cell state, which is the "memory" of the network.
The sigmoid function, also known as the "gate," is used to determine which information is allowed into the cell state. Specifically, it controls the flow of information through the "input gate," which determines which new information is added to the cell state.
The tanh function is used to regulate the values in the cell state. This is important because the cell state can become very large or very small over time, which can make it difficult for the network to learn. The tanh function ensures that the values in the cell state are always within a reasonable range (-1 to 1), which makes it easier for the network to learn.
Together, the sigmoid and tanh functions work together to regulate the flow of information through the LSTM network, allowing it to selectively update its memory while keeping the values in the cell state within a reasonable range. This makes the LSTM network especially effective for tasks that require long-term memory and the ability to selectively forget or remember information over time.


text:
What does it mean to apply decomposition at inference-time in a machine translation system?
I'm reading this paper for sub-character decomposition for logographic languages and the authors mention decomposition at inference-time. They're using Transformer architecture.
More specifically, the authors write:

We propose a flexible inference-time sub-character decomposition procedure which targets unseen characters, and show that it aids adequacy and reduces misleading overtranslation in unseen character translation.

What do inference-time and inference-only decomposition mean in this context? My best guess would be that inference-time would be at some point during the decoding process, but I'm not 100% clear on whether that's the case and, if so, when exactly.
I'm going to keep digging and update if I find something helpful. In the meantime, if anyone needs more context just let me know.

I've found this article that seems to answer my question:
https://hazelcast.com/glossary/machine-learning-inference/
From this, my understanding is that inference-time describes when a machine learning system is put into use following training; so basically at the time of task application.
I think this would mean that the paper's authors are stating that the decomposition of sub-characters is occurring whenever the model is actively translating languages in a production environment.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
What does the line of code "self.buffer[-1] = observation" do in this BufferWrapper class for DQN?
So the code is related to using a buffer
class BufferWrapper(gym.ObservationWrapper):
    def __init__(self, env, n_steps, dtype=np.int):
        super(BufferWrapper, self).__init__(env)
        self.dtype = dtype
        old_space = env.observation_space
        self.observation_space = gym.spaces.Box(old_space.low.repeat(n_steps, axis=0),
                                                old_space.high.repeat(n_steps, axis=0), dtype=dtype)

    def reset(self):
        self.buffer = np.zeros_like(self.observation_space.low, dtype=self.dtype)
        return self.observation(self.env.reset())

    def observation(self, observation):
        self.buffer[:-1] = self.buffer[1:]
        self.buffer[-1] = observation
        return self.buffer

It is used to basically do some image processing so that the DQN is fed some transformation of the image. This link provides some higher-level logic behind some operations.
How can I actually understand what's the reason behind the code? Almost all repos have the exact same lines with no explanation (e.g. atari games GitHub repo).
My specific question is what is the purpose of the line self.buffer[-1] = observation?
In my case, my observation is a (7*1) array and I have to return that in an appropriate manner from the observation function.
The book has some mention of this class but I couldn't understand much from it

I don't know if you're confused about this code because you're not very familiar with Python or reinforcement learning (specifically, DQN and experience replay), but that code should be very clear to you if you know Python, but maybe you're not very familiar with DQN.
Let's take a look at the observation method.
def observation(self, observation):
    self.buffer[:-1] = self.buffer[1:]
    self.buffer[-1] = observation
    return self.buffer

self.buffer[:-1] = self.buffer[1:] essentially drops/forgets the first (which is also the oldest) observation in the buffer.

self.buffer[-1] = observation adds the new observation (passed as parameter) as the last element of the buffer

You can execute the following code to see what that method does.
buffer = [10, 7, 3]
print(buffer)

buffer[:-1] = buffer[1:]

observation = 5
buffer[-1] = observation

print(buffer) # [7, 3, 5]

If you are not familiar with the experience replay technique, you can take a look at this or this answers.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How do I use machine learning to create an optimization algorithm?
Let's say that I want to create an optimization algorithm, which is supposed to find an optimum value for a given objective function. Creating an optimization algorithm to explore through the search space can be quite challenging.
My question is: can machine learning be used to automatically create optimization algorithms? Is there any source to look at for this?

if you can explain the challenges that you know, or even break it down into parts, it will give useful hint of some sort of structure for the answer

Machine learning has been used to automatically learn new optimization/learning algorithms. This task is often known as meta-learning, i.e. you learn to learn, in this case, an optimization algorithm, but note that meta-learning does not just refer to learning optimization algorithms (see this blog post).
The blog post Learning to Optimize with Reinforcement Learning (2017) is a good introduction to the topic and focuses on the approach proposed in this paper Learning to Optimize (2016), which uses reinforcement learning to solve this problem: more specifically, they learn a policy (in practice, represented as a neural network) that represents the learned optimization algorithm.
There are other related approaches: for example, you may be interested in the paper Learning to learn by gradient descent by gradient descent (2016, NeurIPS).


text:
How can I use a ResNet as a function approximator for pixel based reinforcement learning?
I'd like to use a residual network to improve learning in image-based reinforcement learning, specifically on Atari Games.
My main question is divided into 3 parts.

Would it be wise to integrate a generic ResNet with a DQN variant?

I believe ResNets take a long to train, and therefore would it be realistic to train on an Atari simulator? What would the downsides be?

Are there any fast ResNets that can be used for such purposes? Perhaps a fast ResNet that is specifically designed for online settings?

Residual Network are usually deeper and hence take more time to train. EfficientNet are trying to tackle this. However, the latest advice show that the architecture tend to play a crucial role in the performance of an RL algorithm, which might motivate you to do this.
There is recent work on Neural Architecture Search applied to RL tasks (cf https://arxiv.org/pdf/2106.02229.pdf)
However, I can advise you to look at Rational Activation Functions (cf https://arxiv.org/pdf/2102.09407.pdf) employed in RL which provide a huge boost to the agents, while adding negligible parameters and negligible extra training time.


text:
What is the name of this letter $\mathcal{J}$?
What is the name of this letter $\mathcal{J}$ in the following deep learning equation? And what alphabet it is from?
$$\mathcal{J} = \frac{1}{m} \sum_{i=1}^m \mathcal{L}^{(i)}$$

I’m voting to close this question because it seems to me that it belongs to https://tex.stackexchange.com/.

It's an uppercase "J" from the math calligraphy alphabet, i.e. \mathcal{J} in latex.
$\mathcal{J}$


text:
How to code an $\epsilon$-soft policy for on-policy Monte Carlo control?
I was trying to code the on-policy Monte Carlo control method. The initial policy chosen needs to be an $\epsilon$-soft policy.
Can someone tell me how to code an $\epsilon$-soft policy?
I know how to code the $\epsilon$-greedy. In $\epsilon$-soft, there are inequalities in place of equalities which is the issue for coding the $\epsilon$-soft.

You cannot code an $\epsilon$-soft policy directly, because it is not specific enough.
A policy is $\epsilon$-soft provided that there is at least a probability of $\frac{\epsilon}{|\mathcal{A}|}$ for choosing any action, where $\mathcal{A}$ is the set of all possible actions.

I know how to code the $\epsilon$-greedy.

Then you already know how to code the most commonly-used $\epsilon$-soft policy, because an $\epsilon$-greedy policy is an $\epsilon$-soft policy.

there are inequalities in place of equalities which is the issue for coding the $\epsilon$-soft

That is correct. In fact $\epsilon$-soft can be thought of as a constraint or test. So you could write some code that tested whether any policy was an $\epsilon$-soft policy for any given value of $\epsilon$. Or you could write code that determined the value of $\epsilon$ for any policy.
Slightly harder would be code that forced a supplied policy to meet constraints of being $\epsilon$-soft, because adjusting any low probabilities to be high enough would mean reducing other probabilities, and there are multiple ways you could do that.
However, a really simple way to make any starting policy $\pi$ into an $\epsilon$-soft variant is to make the policy choice in 2 steps - first step choose between the original policy with probability $(1-\epsilon)$, and with probability $\epsilon$ choose a fixed policy with equal probability for each action. Second step, evaluate whichever policy the first step chose to determine the action.


text:
How can I estimate how many photos I need to train ResNet-50 for image classification?
I am working on a project where I have to classify around 1000 unique objects. I'm trying to plan how much training data I will need to collect. I was planning on using ResNet-50. Is there anyway I can estimate the amount of photos I should plan to collect ahead of time (assuming I will collect an equal amount of photos of each class)?

What you want to calculate/estimate is known as the sample complexity in computational learning theory. If you knew the VC dimension of the neural network, you may be able to estimate the sample complexity, but your estimate (bound) may not be very tight anyway (maybe because the estimate of the VC dimension is also not tight). Here is more info about the VC dimension and the VC dimension of a few neural networks.
If we were able to easily compute (an accurate estimate of) the sample complexity in all cases, this would be extremely useful, as we could exclude (or not) that the size of the training dataset is the problem of under-fitting/over-fitting.
Nowadays, I think that most people do not usually estimate the sample complexity (I could be wrong, i.e. there may be simple ways to do this approximately, apart from saying that you probably need more training data points than the number of parameters, which is what many practitioners will tell you, maybe misleadingly), but, instead, collect as much data as possible, then train the neural network with that data; if it performs poorly on the test dataset, then you may try to collect more data (given that it's an indication that the neural network did not learn the target function); if you see that the test performance initially increases but then decreases while the training performance is always increasing, it may be an indication of over-fitting, so you may want to use a regularisation technique (like dropout), which you should probably use anyway, especially when you have a small dataset.
I also want to point out that, with the discovery of the phenomena double descent and grokking, these traditional guidelines or practices (e.g. early stopping) may sometimes be misleading (I don't know when or why), i.e. just keep in mind that sometimes they may not be applicable.

Worth adding a standard way to assess whether more data would help = plotting a learning curve where the x axis is amount of data used to train, and y axis is performance of the model on a (fixed) validation set. If it plateaus, then more data may not help (with the specific model being tested), if it is still trending upwards, more data probably will help.


text:
Is Positional Encoding always needed for using Transformer models correctly?
I am trying to make a model that uses a Transformer to see the relationship between several data vectors, but the order of the data is not relevant in this case, so I am not using the Positional Encoding.
Since the performance of models using Transformers is quite improved with the use of this part, do you think that if I remove that part I am breaking the potential of Transformers or is it correct to do so?

Maybe you could make a little example of the inputs and how they should be given to the transformer, to make it clear why the order of the data should not be relevant.

Positional Encodings in Transformers exist to give the model some information about the position of the embedding. This makes sense in fields like NLP or Time Series Data, since the position(order) matters in this case.
However, since you say that order of the data is not relevant in your use case, positional encoding would not be necessary.


text:
How to calculate the entropy in the ID3 decision tree algorithm?
Here is the definition of the entropy
$$H(S)=-\sum_{x \in X} p(x) \log _{2} p(x)$$
Wikipedia's description of entropy breaks down the formula, but I still don't know how to determine the values of $X$, defined as

The set of classes in $S$

and $p(x)$, defined as

The proportion of the number of elements in class $x$ to the number of elements in set $S$.

Can anyone break this down further to explain how to find $p(x)$?

Suppose you have data:
color  height  quality
=====  ======  =======
green  tall    good
green  short   bad
blue   tall    bad
blue   short   medium
red    tall    medium
red    short   medium

To calculate the entropy for quality in this example:
X  = {good, medium, bad}
x1 = {good}, x2 = {bad}, x3 = {medium}

Probability of each x in X:
p1 = 1/6 = 0.16667
p2 = 2/6 = 0.33333
p3 = 3/6 = 0.5

for which logarithms are:
log2(p1) = -2.58496
log2(p2) = -1.58496
log2(p3) = -1.0

and therefore entropy for the set is:
H(X) = - (0.16667 * -2.58496) - (0.33333 * -1.58496) - (0.5 * -1.0)
     = 1.45915

by the formula in the question.
Remaining tasks are to iterate this process for each attribute to form the nodes of the tree.

how are p1, p2, and p3 calculated?

p1 = 1/6 = 0.16667
p2 = 2/6 = 0.33333
p3 = 3/6 = 0.5

Very nice example that makes perfect sense to me. To get the value into expected range [0..1] normalization is needed.
$H(S)_{norm}=\frac{H(S)}{log_2(|S|)}=\frac{H( \{good, bad, bad, medium, medium, medium\} )}{log_2(3)}=\frac{1.459147917027245}{1.584962500721156}=0.9206198357143052 \in [0..1]$
✔

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is there a service that can search for answers to general questions?
I was looking for an API service where I can ask it a general question (for example, when was Einstein born?) and retrieve an answer from the web.
Is there any available service to do that? Have tried Watson services, but didn't work as expected.

https://www.wolframalpha.com/input/?i=when+was+Einstein+born

I need an API to use it

You can use Google
https://encrypted.google.com/search?hl=en&q=when%20was%20Einstein%20born
and parse the response.
Wolfram ALPHA is another candidate.
http://m.wolframalpha.com/input/?i=what+year+was+Einstein+born&x=0&y=0
You can parse the returned html and see "Result:" div.

You could use dbPedia and/or wikidata.  I think Wikidata supports SPARQL now, but don't quote me on that.  dbPedia definitely supports SPARQL.
If you're not interested in writing SPARQL queries by hand, you could use something like Quepy. In fact, the Quepy demo demonstrates doing natural language queries against Freebase and/or dbPedia.
You could possibly also incorporate OpenCyc.
If you want to roll something of your own, you might want to read some / all of the research papers published by the team from the START project at MIT.


text:
What is the most effective way to build a classifier?
At the moment, I am working on a project which requires me to build a naive Bayes classifier. Right now, I have a form online asking for people to submit a sentence and the subject of the sentence, in order to build a classifier to identify the subject of a sentence. But before I train the classifier, I intend on processing all entries for the parts-of-speech and the location of the subject. So my training set will be formatted as:
Sentence: Jake moved the chair.   Subject: Jake
POS-Tagged: NNP VBD DD NN         Location: 0

Would this be an effective way to build the classifier, or is there a better method?

Your approach would definitely work. I would recommend training a variety of classifiers and comparing their performance using multiclass roc analysis. Also, think about other useful features in addition to the ones you mentioned (e.g. pos tag). Feature engineering is one of the most important factors in building good predictive models. Another thing to keep in mind is that the classes could be highly imbalanced which might influence your model's performance.


text:
How to find proper parameter settings for a given optimization algorithm?
Is there any methodology to find proper parameter settings for a given meta-heuristic algorithm, e.g. the firefly algorithm or the cuckoo search? Is this an open issue in optimization? Is extensive experimentation, measurements and intuition the only way to figure out which are the best settings?

How to find the best configuration for an algorithm is an open research question in AI. The topic in general is known as `hyper-parameter optimization' and there are a range of possible methods:
One of the most popular is IRace, but other possibilities include:

Spearmint: uses wrappers in Matlab or Python. It uses MongoDb,
and Bayesian optimisation algorithms.
SMAC requires a python wrapper for the algorithm to be optimized
and has a command line interface.
Hyperopt: a Python library which uses Random Search and Tree of
Parzen Estimators.

This paper argues that Spearmint performs the best, compared with SMAC and Hyperopt, but with significantly longer running times in some cases.


text:
Is there an efficient way to implement a random crossover of individuals stored in a matrix?
I am using a GA to optimise an ANN in Matlab. This ANN is pretty basic (input, hidden, output) but the input size is quite large (10,000) and the output size is 2 since I have to classes of images to be classified.
The weights are in the form of 2 matrices (10,000*m) and (m * 2). I am now trying to do the genetic cross over with mutation.
Since the weights are in a matrix, is there an efficient way to implement a random crossover (with mutation) without doing it in a point-wise fashion?

How to combine two random solutions with crossover? I am looking for a suggestion.

Firstly, before we commence I will recommend that you refer to a similar questions on the network https://stackoverflow.com/questions/828486/neural-net-optimize-w-genetic-algorithm
The majority of ML studies focus on gradient algorithms, usually a variation of back-propagation to obtain the weights of the model.
However since genetic algorithms are powerful searching algorithms they can be utilized to tune and optimize the structure and parameters of neural networks.
Genetic algorithms are stochastic search techniques that guide a population of solutions towards an optimum using the principles of evolution and natural genetics.
Genetic algorithms are especially capable of handling problems in which the objective function is discontinuous/non differentiable, nonconvex, multimodal or noisy. (Dharmistha D)
Algorithms, which combine genetic algorithms and error back-propagation, have been shown to exhibit better convergence properties than pure back-propagation.
To facilitate the crossover operation for the exchange of information between two parents we need to define four components within a genetic algorithm.

A way of coding solutions to the problem on chromosomes;
A fitness function which returns a value for each chromosome given to it;
A way of initializing the population of chromosomes;
Operators that may be applied to parents  to alter their genetic composition i.e. mutation.

In these hybrid systems weights in different layers of the network are optimized using a genetic algorithm. Experimental results demonstrate that the genetic algorithms can in some cases optimize the hyper-parameters of an ANN better than a hand tuned model.
Regarding an efficient way to implement a random crossover, here is something to get you started
(Code courtesy of Dan Golding 2013)
%Randomly choose 2 individuals

n = size(new_pop, 1);
l = size(new_pop, 2);

breeders = new_pop(randperm(n,2),:);

%Choose a crossover point

cp = randperm(l, 1);

%Crossover

b1 = [breeders(1, 1:cp), breeders(2, cp+1:end)];
b2 = [breeders(2, 1:cp), breeders(1, cp+1:end)];

For further reference I recommend that you kindly look at the below links
Matlab crossover genetic algorithm
https://stackoverflow.com/questions/17696323/matlab-crossover-genetic-algorithm
Single point ordered crossover in matlab
https://stackoverflow.com/questions/16302382/single-point-ordered-crossover-in-matlab
To optimize a neural network of multiple inputs using a genetic algorithm.
https://www.mathworks.com/matlabcentral/answers/180513-to-optimize-a-neural-network-of-multiple-inputs-using-a-genetic-algorithm?s_tid=gn_loc_drop
How can I use the Genetic Algorithm (GA) to train a Neural Network in Neural Network Toolbox
https://www.mathworks.com/matlabcentral/answers/100323-how-can-i-use-the-genetic-algorithm-ga-to-train-a-neural-network-in-neural-network-toolbox


text:
How do I handle negative rewards in policy gradients with the cross-entropy loss function?
I am using policy gradients in my reinforcement learning algorithm, and occasionally my environment provides a severe penalty (i.e. negative reward) when a wrong move is made. I'm using a neural network with stochastic gradient descent to learn the policy. To do this, my loss is essentially the cross-entropy loss of the action distribution multiplied by the discounted rewards, where most often the rewards are positive.
But how do I handle negative rewards? Since the loss will occasionally go negative, it will think these actions are very good, and will strengthen the weights in the direction of the penalties. Is this correct, and if so, what can I do about it?

Edit:
In thinking about this a little more, SGD doesn't necessarily directly weaken weights, it only strengthens weights in the direction of the gradient and as a side-effect, weights get diminished for other states outside the gradient, correct? So I can simply set reward=0 when the reward is negative, and those states will be ignored in the gradient update. It still seems unproductive to not account for states that are really bad, and it'd be nice to include them somehow. Unless I'm misunderstanding something fundamental here.

Just clarifying, this is for discrete actions?

Did you ever get to the bottom of this?  I'm still struggling with it.  The issue for me seems to be that with a negative reward, the loss can be driven to -inf in the limit.  I think the optimizer biases strongly toward driving the negative reward cases to zero, rather than pushing the positive reward cases to 1.

It depends on your loss function, but you probably need to tweak it.
If you are using an update rule like loss = -log(probabilities) * reward, then your loss is high when you unexpectedly got a large reward—the policy will update to make that action more likely to realize that gain.
Conversely, if you get a negative reward with high probability, this will result in negative loss—however, in minimizing this loss, the optimizer will attempt to make this loss "even more negative" by making the log probability more negative (i.e. by making the probability of that action less likely)—so it kind of does what we want.
However, now improbable large negative losses are punished more than the more than likely ones, when we probably want the opposite. Hence, loss = -log(1-probabilities) * reward might be more appropriate when the reward is negative.

While negative rewards being valid, wouldn't they really screw up your gradient - in the sense that the gradient for positive and negative rewards would be highly asymmetric? Positive rewards will cause a diminishing gradient the closer the action probability goes to 1, whereas negative rewards will cause a strongly increasing gradient the closer the action probability goes to 0.

I think you meant loss = -log(probabilities) * reward (minus before the log)

@user12889 I'm thinking it is symmetric in the sense that a -1 reward has precisely the opposite gradient of a +1 reward. I think the asymmetry you're describing is related to using negative log probabilities as opposed to using something like 1-p. But this asymmetry of the loss function will impact both positive and negative rewards the same.

@Tahlor Yes, but over time, as the model learns, you would expect probabilities for +1 rewards to move closer to 1 and probabilities for -1 rewards to move closer to 0, which then leads to generally highly non-symmetric gradients for +1 and -1 rewards.

Oh right, so couldn't you just invert and shift your loss function for negative rewards?
So -log(1-probabilities) * reward for negative rewards and the regular one for positive rewards to preserve that symmetry?

The cross-entropy loss will always be positive because the probability is in the range $[0, 1]$, so $-ln(p)$ will always be positive.

Right, I think the issue is he's multiplying -ln(p) by a potentially negative number (his reward). This loss function expects only positive rewards.

@Tahlor I think you are right about the reward needing to be positive. I'm having the same problem and came to the same conclusion. However I'm having a hard time finding an authoritative source for this. Do you have one at hand?


text:
When will artificial intelligence equal human intelligence?
What are the current best estimates as to what year artificial intelligence will be able to score 100 points on the Stanford Binet IQ test?

Not going into details of Stanford–Binet test, but just looking at wikipedia page it shows many subtests like knowledge, reasoning, verbal tests etc. Most of the efforts in the artificial intelligence today is directed into research of specific areas like computer vision, natural language processing, machine learning, but also combination of fields like implementation of self driving cars.
Within every field there are still other subfields and problems that are not solved yet. For example, development of human-like natural language processing (NLP) is necessary for intelligent agent to pass any verbal tests, or even non-verbal tests that requires processing of sentences of human language. Famous test that tests intelligence by asking questions in natural language and expects answers in the same form is Turing test. NLP still struggles with many (basic) human skills like listening, speaking, parsing and forming sentences. No one knows when we'll have system that can do these things as good as human. Since this system is crucial, but also far from human-like it's likely cause of delay in developing AI that passes intelligence test. Are these problems AI-hard? Do we need to develop strong AI to solve them?
You can look at speech and listening as interfaces used for expressing and affecting inner processes of human brain. Same goes for other senses like eyesight which is being approximated by computer vision. One could say that we only need to develop convincing mimics of human senses and incorporate them in one big system that will become first human-like AI. That is the minimum requirement. I doubt this will be achieved in this century.
(Other thoughts)
What truly defines intelligence is brain activity. Since it's really complex and one artificial neuron is not equal to one neuron in brain, increase in computation power will not necessarily help achieving human-like AI. Also recognizing such system by mere intelligence test is questionable. For now it's only philosophical discussion but by the time we are able to design such machine I think we'll also have better understanding of human brain. Someone in 2100 might not read this answer on quantum computer with integrated AI OS powered from fusion reactor in his self-flying car, but will probably have many systems that help him in everyday tasks far more than we imagine today.

Nobody knows.
However according to Kurzweil it's late 20s:

2020s:
Early in this decade, humanity will have the requisite hardware to emulate human intelligence within a $1000 personal computer, followed shortly by effective software models of human intelligence toward the middle of the decade: this will be enabled through the continuing exponential growth of brain-scanning technology, which is doubling in bandwidth, temporal and spatial resolution every year, and will be greatly amplified with nanotechnology, allowing us to have a detailed understanding of all the regions of the human brain and to aid in developing human-level machine intelligence by the end of this decade.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Could artificial intelligence cause problems for humanity after figuring out human behavior?
This BBC article suggests that intelligent algorithms, like those that select news stories and advertisements for display, could control our experience of the world and manipulate us.
Will Artificial Intelligence someday become a problem to humanity after learning human behaviors and characteristics?

Will Artificial Intelligence some day become a problem to humanity
  after learning human behaviors and characteristics?

It can be answered in both ways, I think.
Yes, they may become a problem.
With the increasing integration of loads of apps and smart devices in our life, almost everything defining an individual human being is digitalised. For instance, our fingerprints, voice, facial image etc. Apart from these data, we use those apps and devices to track our health (heart rate, calorie intake etc), to plan our schedules, and most importantly to communicate. If some sort of AI engine is integrated into a chat application, for instance, it can learn our typing patterns, conversation style, and hundreds of other unforeseen parameters. Imagine what can be learned about a person if such AI is coded inside every device and every app in your day-to-day use.
We use smart devices and apps to harness their functionalities and features which ease our way of life, and we give them, unintentionally, our identity and sometimes, even our personality. For them, these are the parameters that can be input to some machine-learning algorithm and predict what we will do the next day, or what will happen to us the next day.
This sounds like a major problem, especially when these technologies are indispensable.
No, they may not become a problem.
Humans are really complex creatures and possess the most advanced intelligence technology called the brain. I think the brain can be thought of as a technology in this context. There are several tissues inside the brain that can learn to do certain things themselves. It was proven in a research that some tissues have the capability to perform the functions of other tissues using neuro-rewiring techniques. Imagine that the same tissue that has helped you see until now can be made to help you hear instead. Now imagine mimicking such a technology.
While it is not impossible, for an AI to achieve the brilliance of a human brain is a topic of ongoing research. To train a machine for the purpose, we would have to feed it with gazillion behaviors and characteristics, which it may not be able to handle! After learning some behavior and characteristics, the AI would be said to be smart, but it would still predate us.
So, they may not become a problem at all because of our brain.
The algorithms (like recommender systems) used by Amazon and Facebook influence us or even manipulate us. But, that manipulation is either very obvious (like viewing promoted products) or is in company's best interests (like viewing a certain news piece). It may be even possible that several external parameters are used by these systems to improve your experience. For instance, Google ads show us what we were looking for on an online store when we visit any random website. In most cases, what you see is a result of what you were looking for before. If any attempt to influence does happen, we may learn to avoid it through careful observation or even experience.

Some may look at the algorithmic dissemination of information in the run up to the 2016 US presidential election, for instance, on Facebook, and conclude that this has already become a problem...


text:
How can an artificial general intelligence determine which information is true?
After the explosion of fake news during the US election, and following the question about whether AIs can educate themselves via the internet, it is clear to me that any newly-launched AI will have a serious problem knowing what to believe (that is, rely on as input for making predictions and decisions).
Information provided by its creators could easily be false. Many AIs won't have access to cameras and sensors to verify things by their own observations.
If there was to be some kind of verification system for information (like a "blockchain of truth", for example, or a system of "trusted sources"), how could that function, in practical terms?

When you talk about a General Artificial Intelligence,it means a strong A.I or the ability of a machine[Intelligent Agent] that is capable of performing  "General Intelligent Actions." Therefore,such A.I can have full access to cameras and sensors to, verify things by their own observations in the targeted environment.And so someone to answer this question should not forget the keywords in the question.or else a vague answer(explanation) is to happen.

@quintumnia, while it will be possible for AIs to use cameras and sensors, it is by no means guaranteed that all of them will have access - unless you are proposing some kind of global open access system for all cameras and sensors worldwide?

If an AI were capable of translating natural languages into mathematical logic, it could detect logical contradictions using an automated theorem prover.

Not possible without some big restrictions. What it can do is look at known "good" sites and compare news with site that is potentially "bad". Obvious problem here is defining some sites as absolute truth. For example it can recognize, while reading text, that some politician said something. These sentences can be compared with other sites, and if there is significant difference, that news is candidate for false news.
In practical terms, program would extract sentences "i like cats", "says he likes cats", "cats that John likes" etc. We need part that recognizes something as a quote, part that extracts it and finally parser so we end up with structure stored in some form that contains meaning of sentence (john-like-cats). Also it can keep information of time and context in which it was said, like timestamp of an article, some proper nouns that can indicate place (XY conference, London...). Now, suspicious article can be compared and checked if it matches time, place, some context and contains quote that is similar. Finally it needs to compare how different it is from other quotes. "...hates cats" should be labeled as potential fake news, but "likes dogs", "thinks cats are OK", "sings well" etc. should not. This can be expanded into comparison of whole articles.
There are many features that can be used to define particular article as fake. Interesting feature  for finding fake sites could be bias when it comes to particular (political, economical, ecological...) opinion. But in the end machine can't decide if the article is fake without comparing it to other articles. It is bound to closed system that reflects real world in subjective way.

The problem, as i see it, is that you can't start by defining anything as absolute "truth". Perhaps you could give weightings - for example, something which is known to be extensively fact-checked against current scientific research, for example Encyclopaedia Brittanica, would have a higher "truth quotient" than, say, Wikipedia.

I am talking about particular facts, rather than entire sites or articles. And facts, not opinions (liking or not liking cats) So, for example, "dissolved barium in the ocean causes ice balls on SIberian beaches". How does an AI access the notion of hypothesis, evidence, etc., (or does that need to be programmed in) and how do they know which sources are connected with that truth-seeking process, vs just quoting things from doubtful sources?

Your example is scientific fact which can be "programmed" by giving machine reputable sources like dictionaries, papers etc. You must build knowledge base system. It can evaluate new propositions against old ones. But US election and similar themes which occur in news are highly speculative and are based on certain events that can't be proofed like scientific theories. Also interpretations of such events are mostly subjective. We must not think in terms of predicate logic. Spotting fake facts can be done by checking how big is the difference between test fact and mainstream fact.

For your specific example we can encode sentence like this [dissolved barium]~[ice balls on Siberian beaches] where ~ is our causes operator. If we get some sentences that have same elements (barium and Siberian beach) but are connected differently (for example "does not cause") we will label it as false.

I strongly disagree with all of the aforementioned answers for this reason: -
If we, as humans can be fooled and disceived by what "we" consider a good sources of news, how can an artificially intelligent computer have any chance?
However, the challenge would be that an AI would have to be able to "test" a source of information against a known medium in order to get to the truth. This is a far different dynamic set of circumstances than what has been touted above.
For example, if it was claimed by a woman that a man raped her - which was not reported to the police - it is not enough to compare one person's statements to another in order to determine truth. This is because collusion, influenced or coherced third parties, mistaken perceptions and false beliefs would give false positives.
However, if an AI could establish from her statement that on the day she claimed to have been raped, that the alleged assailant was incapacitated while in her company, until she left his home, because the police report stated that she was upset with the assailant because he was asleep because of drugs during her whole stay. But, this police report comes from an independent source who states, Mr. "x" was asleep that day.
Doing a strict textual check is not going to give the correct answers. analysing her friends and associattes chatter could also confirm a false report as being true.
Therefore, an AI has to have the ability to "test" written reports outside of the criteria of what was spoken.

Input -> Prediction -> Output -> Input -> Prediction -> Output -> Input -> ...
AGI can easily determine which input is true/real. It will use the same method which every organism uses: any input is true and real, unless you misidentified some other stuff as "input".
I would define input as: what crosses the boundary and enters your mind from outside of your mind. The minimum hardwired check is to make sure that signals generated inside a mind are not misidentified as coming from outside (aka "I hear voices"). That's all. This is where the blockchain of truth begins and where it ends.
An Internet article? The input to AI is rather: one of AI's network interfaces received many bytes. Once it's verified they are from the network, and not imaginary, they cannot be unreal or untrue in any meaningful way. By that definition of input, it is in fact the only thing we can be sure is true and real.
Of course AI will likely form hypotheses regarding these bytes that happen to contain ASCII strings like "Trump", "John Smith", "ice balls on Siberian beaches". Then AI will hopefully make predictions based on these hypotheses, maybe interact, maybe get some new input, reject the hypothesis and make a new one, rinse and repeat.
The first hypothesis will be super-naive, but the hundredth, the thousandth?
If you end this process prematurely - maybe for lack of processing power -  you will get something you called a "belief". (Like a belief that some emotional web page might actually reveal a significant truth about our political system.) That belief is a synonym of "tired with trying new hypotheses, will stick to this one". Typical human thing. AI will have less of that, I hope, due to having much much more processing capabilities. AI will stick less to the high-school-level truth that you should assign great credibility to statements written in a form of a newspaper article, it will hopefully form more and more generations of hypotheses, and check them.
In effect AI will depend less on believing various statements generated in the outside world.

While the experiment I link here is a very narrow awareness, it is as such: A robot has just passed a classic self-awareness test for the first time. If the agent can prove something to itself, we can then say it "Knows." Of course the level of awareness you're asking about is very tricky.
In short, it can't know that what it's experiencing is real with absolute certainty because sensory of any kind can be falsified. Do you know what is true/real? You think you do but can you prove it? No. Awareness is subjective.


text:
Are Cellular Neural Networks one type of Neural Networks?
I am researching Cellular Neural Networks and have already read Chua's two articles (1988). In cellular neural networks, a cell is only in relation with its neighbors. So it is easy to use them for real-time image processing. Image processing is performed with only 19 numbers (two 3x3 matrices, called A and B, and one bias value).
I wonder if we can call cellular neural networks neural networks, because there is no learning algorithm.  They are neither supervised nor unsupervised.

From a technical point of view, cellular neural networks unlike many recent deep learning networks perform local computations. It is true that recent deep learning algorithms often use (self-)supervised backpropagation-based learning algorithms. Other algorithms, however, such as Hinton's recently introduced Forward-Forward algorithm also use local computations and is certainly considered a neural network.
Since this question deals with terminology let's also look at the term's origin.
The study of the nervous system tissue (as it can be found in the brain, spinal cord, and peripheral nervous system) known as neurons is what most people consider neuroscience. This multidisciplinary endeavor mainly investigates the origin of behavior through a biological lens. Learning within neural tissue occurs on the micro-, meso- and macro-level and across several temporal orders of magnitude. Molecular DNA-based changes, single neuron plasticity, or network-level changes can all be considered learning.
Among the first to model learning on a specific level of granularity using mathematical modeling were Warren McCulloch and Walter Pitts in the 1940s. Their neurophysiological model of neural activity is considered the origin of neural networks. Since then work on neural networks (such as the work by Hebb, Rosenblatt, Rumelhart & McClelland, and Hinton) is at the intersection between cognitive science, computational neuroscience, and artificial intelligence. Source of the image.

Given the widespread use of the term neural network within difference disciplines, the rich history of artificial neural networks and the actual existence of a wide range of mechanisms within biological tissue, cellular neural networks can certainly be called neural networks.


text:
Are probabilistic models dead ends in AI?
I am a strong believer of Marvin Minsky's idea about Artificial General Intelligence (AGI) and one of his thoughts was that probabilistic models are dead ends in the field of AGI.
I would really like to know the thoughts and ideas of people who believe otherwise.

Where did Minsky say that probabilistic models are dead ends? Can you provide a reference, interested.

I think Minsky deprecated the suggestion that probabilistic models could be surrogates for component models for intelligence that he suggested were grounded in principles and processes that interact (i.e. Society of Mind).  But I don't believe he ever referred to probabilistic models as dead ends.  All intelligence models must employ awareness of likelihoods and exceptions.  Many probabilistic techniques like bayesian inference and markov chains likely will prove essential to the eventual mechanisms of any AGI.
However I would agree with you that Minsky would be unhappy at the prospect of designing AGI systems using techniques like neural nets that 1) are shaped only by supervised training examples, and 2) are unable to explain their reasoning or revise it directly and efficiently.  These approaches (disembodied probabilities) do not model thinking but only the outcome of thinking.  That's akin to not understanding the question but guessing correctly anyway, which is certainly useful, but it's also pretty uninteresting for someone like Minsky who wanted to understand the thought process, not just build a Magic 8 Ball.
How would Minsky respond to recent advances in deep nets like variational autoencoders or adversarial nets or compound hierarchies of nets or end-to-end nets that implement broad skills?  Assuming they can scale up to implement AGI successfully, I believe he'd be rather unhappy with them.  The thought that his Society of Mind could turn out to me no more than a society of ten(?) kinds of deep nets that match patterns probabilistically and interact combinatorically -- I think this would be deeply disappointing to someone who spent a lifetime imagining and refining axioms and theories and proofs of cognition, only to find that the exquisite minds of Einstein and Beethoven might be little more than largely reactive engines highly attuned to variations in rote.

Hey, @Randy. I think the only plausible interpretation of it is that AGI design process isn't just a mathematical optimization problem, which is what most if not all ML models do, they optimize the set of actions and preferences. To some extent, the way we play with numbers without truly understanding them what they mean is, in fact, a dead end. Regardless, thank you for your answer. Best!

When considering effective approaches to AGI, one must extrapolate outwards to the types of modelling (and therefore inputs) that would be necessary to achieve any general utility. One consideration might be the fundamental "building blocks" of our physical world, and understanding the movements of these, can lead to accurate predictions of (all) occurrences. These fundamental elements are called (generally) subatomic particles and are anything but discrete values. In quantum field theory, the more accurate you are able to measure position, the less accurate you may know a quarks momentum (and vice versa). Our world, at the most fundamental layer, is probabilistic when observed. This is all not to say that an understanding of quantum mechanical kinematic descriptions is the only methodology to achieve true AGI, but to say probabilistic models, and therefore uncertainty, is a dead-end seems radically inaccurate.
That said, Dr.Minsky didn't really feel probabilistic models were dead ends. The emerging view in the field, one that Dr.Minsky urged for years, is that connectionism alone couldn't exclusively lead to AGI due to its uniformed structure. If you are unaware, connectionism is the concept of creating models around discrete units of representation (neurones in a neural net for example). You see the problem we have identified isn't that probabilistic models are inaccurate, it's that our current approach doesn't express the biological realism necessary for AGI (although sufficient for specific intelligence).
[I briefly worked with Dr.Minsky at the AI Lab before his passing last year, a hilarious man and brilliant scientist.]

I don't think "quantum mechanical kinematic descriptions" have anything whatsoever to do with achieving true AGI, but I'd love to see you expand on the second paragraph.


text:
How is the distribution of the state related to the distribution of the units in Boltzmann machines?
I'm trying to understand Boltzmann machines. Tutorials explain it with two formulas.
Logistic function for the probability of single units:
$$p(unit=1)=\frac{1}{1+e^{-\sum_{x}wx } }$$
and, when the machine is running, every state of the machine goes to the probability:
$$p(State= state\ with\ energy\ E_i )=\frac{e^{-E_i}}{\sum_i e^{-E_i}}$$
So, the state depends on the units, and then, if I understand correctly, the second formula is a consequence of the first.
So, how can it be the proof that the distribution of $p(state)$ is a consequence of $p(unit)$?

Although you have asked this question more than 3 years ago, I think it would be nice if you could clarify what those symbols in the equations mean. More precisely, what is $w$ and $x$, and what does "$State= state\ with\ energy\ E_i$" mean?


text:
Using ConceptNet5 to find similar systems to solve specific problems?
I installed a locally running instance of the ConceptNet5 knowledgebase in an elasticsearch server. I used this data to implement the so-called "Analogietechnik" (a creativity technique to solve a problem from the perspective of another system) as an algorithm.
The technique works as follows:

Choose a feature of a system
Find systems that have this feature also
Solve the problem from the perspective of these other systems
Apply the found solutions to the issue

As an example, here is the problem of marketing a shopping mall: A Shopping mall has many rooms and floors (1). A museum has also many rooms and floors (2). How are museums marketed? They present many pictures or sculptures (3). We could use our rooms and floors to decorate them with pictures and sculptures (4).
Of course, the idea to implement that as an artificially intelligent algorithm was not far. However, I feel a little bit overwhelmed by the number of methods that exist out there. Neural networks, Bayesian inference, and so on. My current experience doesn't go further than simple machine learning, like k-means clustering.
Do you think it would be very hard to find a solution to this problem?
I'm thinking of a console application, where you can enter a conceptualized problem like "methods for creative writing", for example, and it uses the above method to find possible solutions of the issue. Of course, no solution with extensive depth, more something like basic ideas derived from the knowledge database I have.
Let's take as an example a console application where someone asks "how to write a novel":

It should find out first that the system all is about is in the term "novel". To find a feature of that system, it just searches concepts containing that term: it finds out "Novel is a story" So that's a feature.

Which systems are also stories? A good concept it should find is e.g. "Plot is a story". (Of course only when I am selecting the search results manually)--> How to find the best concepts of a list when not knowing which fits best?

It should then find out that a plot is written using a storyline: "storyline is a plot"

One possible answer of the AI would in this case be: "By writing a storyline"

Do you know some helpful libraries, algorithms or other resources that might help me? I know this is not an easy thing to program, but you might agree that it's highly interesting.

Hierarchical Temporal Memory should help with this. You would have to encode the text data into SDRs. You would then have a coincidence detector. Could you get the right information back out in the way you're trying to? I think so. I'm not fully learned in HTMs yet but check out Nupic (Open source).

Wouldn't I need annotated test Data for this method first? Could you specify more clearly how you would use Nupic?


text:
Why is Sanskrit the best language for AI?
According to NASA scientist Rick Briggs, Sanskrit is the best language for AI. I want to know how Sanskrit is useful. What's the problem with other languages? Are they really using Sanskrit in AI programming or going to do so? What part of an AI program requires such language?

This is a very old question, but you should have provided a link to the article or source where the person claims this.

Rick Briggs refers to the difficulty an artificial intelligence would have in detecting the true meaning of words spoken or written in one of our natural languages. Take for example an artificial intelligence attempting to determine the meaning of a sarcastic sentence.
Naturally spoken, the sentence "That's just what I needed today!" can be the expression of very different feelings. In one instance, a happy individual finding an item that had been lost for some time could be excited or cheered up from the event, and exclaim that this moment of triumph was exactly what their day needed to continue to be happy. On the other hand, a disgruntled office employee having a rough day could accidentally worsen his situation by spilling hot coffee on himself, and sarcastically exclaim that this further annoyance was exactly what he needed today. This sentence should in this situation be interpreted as the man expressing that spilling coffee on himself made his bad day worse.
This is one small example explaining the reason linguistic analysis is difficult for artificial intelligence. When this example is spoken, small tonal fluctuations and indicators are extremely difficult for an AI with a microphone to detect accurately; and if the sentence was simply read, without context how would one example be discernible from the other?
Rick Briggs suggests that Sanskrit, an ancient form of communication, is a naturally spoken language with mechanics and grammatical rules that would allow an artificial intelligence to more accurately interpret sentences during linguistic analysis. More accurate linguistic analysis would result in an artificial intelligence being able to respond more accurately. You can read more about Rick Brigg's thoughts on the language here.

Adding some to what Christian said. Facts are taken from the book, Artificial Intelligence: A Modern Approach.
Burrhus Frederic Skinner, a psychologist and behaviourist, published his book Verbal Behaviour in 1957. His work contains a detailed account of the behaviourist approach to language learning.
Noam Chomsky later wrote a review on the book, which for some reason became more famous than the book itself. Chomsky has his own theory of Syntactic Structures for this. He even mentioned that the behaviourist theory did not address the notion of creativity in language as it did not explain how a child could understand and make up sentences that he/she has never heard before. His theory based on syntactic models are dated back to Indian linguist Panini (350 B.C.) who was an ancient Sanskrit philologist, grammarian, and a revered scholar.


text:
Why teaching only search algorithms in a short introductory AI course?
I understood that the concept of search is important in AI. There's a question on this website regarding this topic, but one could also intuitively understand why. I've had an introductory course on AI, which lasted half of a semester, so of course there wasn't time enough to cover all topics of AI, but I was expecting to learn some AI theory (I've heard about "agents"), but what I actually learned was basically a few search algorithms, like:

BFS
Uniform-cost search
DFS
Iterative-deepening search
Bidirectional search

these search algorithms are usually categorized as "blind" (or "uninformed"), because they do not consider any information regarding the remaining path to the goal.
Or algorithms like:

Heuristic search
Best-first search
A
A*
IDA*

which usually fall under the category of "informed" search algorithms, because they use some information (i.e. "heuristics" or "estimates") about the remaining path to the goal.
Then we also learned "advanced" search algorithms (specifically applied to the TSP problem). These algorithms are either constructive (e.g., nearest neighbor), local search (e.g., 2-opt) algorithms or meta-heuristic ones (e.g., ant colony system or simulated annealing).
We also studied briefly a min-max algorithm applied to games and an "improved" version of the min-max, i.e. the alpha-beta pruning.
After this course, I had the feeling that AI is just about searching, either "stupidly" or "more intelligently".
My questions are:

Why would one professor only teach search algorithms in AI course? What are the advantages/disadvantages? The next question is very related to this.
What's more than "searching" in AI that could be taught in an introductory course? This question may lead to subjective answers, but I'm actually asking in the context of a person trying to understand what AI really is and what topics does it really cover. Apparently and unfortunately, after reading around, it seems that this would still be subjective.
Are there AI theories that could be taught in this kind of course?

You should probably change your proposition that the listed search algorithms are called "blind" - because it's wrong. "blind" is usually used as a synonym for being "uninformed". However, A*, IDA*, and Heuristic search are by definition "informed" search algorithms (because they base upon heuristics", so they are not blind.
Further, are you sure the "A algorithm" exists? I only know A* and I was not able to find any mentioning of this algortihm in the web. If it actually exists, a link would be nice (maybe in the comments).

@Prof.Chaos The A algorithm is A* when you don't know that the heuristic is optimal. Indeed the * in A* should evoke something in our heads. Regarding the "blind" argument, it's probably unfair to consider A* as blind as BFS or DFS, so I agree with you.

There is lots of misconceptions about AI, specifically the idea that it is about making computers "think" like humans, simulating brain, the sci-fi robots taking over the world, all the philosophical discussions around brain as machine etc. The practice/reality of AI is about "using computing to solve problems" which basically means you take any problem, represent it as a computing problem and then design the algorithm to solve the computing problem which lead to solving the original problem. These search algorithms are general purpose algorithms for general purpose computing problems i.e any real world problem can be represented by these general purpose computing problem and then these algorithms can be used to solve them.
Remember, its about problem solving and its about general purpose computing problems that can represent any real world problem.

What it comes down to is that most AI problems can be characterized as search problems. Let's just go through some examples:

Object recognition & scene building (e.g. the process of taking
audio-visual input of your surroundings and understanding it in a 3D
and contextual sense) can be treated as searching for known objects
in the input.
Mathematical problem solving can be treated as searching for a solution.
Playing a video game can be treated as searching for the correct response to a given gamestate.

Even rudimentary chatbots can be characterized as finding the 'correct' response to a given input phrase to emulate human language!
Because of this generalization of search, search algorithms were among some of the first algorithms considered 'AI', and often form the basis of many AI teaching courses. On top of this search algorithms are intuitive and non-mathematical, which makes the somewhat terrifying field of AI accessible. This might sound like hyperbole, but I guarantee that if your lecturer had opened with Manifold Learning Techniques half of your class would have bolted for the door by the time they mentioned 'eigenvalue of the covariance matrix'.
Now search algorithms aren't the only way to address these problems. I recommend every AI practitioner is familiar with the notion of Data Science and Machine Learning Algorithms. ML is often related to search algorithms but the techniques they use can vary heavily from iterative building of a classifier/regression (e.g. C4.5 builds a decision tree), meta-heuristics as you noted, and classifiers/regression that are statically generated from analysis of training data (e.g. Naive Bayesian is literally a classifier built on Bayesian analysis of the given data assuming that input fields are independent - this is the 'naivety' from which it gets its name). Often ML algorithms are developed in AI research groups and can sometimes be designed for specific problems instead of being general form algorithms. In contrast to the general field of AI, which is often centered on Intelligence problems and is therefore (in my view) vulnerable to too much blue sky thinking, ML is applied to all sorts of real life problems and is often very practical in its design and performance driven in its evaluation.

Why would one professor only teach searching algorithms in AI course? What are the advantages/disadvantages?
My answer to this question is that there are lots of problems where the solution can be found using searching. Take an example of Tic Tac Toe. If you are designing an intelligent computer player for this, then what you will do is that you will form a search space and then you will search for most optimal move which can be made to conclude the game. In these, scenarios you must be aware of optimal search strategies. Let's take another example, suppose if you are driving and want to got to an unknown person's house. It's far from your place and you decide to use GPS. Your GPS will use search algorithms to find the most optimal route that you can take to reach to the destination (of course there will be lots of factors to consider like traffic, etc. but this is the basic idea).
Disadvantages are only in terms of processing and storage. For slow algorithms you will be wasting lots of CPU time and storage as well but for good and efficient algorithms, you can preserve lots of space and also execute your task very fast. Of course, just learning about searching isn't AI. There's lot more to it.
What's more than "searching" in AI that could be taught in an introductory course?
There is lots of things in AI other than searching. For example, learning techniques (supervised, unsupervised, reinforced), planning when one wants to design a system that will do certain actions independently and intelligently, representation of knowledge (known and unknown) and inference in agents which includes propositional logic and first-order logic, etc.
Are there theories behind AI that could be taught in this kind of course?
Some topics could be taught like about different types of agents (simple reflex, model based, goal based, utility based and learning agent), different types of environments in which agents work, evaluation of agents. There could be some additional introductory topics like natural language processing, expert systems, etc.


text:
Are we supposed to "decide" ethics for intelligent systems?
Ethics is defined to be the set of moral principles that governs a person's or group's behavior.
Innately, shouldn't any system devise ethics for itself? Most of our articulations, in either direct or indirect manner, talk about intelligent systems in context of human presence. So, why, in first place, are we studying AGI ethics?
Given Roseau's reasoning for a society

Human beings can improve only when they leave the state of nature and enter a civil society.

Doesn't that also automatically apply to any intelligent system? If it does, then can't we imply that any inorganic intelligent system should, perhaps, come as an offset of a network of intelligent computers (a society per se)?
Who are we to constitute ethics and rules, morale for another society which, perhaps, could be much more intelligent than us?

Nonhuman ethics can be totally arbitrary
Regarding your citation, perhaps it's useful to consider why should anything that philosophers have said about human ethics apply for arbitrary nonhuman systems? We all share a common biological background, our behavior, emotions and especially positive/negative feelings arise from the particular way how our brains are built. Man is by nature a social animal, and much of our behavior and feelings of empathy, compassion, fairness, greed, social status, etc are formed and expressed in society. Our minds are adjusted for this, as are the minds of other mammals. A human that forms an ethical system when joining a civil society does so in the basis of all this shared context, and within the limitations of it.
However, all this is orthogonal to intelligence. All the variety of mankind encompasses a tiny island in the whole space of possible intelligences and their goals. Any system of ethics is feasible, a vast majority of them having no relationship whatsoever to what we could consider ethical. A human can be expected to join a society and learn ethics because our brain is hardwired to respond to social stimuli - an AI would learn ethics from a society only if its previous ethics already happen to value that highly, and that isn't likely to happen without careful design.
There is a thought experiment that is commonly used as an example - the "paperclip maximizer", an artificial agent with an ethical system that defines 'good' proportionally to the number of paperclips that exist in the universe and only that - and that is a perfectly reasonable example, because a completely random ethical system is overwhelmingly likely to reduce to something stupid like that. The space of "reasonable" (to us) ethical systems is very tiny compared to all ethical systems possible, and expecting that a randomly selected ethical system will just happen to land there is like relying on winning a high-stakes lottery.
We want their ethics to contain certain things
So the main reason why we are studying AGI ethics is because we (or some of us) really, really, want the AGI ethical system to include certain things. For example, I'd like that ethical system to include my survival. I'd prefer that ethical system to be consistent with my happiness and wellbeing - and the very existence of a very powerful entity that simply happens to not care about me at all is not compatible with my survival and wellbeing, so I'd really prefer for it to care about me, or alternatively never become very powerful.
One could certainly argue that we have no right to constitute ethics, rules and morale for another society which could be much more intelligent and better than us. That another society might be "better" in some manner - but better for whom ? If it's better for "them" but worse for me (and humanity), then I have strong motivation to simply claim that right and try my hardest to ensure that this hypothetical future society includes me, my desires and my ethics.


text:
What's the name of the value that you add or subtract from a minimax tree node?
I am coding a tic-tac-toe program that demonstrates reinforcement learning. The program uses minimax trees to decide its moves. Whenever it wins, all the nodes on the tree that were involved in the game have their value increased. Whenever it loses, all the nodes on the tree that were involved in the game have their value decreased, etc.
What is the name of the value that each node is decreased by?

welcome to ai.stackexchange. Your question is a little bit too vague. Can you give us an example? For instance, I do not understand what you mean by increasing or decreasing all values which where involved in the tree.

I think you mean the leave nodes only which are changed. The other nodes in the tree are calculated during calculating the best move with this tree. The values at the leaves are called utility values in Russel and Norvig's "Artificial intelligence: a modern approach". Some times it is called heuristic value; see https://en.wikipedia.org/wiki/Minimax.

In reinforcement learning, you can keep the name of value as it comes from the value function which estimates how good it is to be in this node w.r.t. the objective. Depending on the problem it can be a cost, utility, reward...


text:
How do you generate the transition probabilities of a non-trivial MDP?
I understand an MDP (Markov Decision Process) model is a tuple of $\{S, A, P, R \}$ where:

$S$ is a discrete set of states
$A$ is a discrete set of actions
$P$ is the transition matrix ie. $P(s' \mid s, a) \rightarrow [0,1]$
$R$ is the reward function id. $R(s, a, s') \rightarrow \mathbb{R}$

For a non-trivial MDP, say $1000$ states and $10$ actions, the transition matrix has theoretically $S \times A \times S = 10,000,000$ entries (though many entries will be $0$).
I understand that one way of generating the $P$ matrix is to estimate it via Monte Carlo sampling, by simulating the environment. However, with non-trivial state space and simulation costs, this could be prohibitively expensive.
In practice, when a non-trivial MDP is being formulated, what are the different ways an accurate $P$ matrix can be produced?

In non-trivial cases, the transition matrix is (generally) not maintained in the traditional tabular form. If the representation used factored notation (Factored MDP) then Dynamic Bayesian networks can be used.
Another approach would be to abstract state spaces so that you have reduced number of states in the representation of P. These can be roughly classified as model minimization techniques.
Hierarchical representations can be adopted if some tasks can be split. E.g., the classical "taxi problem" has an explicit Get-passenger and Put-passenger task, that form the hierarchy. Now each of the smaller tasks can possibly have a well formed transition function.
It may also be prudent to use model free learning methods in such cases.


text:
What makes neural networks so good at predictions?
I am new to neural-network and I am trying to understand mathematically what makes neural networks so good at classification problems.
By taking the example of a small neural network (for example, one with 2 inputs, 2 nodes in a hidden layer and 2 nodes for the output), all you have is a complex function at the output which is mostly sigmoid over a linear combination of the sigmoid.
So, how does that make them good at prediction? Does the final function lead to some sort of curve fitting?

Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily.
The mathematical reason for the neural networks prowess at classifying is the universal approximation theorem. Which states that a neural network can approximate any continuous real-valued function on a compact subset. The quality of the approximation depends on the number of neurons. It has also been shown that adding the neurons in additional layers instead of adding them to existing layers improves the quality of the approximation faster.
Add to that the not well-understood effectiveness of the backpropagation algorithm and you have a setup then can actually learn the function that the UAT promises or something close.

With Neural Networks you simply classify datas. If you classify correctly, so you can do future classifications.
How It Works?
Simple neural networks like Perceptron can draw one decision boundary in order to classify datas.
For example suppose you want to solve simple AND problem with simple Neural Network. You have 4 sample data containing x1 and x2 and weight vector containing w1 and w2. Suppose initial weight vector is [0 0]. If you made calculation which depend on NN algoritm. At the end, you should have a weight vector [1 1] or something like this.

Please focus on the graphic.
It says: I can classify input values into two classes (0 and 1). Ok. Then how can I do this? It is too simple. First sum input values (x1 and x2).

0+0=0
0+1=1
1+0=1
1+1=2

It says:

if sum<1.5 then its class is 0
if sum>1.5 then its class is 1

Neural networks excel at a variety of tasks, but to get an understanding of exactly why, it may be easier to take a particular task like classification and dive deeper.
In simple terms, machine learning techniques learn a function to predict which class a particular input belongs to, depending on past examples. What sets neural nets apart is their ability to construct these functions that can explain even complex patterns in the data. The heart of a neural network is an activation function like Relu, which allows it to draw some basic classification boundaries like:
By composing hundreds of such Relus together, neural networks can create arbitrarily complex classification boundaries, for example:
In this article, I try to explain the intuition behind what makes neural networks work: https://medium.com/machine-intelligence-report/how-do-neural-networks-work-57d1ab5337ce

In Neural Networks we consider everything in high dimension and try to find a hyperplane that classify them by small changes...
Probably it is hard to prove that it works but intuition says if it can be classified you can do it by add a relaxed plane and let it move amongst data to find a local optimum...


text:
What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent?
Printing action_space for Pong-v0 gives Discrete(6) as output, i.e. $0, 1, 2, 3, 4, 5$ are actions defined in the environment as per the documentation. However, the game needs only 2 controls. Why do we have this discrepancy? Further, is that necessary to identify which number from 0 to 5 corresponds to which action in a gym environment?

Programming questions (including how to use a certain library or API) are off-topic here. This type of question is more appropriate for Stack Overflow, even though the question is still related to Artificial Intelligence/Reinforcement Learning. Here, we focus on the theoretical, social, and philosophical aspects of AI. Please, see https://ai.stackexchange.com/help/on-topic for more details.

You can try the actions yourselves, but if you want another reference, check out the documentation for ALE at GitHub.
In particular, 0 means no action, 1 means fire, which is why they don't have an effect on the racket.
Here's a better way:
env.unwrapped.get_action_meanings()

You can try to figure out what exactly does an action do using such script:
action = 0  # modify this!
o = env.reset()
for i in xrange(5): # repeat one action for five times
    o = env.step(action)[0]
IPython.display.display(
    Image.fromarray(
        o[:,140:142]  # extract your bat
    ).resize((300, 300))  # bigger image, easy for visualization
)

action 0 and 1 seems useless, as nothing happens to the racket.
action 2 & 4 makes the racket go up, and action 3 & 5 makes the racket go down.
The interesting part is, when I run the script above for the same action(from 2 to 5) two times, I have different results. Sometimes the racket reaches the top(bottom) border, and sometimes it doesn't. I think there might be some randomness on the speed of the racket, so it might be hard to measure which type of UP(2 or 4) is faster.

There seems to be no difference between 2 & 4 and 3 & 5. The inconsistency mentioned by Icyblade is due to the mechanics of the Pong environment.
"Each action is repeatedly performed for a duration of k frames, where k is uniformly sampled from {2,3,4}"
So the action is just repeated a different number of times due to randomness


text:
Is there a proof that states that an AI can become smarter than its creator?
Is there a proof that states the possibility or impossibility of an AI system to acquire more sophisticated capabilities (in terms of generic cleverness) than its own creator?

there's no such theory. But even if it would be there, it has to be a conspiracy theory.

Regarding Artificial General Intelligence, which does not currently exist and is still highly theoretical, this cannot be determined at this time.
What I would say is that "strong narrow AI" has already proven the ability to become "smarter" than it's creators in specific tasks.  (See Alphago, etc.)
Under the idea that some form of AGI might come out of an algorithm comprised of an ever expanding set of strong narrow AIs, it would logically follow that such an automata could become smarter than it's creators in any given task.

If you look at the work of Howard Gardner and his theory of multiple intelligences, you will see that the term of "intelligence" respectively "generic cleverness" is much more diverse and not entirely clarified. Without a entire notion of it, a proof is forlorn.

The loaded term in your question is "generic cleverness." There's no such thing. What is "smart" is only smart relative to a criterion. Provide a complete criterion and we can talk rationally about "levels of sophistication." Until then, there is no measure against which "involuntarily acquired" capabilities can be regarded as more or less "smart."

authors: Денис Антюхов


text:
How to perform back-propagation in Decoupled Neural Interfaces?
I am attempting to create a fully decoupled feed-forward neural network by using decoupled neural interfaces (DNIs) as explained in the paper Decoupled Neural Interfaces using Synthetic Gradients (2017) by Max Jaderberg et al. As in the paper, the DNI is able to produce a synthetic error gradient that reflects the error with respect the output:
$$
\frac{\partial L}{\partial h_{i}}
$$
I can then use this to update the current layer's parameters by multiplying by the parameters to get the loss with respect to the parameters:
$$
\frac{\partial L}{\partial \theta}=\frac{\partial L}{\partial h_{i}} * \frac{\partial h_{i}}{\partial \theta}
$$
In the paper, the layer's model is then updated based on the next layer sending the true error backwards.
My question is, given that I am able to calculate the error with respect to the current output, how do I use this to calculate the loss with respect to the previous layer's output?

I kid you not when I tell you I just started with decoupled networks this morning. But the issue is terribly worded by most computer scientists. The way a decoupled neural interface works is not by sending the error backwards, but by storing it in a form of learned error inside a separate network. So not only do we have our nodes and weights, but also the Synths. The Synths modify the weight tables using past error. So instead of passing error back like the article suggests, it actually is passing it forward in a kind of "hey don't do this." What's really fun is that to you never have to mix up your thinking to reverse a network for backprop if you're using a decoupled network. The amazing part is that it is actually easier to code this decoupled network, than it is to write a "simple" ANN, even better still is that it functions a lot like an LSTM network without all the programming mumbo-jumbo. You can see for yourself two different forms of the same network from here.
The code from the demo in the repository is from https://towardsdatascience.com/only-numpy-implementing-and-comparing-combination-of-google-brains-decoupled-neural-interfaces-6712e758c1af it was a fundamental resource for me, and it may be for you. It also helps you with loss, or cost.

Sorry about the dirty code by the way.


text:
Can games be solved without an evaluation function?
Fundamentally, a game-playing AI must solve the problem of choosing the best action from a set of possible actions.
Most existing game AI's, such as AlphaGo, do this by using an evaluation function, which maps game states to real numbers. The real number typically can be interpreted as a monotonic function of a winning probability estimate. The best action is the one whose resultant state yields the highest evaluation.
Clearly, this approach can work well. But it violates one of Vladimir Vapnik's imperatives (in his book "Estimation of Dependences Based on Empirical Data"): "When solving a problem of interest, do not solve a more general problem as an intermediate step." In fact, he specifically states as an illustration of this imperative,

Do not estimate predictive values if your goal is to act well. (A good strategy of action does not necessarily rely on good predictive ability.)

Indeed, human chess and go experts appear to heed his advice, as they are able to act well without using evaluation functions.
My question is this: has there has been any recent research aiming to solve games by learning to compare decisions directly, without an intermediate evaluation function?
To use AlphaGo as an example, this might mean training a neural network to take two (similar) board states as input and output a choice of which one is better (a classification problem), as opposed to a neural network that takes one board state as input and outputs a winning probability (a regression problem).

Human chess and go experts clearly use evaluation functions. They do come up with moves that look sensible without evaluating the board position, but to validate these candidate moves they evaluate board positions that occur at the end of the variations they calculate. Pretty similar to AlphaGo.
Inputting two board states and outputting a preference is a (much) more complex task than mapping one board state into the real numbers. And it gives you less information. So its a lose-lose choice. (I did try something very similar and it didn't work at all. The reason is that you didn't just double the input size, rather you made the input space quadratically bigger.)
If you compare two board states that differ just by one move, then your input space doesn't quite explode as much, but you have to do a ton of comparisons to make a decision. The logical choice would be to output a preference distribution over all possible moves - but that's exactly what AlphaGo does in its policy network. There is also an earlier paper which trained a network to predict expert moves, which comes down to the same thing. And yes, both these networks play quite strongly without any search or board evaluation. But nowhere near AlphaGo-level.

Humans sometimes use evaluation functions. But not always. To use a trivial example in go, consider a suicide move of placing a stone directly in an opponent's single-space eye. This move is strictly dominated by the option of passing the turn. I can thus rule out the move even without looking at the rest of the board. In other words, if f() is an implicit evaluation function, I'm able to reason that f(A)>f(B), without actually calculating f(A) or f(B).

To provide another illustration: imagine two board positions, A and B, where A is clearly-but-slightly superior to B. If I ask 100 experts which is superior, all 100 will say A. But if I ask half of them to estimate A's winning probability, and the other half to estimate B's winning probability, and they create their estimates independently, I am likely to get overlapping distributions. Forcing them to independently map board states to numbers effectively handicaps their ability to compare positions.

As long as every expert maps A to a higher probability than B, there is no handicap at all. And of course the ability to consciously compare two positions might come from an unconscious independent mapping of both.

Of course if you ask one expert to compare A and B, he will rate A>B. But if you ask that expert to rate A, and tell him, "by the way, I have in this sealed envelope the true win probability of B, and after you rate A I'm going to open up the envelope and choose between A and B by comparing the numbers", you've made the expert's life considerably more difficult.

Yes, because the expert isn't an expert in assigning accurate probabilities. He is an expert at making decisions. You ask him to do something he didn't train to do. Assigning accurate probabilities isn't relevant to playing well, assigning consistent probabilities is.

Yes, that's exactly my point. For humans, comparing options is an easier computational task than assigning accurate probabilities to the options. Accurate probability estimates are sufficient but not necessary for optimal decision making. Demanding that the expert perform his comparisons on the real number line handicaps him. So demanding the same of our AI systems might be a handicap as well.

To be clear, it is a handicap when we force the mappings of A and B to the real number line to be made independently. That's why in my illustration I have half the experts estimate A and the other half estimate B without getting to talk to each other. Perhaps it might make sense to think of a "comparison" as a joint-estimate-vector with an uncertainty covariance matrix U; the handicap can be interpreted as restrictions on U.

I don't yet see why comparing options would be an easier computational task than assigning accurate probabilities for humans. Humans are better at the former, because the train for it, not because it is easier. In fact I'm pretty sure that chess grandmasters are quite good at predicting engine scores, because they work with engines everyday. And that's pretty similar to probabilities.

Let us continue this discussion in chat.

@dshin Very interesting! Concerning "computational savings versus applying independent evaluations" from the discussion: There are heuristics in game engines giving preference to moves which were shown to be good in other parts of the search tree. This is like to "go through the kingside defensive lines again [... but ...] a lot faster" (again from the discussion). No idea if Alpha Go uses / could use such heuristics. I guess, my comment is only loosely related to your question, but aims at the same goal.


text:
When do mutations in NEAT occur?
I read through the Evolving Neural Networks through Augmenting Topologies (NEAT) paper. I understand the algorithm now, but one thing is still unclear to me.
When does the mutation occur and how does it take place? How is it chosen whether to add a node or to add a connection mutation? Furthermore, how is it chosen where the mutation is taking place in the network (between which connections)?

When does the mutation occur and how does it take place?

Finding a solution in NEAT algorithm is based on evolution strategy. It means that you have Neural Networks which are yours individuals, so mutations and crossing occurs in loop after phase of "fitnessing" (calculation fitness for every individual and removing bad ones).

How is it chosen whether to add a node or to add a connection mutation? Furthermore, how is it chosen where the mutation is taking place in the network (between which connections)?

Randomly: just draw. You can read more about evolutionary algorithms here.
If it could somehow help, I include link to my repository with the implementation of NEAT.

i read through your code and I have some question considering the Crossover. Why are you at first adding the connection to child as described in the Paper, but after that you add all connections from both parents. Why? And if your networks are disjoint and later on they have the same mutation you just ignore it. Because you don't have blank objects or something in your list of connections.


text:
How do we know that now artificial intelligence will really take off?
A lot of people are claiming that we are at an inflection point, and machine learning/artificial intelligence will take off. This is in spite of the fact that for a long machine learning has stagnated.
What are the signals that indicate that machine learning is going to take off?
In general, how do you know that we are at an inflection point for a certain technology?

This has happened in the past where people were really excited and saying things like we will have AI in decade or so. This is happening again. Not sure why people don't learn from history of AI. In both the cases what's happening is this - You develop a technique to solve a particular problem, you apply that technique, the technique seems to be general enough, people start to apply that same technique to various problems, people get excited that this is the silver bullet they were looking for, the technique starts to show its limitations and doesn't work for many problems, hype gets shattered, start over.

Comments are not for extended discussion; this conversation has been moved to chat.

What are the signals that indicate that machine learning is going to take off?

We simply don't know until the consequences of the inflection point determine a remarkable difference between the before and after. In general terms every considerable reaction must be attributed to a particular cause.
One of the biggest limit that bounds artificial intelligence, and apparently makes it stagnating, is the greed of computational power involved in this field; and since the hardware technology improve much slower than the software does, AI remains confined in labs and data centers.

Part of the reason people are so excited about recent Machine Learning milestones is that AlphaGo demonstrated a reproducible method of managing mathematical and computational intractability.
Go is interesting because it's impossible to solve.  It cannot be brute-forced no matter how fast processors get.  Go is so complex humans had failed to produce AI that could win against a skilled human player.  The fact that a computer could teach itself to do something humans couldn't teach it, and something with a complexity analogous to nature to boot, is pretty extraordinary.
Combinatorial games in particular are useful because, unlike nature where it may be impossible to track or even be aware of every variable, intractability can be generated out of a simple set of elements and rules, and outcomes can be definitively evaluated.
As proof-of-concepts for methods go, AlphaGo seems like a pretty strong one.  It allows us to definitively say "Machine Learning works", puts a lot of emphasis on the field, and raises confidence on extending the method to real world problems.
Beyond that, it suggests a feedback loop in which programs can improve at at improving, unrestricted by human limitations.  Increase in processing power is bounded by physical limitations, but algorithms are not.

I am not sure its the AlphaGo. People have been claiming an inflection point in AI for a while now. Case in point: Steve Jurvetson on the famed DFJ ventures gave this talk about how big data and machine learning is going to take over the world. https://youtu.be/czLI3oLDe8M?t=658
perhaps when we think of inflection point, its not really a point in time, rather a period of years or decades when the pace of development accelerates year over year..

Perhaps there is no one signal, but a collection of signals.. Perhaps we should be looking for signals that are accelerating.. AlphaGo is definitely a very good signal...is it a datapoint thats showing an acceleration over past data points?