text:
Can a system of If-Then rules be regarded as AI?
Expert Systems (ES) are regarded as AI. However, ES can be as simple as a system of If-Then rules. But AI seems like a big name for a set of (could be rather simple) If-Then rules. Is this indeed the case that certain systems of If-Then rules are regarded as AI?

Hello. Welcome to AI SE. This question is unclear to me. You seem to be aware of the fact that ES are regarded as AI and that ES may be "if-then" rules, but then you ask if if-then rules are regarded as AI. In any case, your question seems to be a duplicate of this one.

Thank you! But the answer to the referenced question is unclear. This is clear from the comment to the question: confusion between AI and Machine Learning. That's why I decide to ask almost Yes/No question.

I am not fully sure that answer confuses AI with ML (although I wrote a comment there stating that). The author of that answer writes "Machine learning, a subset of AI, extracts useful patterns from data". Then provides the example of decision trees as an example of a ML model that uses if-then statements in a statistical way. Maybe it doesn't clarify well why ES are considered AI.

However, I think that answer at least attempts to clarify this too. From what I understand, the answer (of that author) is "What makes a program an AI is whether or not you use if-then statements to solve problems that would require human expertise" (that's why they are called expert systems). But, yeah, I agree, this is still a bit vague (as most programs that we write would require in a way "human expertise") and doesn't answer your question if ES are really considered AI (and by whom or according to which definition of AI).

Actually, the OP puts it in a different way "the automated reasoning that went into selecting those particular if-then statements", which is a bit vague.

AI is not an objective definition. It's extremely broad, and perpetually up for debate. However, I would say most would agree that yes, any system of If-Then rules can be considered an AI.

If-Then rules cannot exhaustively cover all possibilities that can be encountered in real world situations. One of the hallmarks of A.I is the ability to generalize across problem domains, not just individual problems like Vision or Natural language or Automated driving. Specifically, a true A.I system would not need a programmer to specify each and every problem that it might have to solve or even the specific conditions within each problem that it might encounter.
Consider this, the number and type of problems that potentially exist is virtually infinite. So, to be able to exhaustively catalog all the potential problems is virtually impossible. Furthermore, to be able to then write If-Then rules for all the problems and their individual conditions is a step beyond the impossible. Automated driving is a good example as a problem where there can be an infinite number of If-Then rules to write down if one were to go that route but you can always create a new situation that the programmer didn't think of like what should the system do if a meteor crashes in the middle of the road.
What A.I promises is automation of reasoning ability in domains and problem sets that are broad. In some very narrow fields where the possibilities are countable and finite and it can be taken for granted that the system exists in a pre-defined set of states, one could use If-Then rules to surpass human ability. For example, in Chess simple rule based systems could surpass human abilities a long time ago. But even in this well defined game, a rule based expert system worked well until a new system (AlphaZero) came along which was not designed with If-Then rules so was not limited by the imagination of the programmers either tactically or strategically and blew all the previous ES systems out of the water. But real world problems are seldom well defined and the set of potential states of the system can be large and varying in size.
Note that there are many well defined real world problems that ES can tackle and A.I wouldn't provide any additional advantage but the number of problems that A.I potentially could handle where ES would be sub-optimal is much larger. Also check out LCS that can synthesize rules through learning and exploration.


text:
What is the time complexity of DDPG algorithm?
Suppose we have a DDPG algorithm.
The actor has N input nodes, two hidden layers with J nodes, and S output nodes.
The critic has N+S input nodes, two hidden layers with C nodes, and one output node.
How does the time complexity of this algorithm could be calculated??

Hello. Welcome to Artificial Intelligence Stack Exchange. My question to you is: what have you tried so far to answer your own question? Do you know what the time complexity is?

Thanks. yes, I know what it is, and I've found some ways to calculate the neural network's time complexity. But, I wonder is there a difference between neural network complexity and Rl algorithms? also in a paper related to my work there was a Q-learning time complexity analysis. But I couldn't find anything for actor-critic methods like DDPG.

I would recommend that you first provide the pseudocode of the algorithm (and the link to the paper that contains the algorithm/pseudocode) that you referring to, so that to make sure we're talking about the same thing.

I thought my question was clear enough. It's the DDPG paper link. and I've found this very helpful for neural networks [link] (https://ai.stackexchange.com/questions/5728/what-is-the-time-complexity-for-training-a-neural-network-using-back-propagation)

So, which algorithm in that paper are you talking about? Algorithm 1? As I said above, please, edit your post to directly include these details there.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Rank of gradient-of-loss with respect to layer weights in an MLP
The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3:

The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch elements $\nabla_{w_l}L = \sum_{i=1}^B \delta_{l+1}^i {u_l^i}^T$, where $\delta_{l+1}^i$ is the gradient of the loss for batch element $i$ with respect to the preactivations at layer $l + 1$, and ${u_l^i}^T$ are the inputs to layer $l$ for batch element i.

Suppose that we have an MLP with $k$ hidden layers (every hidden layer is followed by an activation function). Then the weight matrices will be $W_1, W_2, \dots, W_k$ (plus the biases, but they are irrelevant for now), and their sizes will be $(D_1, D), (D_2, D_1), \dots (D_k, D_{k-1})$ correspondingly, where $D$ is the number of input features.
Therefore, hidden layer $l$ has a weight matrix $W_l$ of size $(D_l, D_{l-1})$. Its gradient wrt the loss (for 1 batch element), $\frac{\partial L}{\partial W_l}$, will also be a matrix of size $(D_l, D_{l-1})$.
So if I understand correctly, the authors of the paper are claiming that $\frac{\partial L}{\partial W_l}$ is a rank-1 matrix? That is, every row (or column) can be expressed as a linear combination of 1 only row (or column)? If yes, why? How?

I didn't find the reference in the Goodfellow deep learning book. But here is how I derived it. Let all vectors be column vectors.
The basic claim is that $\nabla_W F(u^\top W) = \nabla F(v^\top) u^\top$, where $v^\top := u^\top W$.
Consider the simplest function $F(v^\top) := v^\top x$. We can compute $\nabla_W (u^\top W x)$ either by looking up matrix differentiation rules, or by hand:
$$\partial_{W_{i,j}} u^\top W x = x_i u_j.$$
This shows that $\nabla_W (u^\top W x) = x u^\top$.
Finally convince yourself that general $F$ behaves the same way:
$$\nabla_W F(u^\top W) = \nabla_W (u^\top W \nabla F(v^\top)) = \nabla F(v^\top) u^\top,$$
since the post-multiplication by $\nabla F(v^\top)$ spans all linear maps taking $u^\top W$ to $\mathbb{R}$.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Reduction of state space of the game Connect Four to apply RL algorithms SARSA and Q-Learning
I would like to implement the reinforcement learning algorithms SARSA and Q-Learning for the board game Connect Four.
I am familiar with the algorithms and know about their limitations regarding large states spaces due to storing all of this information in the Q-Table. Connect Four has a large state space estimated around 7.1*1013, e.g. MIT Slide 7, which is why simply applying these algorithms won't work (though this thesis claims it did)
However, I have found this answer to a similar post that proposes a possible solution to simplify the state space.

For one thing, you could simplify the problem by separating the action space. If you consider the value of each column separately, based on the two columns next to it, you reduce N to 3 and the state space size to 106. Now, this is very manageable. You can create an array to represent this value function and update it using a simple RL algorithm, such as SARSA

Unfortunately, I don't understand the proposed simplification and would like to ask the following questions:

The action space is separated from the state space by considering each column separately. However, if my understanding of SARSA and QL is correct they use Q(S,A) to estimate the value function, therefore the state action pair is assigned a value.
How does one calculate the value of a column based on the two columns next to it?
Also what does next to it mean in this context? If the two adjacent columns of each column are used then we create five pairs (N=5) or are pairs created from the inside out (e.g. middle three columns, middle five, all seven)?
Is a state (of the entire board?) then mapped to the array containing the value function for each action/column?

Any references to other literature or simplifications would be much appreciated!


text:
Object detection: when there's only 1 object in each image
Good day. I have a custom dataset for object detection, which has imbalance that each image has only one object annotation.
I trained the object detection model(Efficientdet-dx) on TensorFlow object detection API with this dataset. But the model predicts only one object in image, even though it has many trackable objects when the training has finished. It looks like the model learned in the wrong way that it should find only one object in a image.

Here's my question: In which way should I train the model, so the model find objects independently to the number of objects in the trainset?
It should be very helpful, if you help me.
What I tried:

Copy and Paste augmentation(CAP)

result: The dataset has no mask annotation unfortunately, so I used a deep learning model trained on open dataset for background subtraction, but the subtraction did not work well. As you can see, the edge of pasted objects is not clear. and some part of the objects are missing.

Mix another dataset like COCO, PASCAL VOC.

result: It wasn't that bad such like CAP. However, the mixed dataset becomes very big size and the unrelated labels also are in the predictable label list.

So to clarify - you're saying you have a training set comprised solely of images with only one object in them, but then at testing time you're inputting images that have multiple objects?

Yes. I put images with multiple objects into my model trained with the dataset that have only 1 object per image. @Recessive

That is highly unusual, you should never train on a dataset that isn't representative of the actual data. It's like training a classifier to classify horses then trying to test it on classifying cars. Obviously it wont perform. Is there anyway you can include the images with multiple objects into your training set?

Sorry for the late reply. Shortly answer to your question about including other images is 'no'. And that's why I'm embarassed when I got this dataset and realized what's wrong. @Recessive


text:
Can neural networks learn noise?
I'm interested in the following graphs. A neural network was trained to recognise digits from the MNIST dataset and then the labels were  randomly shuffled and the following behaviour was observed. How can this behaviour be explained?
What explains the apparent 'mirroring' of the graphs on the RHS, and the fact that training error on the RHS is approx. equal to validation error on the LHS?
It's evident from the graphs that the neural network is not learning noise - so what exactly is it learning?

Hello. Welcome to Artificial Intelligence Stack Exchange. It would be very nice if you can provide the link to the code, for people interested in reproducing those plots and behaviours.

I would be interested to know if validation labels were also randomised. I think there is reason to think not from the second graph.

What explains the apparent 'mirroring' of the graphs on the RHS,

The model starts untrained and no better than random guessing (the baseline). As the training progresses, the model does better than random guessing on the training data, but does worse than initially on the validation data.
The decrease in performance is because the data it is being trained on is now deliberately labelled differently to the validation set. The task has been made impossible to generalise on, For example if there is a handwritten "8" labelled as 4 in the training database, then a very similar looking "8" in the validation database may be labelled 1 - if the trained model correctly matches up features, it will guess 4 consistently. This issue will occur frequently over the dataset, and similar issues will affect validation examples that are not close to training samples (in the feature space they will be "in-between" many training examples, most of which will be labelled differently to the validation label example).
The degree to which the curves on the random labels graph diverge is roughly affected by:

Size of the training data. A larger training dataset will cause less divergence. A near-infinite dataset would result in a flat line close to the baseline for both training and validation results.
Capacity of the model. A model with greater capacity to learn complex functions can better approximate the training data. The better it does this, the closer the training error will get to zero, and the larger the validation error will be as a result.

The mirroring effect between training and validation curves is not perfect, either in practice or "ideally". With a large dataset and very high capacity model capable of overfitting that dataset, then validation scores should be similar to random guessing and thus close to the baseline, whilst the training curve would tend towards zero error.

and the fact that training error on the RHS is approx. equal to validation error on the LHS?

That is a coincidence. Howver you can read it as roughly "despite the scrambled labels, the model does learn to predict labels on the training dataset, about as well as it generalised before on the unscrambled data".

It's evident from the graphs that the neural network is not learning noise - so what exactly is it learning?

It is learning a kind of noise - it is doing its best to approximate a function that returns a fixed set of incorrect, randomly assigned labels. When the random values are frozen in the dataset, they define a complex function that can be learned, even though it is not meaningful or useful for any other purpose.
The problem with validation error is that this noise function is not coherent, so there is no approximation that will ever do better than random chance when you look at ability to generalise. In fact it will usually do worse in rough proportion to how well it learns the training data.
If labels were somehow assigned using coherent noise e.g. Perlin noise in feature space (I'm not sure how you would do this, but it is feasible), then the model may perform better on the validation set.
If the labels were instead randomised each epoch, so that they were not fixed, then the training would learn nothing, and you would end up with a roughly flat line for both training and validation error.


text:
Non-sliding kernels for location-aware processing in Convolutional Neural Networks
My understanding of how CNN operates in image detection is through the use of kernels that slide through the image to detect features (edges and so on). So a single kernel could potentially be learning to detect an edge no matter where it is in the image. This is great for image recognition problems where an image of a dog shifted to the right or inverted is still an image of a dog. This article states "the features the kernel learns must be general enough to come from any part of the image". The article also states how using CNN for categorical data where the order in which data is organised is irrelevant can be "disastrous".
However, there are instances where it is desirable for the algorithm to be location-aware in order to classify better. Take the case of using CNN to train a network that will predict card play in the game of bridge (a version of double-dummy where all cards are laid out open - perfect information, deterministic). At the beginning of the game the cards dealt to the four could look (very unrealistically) something like this.

where Leader = the player playing the lead card in round 1, and the subsequent players organised as Leader.LeftHandOpponent, Leader.Partner and Leader.RightHandOpponent. Each player's cards are organised in four suits starting from the Trump_Suit and then the other suits in the original suit hierarchy. Cards go from highest value in the top 'A' to lowest value in the bottom '2'.
Here is a transpose of the image above.

This layout provides a lot of visual cues in terms of how the gameplay will proceed and who will end up winning how many tricks if viewed it from the perspective of control cards distribution within each suit and hand strength. So, the answer to the question of will CNN actually be able to process this data to provide good predictions is a resounding Yes (at least to me).
However, here is the problem - A regular CNN with a sliding kernel with a (4, 1) stride and no padding would make no distinction between the red boxes when in reality there is a massive difference between them.
Possible Solution? - A filter consisting of non-sliding kernels/kernels that only slide in one direction (perhaps horizontally or vertically) however would theoretically only seek to learn location-aware features and that could potentially improve accuracy? Just shooting arrow in the sky.
Has this been researched? Has anybody implemented this already? Could this work?
P.S: CNN has been used on AlphaGo Zero was great success. Obviously in the game of Go, patterns located in the top of the board carry the same weight as those located in the bottom. The gameplay does not change if the board is flipped 180 degrees. This however is not the case in the game of contract bridge. I am looking at ideas of how this can be resolved.

As you observed, convolutions are "shift/translation equivariant". This is extremely useful and beneficial for image/video/audio processing where this "symmetry" exists in the underlying domain.
This is not the case in your settings. Each card from each suit carries a different meaning. You actually want to have a different (trainable) weight for each card for each player. A fully connected layer seems more suitable for this setting.


text:
Which ML algorithm is the best for predict the next PRNG generated numbers?
I have a homework. The task is to decide, if the PRNG generated lottery is attackable/crackable or not.
Details:
Lottery:
There is a lottery game where you have to choose 8 numbers between 1-20 for the field A and choose 1 number between 1-4 for field B. It is generated every 5 minutes(7:05 - 22:00), so there are ~64k draws/year.
For example:

A: [3,    5,  6,  7,  10, 13, 17, 18]
B: 2

Possible dependent variables:
Timestamp, DrawNum (It is between 85-264 every day. 7:05 is 85 because there are 425 minutes between 00:00 and 7:05. (425/5=85))

Unfortunately we don't have too much dependent variable and there is no clue for the PRNG algorithm.
I think this two dependent variable is not enough to predict the numbers. I am thinking on an LSTM to predict the next 1stNum based on the previous ones and use the same model for the other numbers.
What do you think? How would you predict the next set of numbers?
Which ML algorithm is the best for this use case?

I don't think you need any ML here, but some data exploration and stats tests. See if you can reject the null hypothesis that the generated numbers are truly random.

@NeilSlater Thank you. I will start with that. If it is not truly random, what would be your approach?

If you find that it is not random, you can start with a simple bias towards the more likely values. If correlations between values are sophisticated, then you might benefit from some kind of AI-based approach that could help you find most-likely results. E.g. a GAN may be able to generate tickets that have higher probability of winning.

A completely different approach would be to try and reverse-engineer the PRNG in question. That is hard, not really approachable using current AI, it is more in the domain of cryptography.

For your stats tests you may want to look for correlations between values. If I were designing this homework, my first thought would be to correlate the PlusNum somehow with the draw. That may hide global variations from randomness, but be relatively easy to spot with some deeper analysis

Any PRNG is not truly random, and should in theory be crackable. Since most are based on the seed, its importand to find the current seed value and the boolean operators that it uses. You can use ML to find the later as well.


text:
What are the types of data in which the order of instances does matter?
In general, the order of instances in the datasets that are used in machine learning is immaterial. But there are exceptions. Timeseries data is one such exception I know. Consider the following two excerpts
#1: From 4.3 Representing tabular data of the textbook titled Deep Learning with PyTorch by Eli Stevens et.al.

At first we are going to assume there’s no meaning to the order in
which samples appear in the table: such a table is a collection of
independent samples, unlike a time series, for instance, in which
samples are related by a time dimension.

#2: From an answer

Moreover, there are datasets that contain elements whose order in the
dataset can be relevant for the predictions, such as datasets of
time-series data, while, in mathematical sets and multi-sets, the
order of the elements does not matter.

I want to know the types of such data in which the order of instances does matter. Are there any other kinds of data except Timeseries in which the order of instances does matter?


text:
Predict placement of an object in 3D space
I am trying to find a way to train a model to predict the correct placement of entities like a tree, dog and cat in a natural 3D environment. Any help regarding how I could use textual data to learn correct placement of objects in 3D space would be a great help. I am a little lost on how to approach this problem.

Find some university professor and make your PhD on that topic. You might use as a starting point RefPerSys. You are likely to fail, even in four years of full-time work. You certainly would be able to public interesting peer-reviewed papers.

xd, i think its time to do a PhD then.

Finding a good advisor is an important first step


text:
Neural network for recognizing ship types based on location series
I am building a neural network for recognizing ship types based on a 1000-long series of location data (latitude-longitude, normalized to account for different km/longitude° metrics, so that vector difference yields a consistent distance). The dataset I use consists of around 100 000 distinct day-ship pairs, classified by a 10-valued labeling. The cardinality of the classes are: 30115, 26327, 12798, 10940, 5859, 4211, 4176, 3639, 3521, 2834
I tried two different approaches:

A recurrent network (using LSTM): Dense[relu] -> LSTM -> Dense[relu] -> Dropout -> Dense[softmax]
A 1D convolutional network: Dense[relu] -> Conv1D -> Dense[relu] -> Dropout -> Dense[softmax]

I experimented with the hyperparameters of the above networks, but they all converge to a 40% accuracy, where the model classifies all inputs as class 0 or 1 (choosing the most likely class of the output layer).
I could accept that the data is not well-defined and this kind of prediction is impossible, but the strange thing is that even if I give the same data as training and validation, the model stops getting better at the 40% accuracy mark. Shouldn't it go further, and "memorize" the classes in this case, resulting in ~100% accuracy on the training data?

There are a lot of questions to be asked about your test setup, data preprocessing, and model architecture.  RNNs, or in your case LSTMs, can be tricky when it comes to implementation.  I suspect your problem lies somewhere with the LSTM or preprocessing.
Are all lat/lon data points in one class the same? If not one thing you might want to do is to somehow round or perform some kind of grouping for the lat and longitude within specific classes. This could help simplify the problem for the network.
Perhaps It's also possible you don't have enough features in your model, i.e. too shallow/ not enough layers.
Have you also considered this problem without a RNN? Is it really needed in this case?


text:
Reinforcement Learning applied to Optimisation Problem
Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained in the following picture.
Objective function 1:
Minimise costs = inventory costs + transportation costs + penalty costs + loading/unloading costs

Inventory costs = inventory cost at source airport + inventory costs at distribution centres

Transportation costs = cost of transporting cargo from production centre to source airport (via trucks) + cost of transporting cargo through itineraries (via flight) + cost of transporting cargo from distribution centre to transfer points (via trucks) + cost of transporting cargo from transfer point to customers (via drones)

Penalty costs = cost of operating flight routes and delay penalty costs

Loading/unloading costs = cost of loading cargo on trucks at production centres + cost of unloading cargo from trucks at the transfer point

Mathematical Solution (Using IBM CPLEX solver / Docplex): The complete python code (.ipynb file) with the formulation is present in this Google Drive Link. This gives an optimal solution.
Query: Is there any non-mathematical, non-formulation based method to solve this problem statement? Something on the lines of Reinforcement Learning? If any implementation is also provided, it will be icing on the cake.

if it can be formulated as a sequential decision making process then yes RL can be applied to a problem of this nature. The reward would have to be suitably chosen so that maximising the rewards is equivalent to minimising your cost. To me, it is not clear what the actions would be. What do you have control over in this setting? Do you control any of the scheduling aspects? Maybe you can add this detail to your question.

@DavidIreland so we are given a fixed number of itineraries (ie. number of flights), production centres, destination airports transfer points and also a fixed amount of demand of each customer. And also the parameters such as transportation and storage costs are fixed. By scheduling aspects, if you mean which flight or truck is allocated to which customer then yes it can be chosen optimally.

I am wondering if you are trying to solve the optimization problem for a specific horizon. Is RL taking into consideration the horizon cost as a vector or is it working value by value (realizing the cost at that step)? It is important because I want to be sure the agent has the necessary knowledge in terms of future horizon costs and maybe expected customer load and so on.


text:
Does Godel's incompleteness theorems restricts the scope of connectionist-AI?
It is well-known that Godel's incompleteness theorems restricted the reachability of symbolic-AI, which is dependent on mathematical logic.
But, I am wondering whether it has any impact on the connectionist AI.
I don't think it has any impact on the capability of connectionist AI because of the following reasons I am aware of

Connectionist-AI is more focused on generalization and is not about mathematical logic..
Universal approximation theorem, contrary to Godel's incompleteness theorems says that connectionist-AI is capable of achieving all bounded-continuous functions. I am not sure about the implications of Godel's incompleteness theorems on either unbounded or discrete functions.

So, the incompleteness theorems seem to have no impact on the connectionist AI.
Do the theorems also restrict the reachability of connectionist-AI?


text:
When does an RNN use the connections that help in going backward in time?
Consider the following paragraph taken from chapter 10: Sequence Modeling: Recurrent and Recursive Nets of the textbook named Deep Learning by Ian Goodfellow et al mentioning the connections of RNN to go backward in time.

For the simplicity of exposition, we refer to RNNs as operating on a
sequence that contains vectors $x^{(t)}$ with the time step index $t$
ranging from 1 to $\tau$. In practice, recurrent networks usually
operate on minibatches of such sequences, with a diﬀerent sequence
lengthτfor each member of the minibatch. We have omitted the minibatch
indices to simplify notation. Moreover, the time step index need not
literally refer to the passage of time in the real world. Sometimes it
refers only to the position in the sequence. RNNs may also be applied
in two dimensions across spatial data such as images, and even when
applied to data involving time, the network may have connections
that go backward in time, provided that the entire sequence is
observed before it is provided to the network.

The paragraph says that the RNN can go back in time if and only if the entire sequence is provided. So, I am suspecting that it happens only during the backpropagation/backward pass.
Am I true? Or is it possible for an RNN to use those connections while forward pass also?

The recurrent connections are also used during the forward pass. Take a look, for example, at the following equations that compute (during the forward pass) the hidden state $h_t$ of an LSTM layer.
\begin{align}
f_t &= \sigma_g(W_{f} x_t + \color{red}{U_{f}} h_{t-1} + b_f) \\
i_t &= \sigma_g(W_{i} x_t + \color{red}{U_{i}} h_{t-1} + b_i) \\
o_t &= \sigma_g(W_{o} x_t + \color{red}{U_{o}} h_{t-1} + b_o) \\
\tilde{c}_t &= \sigma_c(W_{c} x_t + \color{red}{U_{c}} h_{t-1} + b_c) \\
c_t &= f_t \circ c_{t-1} + i_t \circ \tilde{c}_t \\
h_t &= o_t \circ \sigma_h(c_t)
\end{align}
The letters in $\color{red}{\text{red}}$ represent the matrices associated with the $\color{red}{\text{recurrent connections}}$, so we use them in the forward pass. Of course, before training, these connections are useless, in the sense that, if we initialize them randomly, they do not initially provide any context. However, during the backward pass, we update them based on our (labeled) data, so, over time, they should capture the temporal/recurrent relations of the sequences.


text:
Should I represent my reinforcement learning as an episodic or continuous task?
I would like the community to help me understand if the following example would be better represented as episodic or continuous task, this will help me structure the problem and chose the right RL algorithm.
The agent start with an initial score x of let's say 100. The agent objective is to maximise it's score. There is no upper bound! Theoretically the agent can get a score up to infinity, and there is no termination based on the number of steps, therefore the agent could play forever. However, the score can't be negative and if the agent get to a score of zero, the episode should terminate and the environment reset. I am undecided what would be the best representation, because if the agent learns how to play, the episode would never terminate, and the agent would theoretically play forever. However if the score get to zero, there is no way for the agent to continue playing so the environment needs to reset. Thank you.

your problem is episodic as there is a terminal state. this is similar to cart pole in that a successful agent would never have an end of episode because it can perfectly balance the pole without it dropping. It may be worthwhile having an artificial stopping criteria for training — e.g. stopping after 5000 steps and restarting (but don’t count this as a termination). The number of steps would depend on your environment and how much it changes as the length of the episode goes on

Thank you for your answer! The environment is quite chaotic and ever changing, and the agent should always try to adapt to it, therefore it is not quite like the cart pole in the sense that  in cart pole, once the agent has learned how to balance the pole will stay always upright. But i see your point, and i think you may be right

In which case, I would definitely recommend using some kind of TD algorithm (Actor Critic/Q-Learning) and letting episodes run as usual. Make sure to add checkpoints to save the model so you can kill it when it is sufficiently trained


text:
Why is training all layers at a time effective for a multi-layer autoencoder?
This training of all layers of a CNN simultaneously is standard practice today. It is found in every CNN (AlexNet (2012), VGG, Inception, GANs, etc) and even pre-CNN networks such as Le et al. 2012.
What is the advantage of training all the layers simultaneously? Wouldn't the later layers be learning from poor lower layers to start with, and have to re-learn to adapt? And why would there ever be an advantage for an autoencoder like Le et al. 2012 where there is no backpropagation to communicate from the later layers to the earlier layers?
I think the conventional answer is that the lower layers can actually learn to provide low-level features that support the layers above. An example of this is learning to detect a horizontal yellow-blue feature to detect the water line in a beach scene.
But couldn't the yellow-blue feature be found just as easily by training the lower layers first? This would be especially true of an autoencoder such as Le et al. 2012, which picks up on patterns in the training set without having ground truth-labels to group them.
Citations to experiments or theoretical work that directly answers this question would be appreciated!
This is a follow-on to an earlier question.

I suspect there are two principles at play here, but I don't know which principle is more important in this case.
The first principle is the conventional wisdom: The lower layers not only learn to summarize the visual features found in the training set, but also learn those features which allow discrimination between categories.  For example, in a beach image, there are likely many subtle yellow vs yellow or blue vs blue features that are technically part of the beach image.  But where we sea the line in the sky may be a stronger indication of what this picture is of.  All the features would be important for reconstructing an image of the beach, but the horizon line may be more important for distinguishing the beach from a picture of the sun in the sky for example.  In a CNN like AlexNet, the horizon line would receive more weight early on in training.
But I don't see how this first principle would apply in the case of an autoencoding network like Le et al. 2012
The second principle is that random weights actually have some value from the very beginning of training, and those features which are useful early in training for a particular high-level decision will become even more useful later in training as they pick up patterns from the images to which they relate.
But I again don't see how this applies to an autoencoder like Le et al.


text:
Watkins' Q(λ) with function approximation: why is gradient not considered when updating eligibility traces for the exploitation phase?
I'm implementing the Watkins' Q(λ) algorithm with function approximation (in 2nd edition of Sutton & Barto).
I am very confused about updating the eligibility traces because, at the beginning of chapter 9.3 "Control with Function Approximation", they are updated considering the gradient: $ e_t = \gamma \lambda e_{t-1} + \nabla \widehat{q}(S_t, A_t, w_t) $, as shown below.

Nevertheless, in Figure 9.9, for the exploitation phase the eligibility traces are updated without the gradient: $ e = \gamma \lambda e $.

Furthermore, by googling, I found that the gradient is simplified with the value of the i-th feature: $ \nabla \widehat{q}(S_t, A_t, w_t) = f_i(S_t, A_t)$.
I thought that, in Figure 9.9, the gradient is not considered because in the next step the eligibility traces are increased by 1 for the active features. So, the +1 can be seen as the value of the gradient, as I found on google, being binary features. But I'm not sure.
So, what is (and why) the right rule to update the eligibility traces?


text:
What inherent quality of a function makes it treated as either loss or evaluation metric?
A neural network model needs a loss function for training. The neural network needs to minimize the loss function.
A neural network is evaluated after training using a metric. The neural network needs to either minimize or maximize the metric depending on the context.
Suppose $L$ is the loss function used and $M$ is the metric/evaluation function. Assume the metric needs to be minimized and is calculated based on the output of the neural network. We can use $L+M$ as the loss function. It looks to me that it may be up to the choice of the designer to use certain function $f$ for either $L$ or $M$ as they try to quantify how good/bad the model is working.
But, in the literature, if we observe, there are some fixed loss functions and fixed metrics for evaluation depending on the underlying task.
With this context, what is the inherent quality of the function that makes it treated as either loss or evaluation metric?

Common loss functions, like the cross-entropy or mean squared error, are chosen because, if you minimize them, you are actually maximizing the likelihood of the parameters given the observed data. In other words, you are trying to find the probability distribution $p(y \mid x; \theta)$, parametrized by $\theta$ (the parameters of your model), which is most consistent with your data $D = \{(x_i, y_i) \}_{i=1}^N$. In the case of binary classification problems, for instance, you assume that $p(y \mid x; \theta)$ is a Bernoulli distribution or a family of Bernoulli distributions (i.e. the likelihood) parametrized by $\theta$ (see more details here). Similarly, in the case of regression problems, you can assume that $p(y \mid x; \theta)$ is a Gaussian. There are other examples of loss functions that are mathematically motivated, for instance, the ELBO or the GAN objective function. The same can be said, for example, for loss functions with regularisation, like the MSE with weight decay. These are mathematically motivated (in this case, for instance, you can derive the MSE with weight decay as a MAP estimate rather than an MLE).
So, loss functions are not chosen arbitrarily, or we don't choose the binary cross-entropy for binary classification problems just because everyone else does (well, ideally, this shouldn't be the reason, but I guess many people don't know the details that I've just mentioned above). We often model our problem with probability theory and we try to find a probability distribution (or the mean/mode or even variance of such a probability distribution) that most likely generated our data.
Now, if you train a neural network with gradient descent and backpropagation, it's assumed that you can also find the derivative of your loss function with respect to the parameters of your model. So, this is one of the properties that a loss function is expected to have in this training regime, i.e. it needs to be differentiable.
One of the most commonly used metrics to assess the quality of a classifier is the accuracy, which I think is not differentiable (although I don't have the proof and I didn't think too much about it). So, that's probably one of the reasons why people wouldn't use the accuracy as a loss function anyway. Now, keep in mind that the accuracy can actually be a very bad metric to assess the quality of your model because, if your dataset is unbalanced (and you didn't check that and the true distribution is not), your classifier could simply always predict the most common class and get a high accuracy, but this would be misleading.
So, in summary, a function is often used as a loss function (in the context of training neural networks with GD) if

it is differentiable, and,
if you minimize it, you're actually trying to learn a probability distribution

Note that the second property may not always be a requirement, but all the loss functions that I can think of right now have these two properties.
On the other hand, a function is used as a metric if you care about what that specific function evaluates. For example, if you use the accuracy, you want to know, on average, how many mistakes the model makes, but you don't care whether it makes more mistakes for one class than the other. However, you can also use other functions, like precision and recall, which evaluate other behaviors, like how many mistakes you make for one class rather than the other. This can make sense, for example, in the context of detecting a disease.
In principle, you could use a loss function also as a metric. However, is that really what you want? For instance, if you used the MSE as a metric, it's not clear how small it must be in order to have a "good" model. It all depends on which behavior of the model that you want to evaluate. You can evaluate different behaviors of the model, so that's why you often have more than one metric.


text:
Is it possible that a deep neural network, with some variations, can be used for multiple tasks?
I am asking this question on deep neural network architectures only. If you want to restrict the domain of tasks then you can choose computer vision for this question.
Suppose there is an architecture that performs well on a task. Is it possible can edit or append the first or last few layers and then it performs similarly well on the other task?
If yes, please provide me an example of such architecture that performs well on at least a couple of tasks.

It seems that you're looking for something similar to transfer learning. Let me know if that's the case or not.

A pre-trained model is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. You either use the pretrained model as is or use transfer learning to customize this model to a given task. #1/#2

Fine-Tuning: Unfreeze a few of the top layers of a frozen model base and jointly train both the newly-added classifier layers and the last layers of the base model. This allows us to "fine-tune" the higher-order feature representations in the base model in order to make them more relevant for the specific task. #2/#2

Yeah @nbro. It is the same-thing I am looking for. It seems that task is not fixed and hence we can use pre-trained model for any task under consideration.


text:
Can CNNs detect image similarity?
I have been running some experiments to see whether a CNN can detect whether two images are the same. However, I can't seem to make it work. I am wondering whether CNNs are not able to do what I am trying, or whether I am making mistakes.
This is the setup: I take two 128x128 images and append them to be 128x256. These two images are either exactly the same:

or different:

I have also experimented with a black bar in the middle. The entire appended 128x256 image(thus containing two images) is fed into a CNN with the following setup:

Depending on the dropout usage, the model is able to learn training data between 80-100% accuracy. But the validation accuracy never gets above around 65%.
I do not understand why the model is not able to get to 100% very easily, as the input images are either EXACTLY the same, or very different. Seems like a trivial task to me. I would assume that the convolutional kernels would create exactly the same feature representations of the images and then feed that into the dense layers.
ps. I know there are better ways to do this but I wanted to experiment with this.

As you state, the main problem is why in the training you can get 80-100% but not in the test, so, in this case a lot of factors may affect this, and we need more information

For example, What is your test data like? how many data train do you have? how about the ratio? do you use the same object/distribution? how do you validate? do you use cross validation? (these questions check if there is overfitting)

"why the model is not able to get to 100% very easily" even in simple image classification with small classess (not similarity), it's not easy to get 100% in validation

Can you please provide the actual code or link to code rather than a screenshot of the code? In reality, almost no question asked on this site should require users to show us code, but sometimes it can be useful. I would like to note that programming questions are off-topic here and I would recommend that you read our on-topic page. https://ai.stackexchange.com/help/on-topic

This is a wild guess, but I have a strong feeling that this problem can be easily solved by using a deeper neural network. Try using well-known architectures like Resnet first and see if you still have the same issue. If not, the model is underfitting due to its lack of complexity.

@DKDK I got 97% accuracy with just 170 model parameters on a more difficult dataset ("identical" images are actually augmented), see my answer below.

An interesting issue: the model is seemingly under-fitting to the train data (since the accuracy isn't that high), but it is also underfitting to the training data. :)
When you flatten the data and use dense layers, you lose all of the spatial information. Before you do that, you might want to subtract the left side of the output (I calculated it having a size of 15x30x128, so a half would be 15x15x128?) from the right side. This is a bit tricky with the Sequential model, but it can be done with a Lambda layer. Note that the tensor's first dimension is None, which is actually the batch size during training.
After doing this you may keep the current flatten + dense structure and see how it goes. Although I suppose it can be much simpler at this point, Conv2d is doing the heavy lifting.
You might want to add dropouts to the earlier part of the model as well, or use enough data augmentation to avoid overfitting. The black bar in the middle is obstructing the image, making them very non-identical.
Alternatively you could make a single feature extraction model and feed it these two inputs separately. This architecture is called a Siamese Network, and requires a bit more code to implement (example in the article, however it uses PyTorch which seems quite verbose when compared to Keras).
Actually the first suggestion is a Siamese Network as well, just in a bit odd structure having just one input. I'd be interested to hear if this works better for you.
Edit: I decided to try out the problem myself. Using the idea described above, I got 99% validation accuracy on images which aren't even identical, using a model with 194 parameters. Half of the examples have a different image, and the other half is a modified copy of the reference (random +/- 10 degrees of rotation, +/- 10% of zoom with TF's ImageDataGenerator).
I won't share the code yet in case you want to figure it out yourself, but here is the summary which shows the dataflow (uses the Sequential API):
Model: "sequential_173"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_132 (Conv2D)          (None, 126, 254, 3)       84
_________________________________________________________________
max_pooling2d_88 (MaxPooling (None, 63, 127, 3)        0
_________________________________________________________________
conv2d_133 (Conv2D)          (None, 61, 125, 3)        84
_________________________________________________________________
max_pooling2d_89 (MaxPooling (None, 30, 62, 3)         0
_________________________________________________________________
lambda_60 (Lambda)           (None, 30, 30, 3)         0
_________________________________________________________________
average_pooling2d_47 (Averag (None, 5, 5, 3)           0
_________________________________________________________________
lambda_61 (Lambda)           (None, 5, 5)              0
_________________________________________________________________
flatten_52 (Flatten)         (None, 25)                0
_________________________________________________________________
dense_435 (Dense)            (None, 1)                 26
=================================================================
Total params: 194
Trainable params: 194
Non-trainable params: 0
_________________________________________________________________

It takes a "twin" RGB image in, and applies two Conv2D layers with just 3 kernels on each of them. This leads to 30 x 31 x 3 shaped image features (the width of max_pooling2d_89's output is actually 62 since there are two images side-by-side).
The two sides are subtracted from each other (cropping out the middle column) and the result is downsampled and aggregated, resulting in a 5 x 5 output. The final Dense layers is just a sigmoid to predict whether the images are nearly identical or not.
Example result is shown below. It has six columns of images: left image, right image, left's features (conv2d_133's output), right's features, feature difference and weighted feature difference. The last column shows that the model has learned to focus more on the center of the image rather than corners, because the used image augmentation distorts the corners the most.

If the image is downsampled to just 1 x 1 before the Flatten layer then the model has only 170 parameters, but the validation accuracy drops a bit to 97%. (edit 2: using elu rather than tanh activation brings this model's accuracy to 99.3%, I originally used tanh since it is easy to visualize as an image due to its limited range)
The used dataset is Indoor Scene Recognition, of which I used 2000 random images.


text:
How to compute the loss for a sequence labeling task without the Softmax distribution?
For a sequence labeling task (NER), we compute the loss by passing the softmax distribution of the classes (e.g. vocabulary) with the gold label to the loss function (nn.CrossEntropyLoss(y, x)):
y = tensor([[ 1.4775,  0.7022,  0.7499, -0.7535, -1.4983, -2.3193, -0.6166, -2.3302,
     -0.8847, -0.1915]])

label:
x = torch.tensor([2])

what if we don't have the softmax distribution and we have the predicted label instead, how can we compute the loss? Following the example above, 2 is given for prediction instead of y.

I guess you will need to assume that the predicted class has a probability $1$ and all other classes have probability $0$, unless you have more info. Can you clarify why the predicted classes/labels in your case are hard rather than soft (probabilities)?

Thanks @nbro .. this is what I actually have done, but I felt that this will confuse the model during training when the predicted label is not the correct one because all the other classes will be treated equally (zeros).


text:
Which data representation of text as input for NLP Deep Learning models?
I have been given a data set with 30.000 text documents (each text file is rather small with respect to its length and consists in most cases of around 20 sentences), which are labelled with 0 or 1. Using this data set, I want to train machine learning and deep learning models in order to be able to classify new text files.
On the one hand, I want to use classical machine learning models (such as logistic regression, random forest, SVM, etc.) with the Bag of Words (BoW)/TF-IDF approach. In this context, the text data are represented by a matrix with 30.000 rows (and a number of columns that correspond to the unique words in the overall data set), where each row stands for one observation (i.e., a text document) and each column stands for a unique word. The entries of this matrix are then (kind of) frequencies of the unique words in a text document. However, these approaches do not take the sequence of words, negations, etc. into account.
On the other hand, I want to use new deep learning models (such as RNN, LSTM, BERT, XLNET, etc.), which take the sequence of words, etc. in a text document into account. Obviously, the data set of text files cannot be represented with the BoW or TF-IDF approach in this case as this would neglect the order of words, etc.
Which data representation technique can be used to input a data set with labelled text files into a deep learning model (such as RNN, LSTM, BERT, XLNET, etc.)? Is there something similar like the BoW or TF-IDF approach that also pays attention to the sequence of words, etc.?

For deep learning models, embedding vectors have become the standard way of encoding text features almost immediately after their introduction.
The reason for this is that neural networks work with data encoded with continuous values ranging from 0 to 1 (or sometimes from -1 to 1). Bag of Words and TF-IDF can be modified to produce values in this range, but it wouldn't make much sense conceptually, especially for Bag of Words techniques where each number is simply a code and it doesn't represent any inner property of the encoded text.
Embedding vectors instead are designed to be dense representation, i.e. they possess all properties of a vector space, allowing us to perform math with them, e.g. computing the "distance" between two words, so different numbers in this case do represent different properties for each word.
Embedding vectors also contain information about sequences. Mathematically they are derived from factorization of frequency (or co-occurrence) matrices, similar to TF-IDF, which are then compressed (with various dimensionality reduction techniques) to a set of vectors of fixed size.
Of course the initial frequency matrix can be designed to encode a single word-pair probability per element, but nothing prevent us to make frequency matrices containing co-occurrence frequencies of triples or quadruplets (or more) of words.
Some embedding are also trained using neural networks, most famous example being Word2Vec, more specifically the CBOW (Continuous Bag of Words) variation, a simple multi layer perceptron trained to project a word to a dense vector given a sequence of one hot encoded context words (see image below).
Given you rather large set of documents, you could train your own domain specific CBOW embeddings using a context widow of your choice. There are though many available pretrained embeddings that can be used directly, trained on massive datasets (billions of documents) like Wikipedia, Amazon reviews or Twitter. These embeddings usually outperform embeddings trained on small datasets due to the huge amount of common sense relationships encoded in them. I would therefore suggest you to try these sources before digging into training your own vectors:

Glove: Global Vectors for Words Represetation
FastText
Google News vectors


text:
How to colorize images with Variational Autoencoder?
CONTEXT
I'm trying to colorize images with Variational Autoencoder. Input is 256x256 gray image. Output is 256x256x2 as I convert image to a LAB color space and then put gray channel as input and other two as outputs.
PROBLEM
My network is training, but loss is not getting smaller epoch after epoch.
Epoch 2/5 720/720 [==============================] - 34s 47ms/sample - loss: 0.1081 - val_loss: 0.0860
Epoch 3/5 720/720 [==============================] - 35s 48ms/sample - loss: 0.1074 - val_loss: 0.0853
Epoch 4/5 720/720 [==============================] - 34s 48ms/sample - loss: 0.1070 - val_loss: 0.0855
Epoch 5/5 720/720 [==============================] - 34s 48ms/sample - loss: 0.1068 - val_loss: 0.0854

After training I can put gray image into the network to get color channels, but I always get the same sort of orange dot in the middle of the screen. No matter how long I train or how many training images I use.
Result:

Tech stuff
Network I use:
            dropout = 0.25

            input_data = tensorflow.keras.layers.Input(shape=(IMG_HEIGHT, IMG_WIDTH, 1))

            encoder = tensorflow.keras.layers.Conv2D(64, (3, 3), activation='relu', padding='same')(input_data)
            # encoder = tensorflow.keras.layers.MaxPooling2D((2, 2))(encoder)
            encoder = tensorflow.keras.layers.Dropout(dropout)(encoder)

            encoder = tensorflow.keras.layers.Conv2D(128, (3, 3), activation='relu', padding='same')(encoder)
            encoder = tensorflow.keras.layers.MaxPooling2D((2, 2))(encoder)
            encoder = tensorflow.keras.layers.Dropout(dropout)(encoder)

            encoder = tensorflow.keras.layers.Conv2D(128, (3, 3), activation='relu', padding='same')(encoder)
            # encoder = tensorflow.keras.layers.MaxPooling2D((2, 2))(encoder)
            encoder = tensorflow.keras.layers.Dropout(dropout)(encoder)

            encoder = tensorflow.keras.layers.Conv2D(256, (3, 3), activation='relu', padding='same')(encoder)
            encoder = tensorflow.keras.layers.MaxPooling2D((2, 2))(encoder)
            encoder = tensorflow.keras.layers.Dropout(dropout)(encoder)

            encoder = tensorflow.keras.layers.Conv2D(256, (3, 3), activation='relu', padding='same')(encoder)
            # encoder = tensorflow.keras.layers.MaxPooling2D((2, 2))(encoder)
            encoder = tensorflow.keras.layers.Dropout(dropout)(encoder)

            encoder = tensorflow.keras.layers.Conv2D(512, (3, 3), activation='relu', padding='same')(encoder)
            encoder = tensorflow.keras.layers.MaxPooling2D((2, 2))(encoder)
            encoder = tensorflow.keras.layers.Dropout(dropout)(encoder)

            encoder = tensorflow.keras.layers.Flatten()(encoder)
            encoder = tensorflow.keras.layers.Dense(16)(encoder)

            def sample_latent_features(distribution):
                distribution_mean, distribution_variance = distribution
                batch_size = tensorflow.shape(distribution_variance)[0]
                random = tensorflow.keras.backend.random_normal(
                    shape=(batch_size, tensorflow.shape(distribution_variance)[1]))
                return distribution_mean + tensorflow.exp(0.5 * distribution_variance) * random

            distribution_mean = tensorflow.keras.layers.Dense(2, name='mean')(encoder)
            distribution_variance = tensorflow.keras.layers.Dense(2, name='log_variance')(encoder)
            latent_encoding = tensorflow.keras.layers.Lambda(sample_latent_features)(
                [distribution_mean, distribution_variance])

            encoder_model = tensorflow.keras.Model(input_data, latent_encoding)
            encoder_model.summary()

            decoder_input = tensorflow.keras.layers.Input(shape=(2))
            decoder = tensorflow.keras.layers.Dense(512)(decoder_input)
            decoder = tensorflow.keras.layers.Reshape((1, 1, 512))(decoder)
            decoder = tensorflow.keras.layers.Conv2D(512, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)

            decoder = tensorflow.keras.layers.Conv2D(256, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)

            decoder = tensorflow.keras.layers.Conv2D(128, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)

            decoder = tensorflow.keras.layers.Conv2D(64, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)

            decoder = tensorflow.keras.layers.Conv2D(32, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)

            decoder = tensorflow.keras.layers.Conv2D(32, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)  #
            decoder = tensorflow.keras.layers.Conv2D(32, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)  #
            decoder = tensorflow.keras.layers.Conv2D(16, (3, 3), activation='relu', padding='same')(decoder)
            decoder = tensorflow.keras.layers.UpSampling2D((2, 2))(decoder)

            decoder_output = tensorflow.keras.layers.Conv2D(2, (3, 3), activation='tanh', padding='same')(
                decoder)

            decoder_model = tensorflow.keras.Model(decoder_input, decoder_output)
            decoder_model.summary()

            encoded = encoder_model(input_data)
            decoded = decoder_model(encoded)

            vae = tensorflow.keras.models.Model(input_data, decoded)

            def get_loss(encoder_mu, encoder_log_variance):
                def vae_r_loss(y_true, y_predict):
                    reconstruction_loss_factor = 10
                    reconstruction_loss = tensorflow.keras.backend.mean(
                        tensorflow.keras.backend.square(y_true - y_predict), axis=[1, 2, 3])
                    return reconstruction_loss_factor * reconstruction_loss

                def vae_kl_loss(y_true, y_pred):
                    kl_loss = -0.5 * tensorflow.keras.backend.sum(
                        1.0 + encoder_log_variance - tensorflow.keras.backend.square(
                            encoder_mu) - tensorflow.keras.backend.exp(encoder_log_variance), axis=1)
                    return kl_loss

                def vae_loss(y_true, y_pred):
                    r_loss = vae_r_loss(y_true, y_pred)
                    kl_loss = vae_kl_loss(y_true, y_pred)
                    return r_loss + kl_loss

                return vae_loss

            vae.compile(loss=get_loss(distribution_mean, distribution_variance), optimizer='adam',
                        experimental_run_tf_function=False)
            vae.summary()

            history = vae.fit(X, y, epochs=5, batch_size=8, validation_split=0.2)

Network summary:
Model: "encoder"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
input_1 (InputLayer)            [(None, 256, 256, 1) 0
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 256, 256, 64) 640         input_1[0][0]
__________________________________________________________________________________________________
dropout (Dropout)               (None, 256, 256, 64) 0           conv2d[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 256, 256, 128 73856       dropout[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, 128, 128, 128 0           conv2d_1[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 128, 128, 128 0           max_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 128, 128, 128 147584      dropout_1[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 128, 128, 128 0           conv2d_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 128, 128, 256 295168      dropout_2[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 64, 64, 256)  0           conv2d_3[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 64, 64, 256)  0           max_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 64, 64, 256)  590080      dropout_3[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 64, 64, 256)  0           conv2d_4[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 64, 64, 512)  1180160     dropout_4[0][0]
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, 32, 32, 512)  0           conv2d_5[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout)             (None, 32, 32, 512)  0           max_pooling2d_2[0][0]
__________________________________________________________________________________________________
flatten (Flatten)               (None, 524288)       0           dropout_5[0][0]
__________________________________________________________________________________________________
dense (Dense)                   (None, 16)           8388624     flatten[0][0]
__________________________________________________________________________________________________
mean (Dense)                    (None, 2)            34          dense[0][0]
__________________________________________________________________________________________________
log_variance (Dense)            (None, 2)            34          dense[0][0]
__________________________________________________________________________________________________
lambda (Lambda)                 (None, 2)            0           mean[0][0]
                                                                 log_variance[0][0]
==================================================================================================
Total params: 10,676,180
Trainable params: 10,676,180
Non-trainable params: 0
__________________________________________________________________________________________________
Model: "decoder"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_2 (InputLayer)         [(None, 2)]               0
_________________________________________________________________
dense_1 (Dense)              (None, 512)               1536
_________________________________________________________________
reshape (Reshape)            (None, 1, 1, 512)         0
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 1, 1, 512)         2359808
_________________________________________________________________
up_sampling2d (UpSampling2D) (None, 2, 2, 512)         0
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 2, 2, 256)         1179904
_________________________________________________________________
up_sampling2d_1 (UpSampling2 (None, 4, 4, 256)         0
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 4, 4, 128)         295040
_________________________________________________________________
up_sampling2d_2 (UpSampling2 (None, 8, 8, 128)         0
_________________________________________________________________
conv2d_9 (Conv2D)            (None, 8, 8, 64)          73792
_________________________________________________________________
up_sampling2d_3 (UpSampling2 (None, 16, 16, 64)        0
_________________________________________________________________
conv2d_10 (Conv2D)           (None, 16, 16, 32)        18464
_________________________________________________________________
up_sampling2d_4 (UpSampling2 (None, 32, 32, 32)        0
_________________________________________________________________
conv2d_11 (Conv2D)           (None, 32, 32, 32)        9248
_________________________________________________________________
up_sampling2d_5 (UpSampling2 (None, 64, 64, 32)        0
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 64, 64, 32)        9248
_________________________________________________________________
up_sampling2d_6 (UpSampling2 (None, 128, 128, 32)      0
_________________________________________________________________
conv2d_13 (Conv2D)           (None, 128, 128, 16)      4624
_________________________________________________________________
up_sampling2d_7 (UpSampling2 (None, 256, 256, 16)      0
_________________________________________________________________
conv2d_14 (Conv2D)           (None, 256, 256, 2)       290
=================================================================
Total params: 3,951,954
Trainable params: 3,951,954
Non-trainable params: 0
_________________________________________________________________
Model: "vae"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 256, 256, 1)]     0
_________________________________________________________________
encoder (Model)              (None, 2)                 10676180
_________________________________________________________________
decoder (Model)              (None, 256, 256, 2)       3951954
=================================================================
Total params: 14,628,134
Trainable params: 14,628,134
Non-trainable params: 0
_________________________________________________________________
Train on 720 samples, validate on 180 samples

Questions

Why my network seems to not doing any learning?
Are loss functions defined properly?

How large is your training dataset? How many epochs have you trained the VAE for? Only 5 epochs? That's probably not enough.

No, I've tried training it up to 500 epochs. Result is still the same as for 2, 100 or 250.


text:
Does reinforcement learning lend itself well to changes in the environment due to external factors?
I've been reading about reinforcement learning, and it seems to me that reinforcement learning assumes the environment is static, and therefore that the reward for taking a particular action will be the same from time to time. But what if something happens outside the agent's control, that changes the reward?
As a simplified example, suppose the agent in a RL algorithm is a grape farmer living on a planet known for extreme weather events. Its goal is to maximize profit/minimize losses. Grape seeds cost 1000 dollars. It takes 1 year for the grape seeds to grow delicious grapes. There is a 20% chance of the weather destroying the grapes during the year, leading to the agent losing 1000 dollars. However, if the weather does NOT destroy the grapes, the agent wins $2000.
The agent starts the year with two choices:

Grow grapes
Don't grow grapes

In its first year, an adverse weather event occurs, killing the grapes. Will the agent ever grow grapes again, or will it forever decide that the best way to win is to not plant grapes? Likewise, if it ever DOES grow grapes, can we trust it to know when not to grow grapes?

Why do you say the weather is external to the environment? Your environment is the weather.

In the question, you are not describing the environment changing. Instead, there is a fixed 20% chance of a bad weather event each year. Such events can me modelled as a static environment with stochastic results.*
If nothing else happened in the year, it is easy to calculate the expected immediate reward for each action:
Not planting seeds. $0$
Planting seeds. $-1000 + 0.8 \times 2000 = 600$
So the question resolves to: Can a learning algorithm discover these odds, and make the correct decision, based only on its experience of individual years? Perhaps it will be unlucky in its first year and simply give up?
The answer is yes, most reinforcement learning (RL) algorithms will cope just fine, and learn that planting seeds is the best option in the simple environment that you have proposed.
There is an important caveat: To learn the true value of each option, algorithm must be one that explores. That means, whichever action the agent thinks is currently best, there must be some chance that it decides to take (what it thinks is) a worse option. If the worse option turns out better than it expected, the agent can adjust its expectations, and maybe choose that one more frequently in future, and vice versa if the option turns out worse than expected.
Nearly all RL methods do this, by setting some kind of exploration parameter. The more the agent explores, the more it will learn about alternatives to its current best guess at optimal behaviour. However, whilst exploring it may pay the price for making many sub-optimal decisions. Your planning agent may decide to not plant anything every few years, just to see if that is better, even after it has established that planting is the best way to get profit.
It is often hard to balance between the learning obtained by trying different options vs doing what the agent currently thinks is best. This is called the exploration-exploitation tradeoff, and is a fundamental issue in RL.

* For comparison, a non-stationary environment in RL would be one in which the probabilities and/or reward distributions change so that values learned from older experience are no longer valid. If climate change impacted your grape-growing region such that after some year the chance of a bad storm was 80%, and the agent had no way of knowing that in advance, then the agent might lose money trying to follow its previously best policy. If it was still learning, it would adapt to these new rules eventually though.

Ok, so suppose the weather changes not by chance, but by motivation. There is some superhuman or other entity that might want to alter the weather such that the grapes die. Can the agent learn to adapt?

@MontanaBurr: That depends on what you mean by "motivation". You need to be clear in your definitions if you want an answer. Most importantly, is the "motivation" based on some non-random factor, and can the agent observe or predict that factor? This is quite different from your original question, needs clarification, and there could be lots of detail in any answer, so I suggest you ask a new question about that.


text:
Why are the learned offsets of anchor boxes in the RCNN object detection models scale invariant?
In the original RCNN paper (https://arxiv.org/pdf/1311.2524.pdf) and continued in later RCNN papers such as faster RCNN (https://arxiv.org/pdf/1506.01497.pdf) the learned offsets of the anchor boxes are scale-invariant. For example the learned x-center offset $d_{x}$:
$d_{x} = (x-x_{a})/w_{a}$
is meant to parameterize the difference between the x-center of the anchor box and the new predicted box. However, it is also scaled by the width $w_{a}$ to make it scale invariant. Why is it important that this is scale invariant? Does it just make it easier for the neural network to learn, similar to something like batchnorm?


text:
Perform clustering on high dimensional data
Recently I trained a BYOL model on a set of images to learn an embedding space where similar vectors are close by. The performance was fantastic when I performed approximate K-nearest neighbours search.
Now the next task, where I am facing a problem is to find a clustering algorithm that uncovers a set of clusters using the embedding vectors generated by the BYOL trained feature extractor [dimension of the vector is 1024 & there are 1 million vectors]. I have no information apriori about the number of classes i.e. clusters in my dataset & thus cannot use Kmeans. Is there any scalable clustering algorithm that can help me uncover such clusters. I tried to use FISHDBC but the repository does not have good documentation.

You can use K-means even without knowing a priori the amount of classes. Take a look at the definition of Silhouette score, it's a generic approach applicable to any clustering method that requires an input value for the final amount of clusters to generate.
The silhouette score represents an average between the inner similarity of each cluster elements and outer dissimilarity between each cluster element and all elements belonging to different clusters. The higher the silhouette score, the better the clusters split the data.
Only drop down of the approach is the necessity to run the clustering algorithm with several different initialization values for n: number of clusters. That's the only way to get a graph like the one show below, in other to understand the optimal value for n (in this fake example 3).


text:
How can PPO be combined with HER?
I ask because PPO is apparently an on-policy algorithm & the HER paper says that it can be combine with any off-policy algorithm. Yet I see GitHub projects that have combined them somehow?
How is this done? And is it reasonable?

Any news on this?


text:
How Come My (D)DQN Fails To Learn?
I am currently trying to teach a (D)DQN algorithm to play a 10x10 GridWorld game, so I can compare the two as I increase the number of moves the agent can take.
The rewards are as follows:
A step = -1
key = 100
Door = 100
Death wall = -100
See the setup of the AI in the code.
My problem is, that regardless of what I do, it ends up following the same strategy of just going to the outer walls and staying there. I presume this is because, the outer walls givve the least amount of punishment per step, as the risk of dying is decreased considerably. At the same time, as the moves increase, the chance of ending up at the outer walls increases as well (see the heatmaps)

I've tried the following:

Drastically decreasing the decay of the epsilon, such that it reaches its final state only
in the last 10% of the training steps.
Running 100k moves just to add to the EMR before I start actually counting the steps.
Increasing the size of the network
Giving out a negative -2 reward for staying in the same tile
Feeding it the whole grid as the input vector

None of this has worked. The longest I have trained a single model was for 14 million training steps. Still, same strategy as before.
The way I evaluate the model is by(in this order):

At every 1 millionth training step I run 50-100k evaluation steps where I record the outcome of every step
Generating a heatmap to see whether or not it remains in the same few places (which are not the key or the door)
Running its best policy and visually estimating whether or not it has improved

CODE:
https://github.com/BrugerX/IISProject3Ugers
Training is done through the TrainingLoop.py and the evaluation is done through the HeatMapEvaluation.py.
What is it, that I have missed? How come, even after 14 million training steps, that the model still hasn't learnt to memorize the path in GridWorld?

I have two suggestions that you can look into. Based on my own work in RL, I believe the first one will require less work to implement.

If the observability of the environment is not an issue, then you could give the agent a relative measure (distance to the goal) as part of the observation to provide it with knowledge of how far away it is. You can also incorporate this as part of the reward function to put further emphasis on how minimising the distance to the goal is the key objective. In a GridWorld you would use the Manhattan Distance to get a relative distance.

You can apply curriculum learning to make the environment less ambiguous during early training. The principle of this strategy is to provide easier training examples which do not require many random actions to reach the goal. As the agents learn the complexity of the environment is increased and it can transfer knowledge. In practice, this can be done in two ways:
2.1. You start with a small environment say 3x3 or 5x5 and increase the size after the success rate reaches some satisfying number.
2.2. When resetting the environment you spawn the agent in close proximity to the goal and increase the distance after the success rate reaches some satisfying number.

The second method is inspired by this paper. I can also highly recommend this blog post from Lilian Weng if you would like to learn more about curriculum learning.


text:
Test accuracy go down after decreasing learning rate
My project include classification of images into several classes.
I'm having a strange issue related to adding mixup augmentation. The accuracy of the training set and the validation set keep rising during all training.
However, the accuracy on the test set improves significantly until epoch 80 (over baseline without mixup - grey), but when the learning rate is reduced it collapses. Note that each 40 epochs I reduce the learning rate.
Is it learning rate/optimizer/scheduler problem? If so..what should I do?
I already tried cosine schedulers and linear schedulers.

what dataset are you using/how large is it? Could be simply over fitting from these graphs.

As you can see these are validation and test graph not training graphs. Validation keep increases. Also, the grey in the test set is also increasing. Only the green (with mixup) is decreasing. The question is how to solve it

Your still optimizing on your validation set, so that doesn't prove it's not over fitting. And as you can see by decreasing learning rate every 40 epochs you keep fine tuning on the same batches again and again, improving slightly (cause after a while the weights basically stop changing) without any benefit on the test set, which remain slightly the same (if you smooth the scores with moving average it's most likely flat, not decreasing). Plus you have a 10 to 20% difference between the 2 sets scores, which is another strong indicator for over fitting.

You are right, in the grey graph the validation is basically flat because lr is low. if i dont change it varies again. But also in low lr, it improves a tiny bit. You think about early stopping? And the differences between the sets is becuase they are from different distributions.

well if your train, test and validation sets are different that's the very first thing to address. They are suppose to be representative of the same distribution, if not the test results are useless and not even to be interpret at all (otherwise it's like comparing apples and bananas and wondering why they don't look the same).


text:
Problem with the Turing Test as Performed
Could anyone explain this problem I have with the Turing test as performed? Turing (1950) in describing the test says the computer takes the part of the man then plays the game as when played between a man and a woman. In the game, the man and the woman communicate with the hidden judge by text alone (Turing recommends using teleprinters).
If the computer takes the part of the man, then it will have an eye and a finger in order to use the teleprinter as the man would have done. But in the TT as performed, the machine is not robotic. It has no eyes and no fingers but rather is wired directly into the judge's terminal. The only thing the machine gets from the judge is what flows down the wire. But the problem I have is, what flows down the wire is not text. The human contestant gets the text. The judge's questions print out on the teleprinter paper roll. The man sees the shapes of the text, and understands the meanings of the shapes. But the computer is never exposed to the shapes of the questions, so how could it possibly know what they mean?
I've never seen anyone raise this problem, so I'm very confused. How could the machine possibly know the judge's questions if it is never exposed to the shapes of the text?

Rather than writing "Problem with the Turing Test as Performed", can you just put your specific question in the title? Thanks.

Regarding: "the computer is never exposed to the shapes of the questions, so how could it possibly know what they mean?" could you please clarify. OCR software doesn't "understand" the characters it reads. It may have a language model, similar to the AI's one, to help it disambiguate e.g. S from 5, but it doesn't have any representation of what objects it may convert text for actually are. This is a difficult problem in all AI to date, but it is not clear whether you are asking about this. Your description of a physical typing robot would have the same issue.

@Neil Slater, so computers internally manipulate instances of binary difference. Text has meaning because a community has assigned an interpretation to the shapes of the text. But no one has assigned meanings to the shapes of the (individual or groups of) binary difference computers internally manipulate (semiconductor switch states and clocked groups of electrons). So what the machine gets from the judge's keyboard and internally manipulate have no meaning. The machine could never understand the judge's questions. But the TT is a test of whether the contestant understands the questions.

@Neil Slater (cont), so for the robot contestant, it "sees" the judge's text then types answers on a keyboard, as a human does. So what travels from the robot "eyes" and to the robot fingers, is electrical pulses (the shapes of which have not been assigned meanings and hence are not text). But then the shapes of the electrical pulses which travel along the optic nerve to the organic brain haven't been assigned meanings either. Yet we understand things. So the key is understanding how knowledge is acquired then accessed, when the things that propagate from the senses have no meanings.

@Neil Slater (cont2), so for OCR, I think that what happens is a direct prescription of human intelligence. So it's the old Ada Lovelace objection that computers (as per Babbage) can only do what they are designed to do (Note G). The human specifies the language model. I think the thing here is to understand how the machine could learn a language without having a pre-existing language model, and perhaps even without having a universal grammar.

(As @nbro writes, your question is not very specific; I'm answering here how I understand it from the current version)
In an ideal world, a computer would see written text (via a camera), scan it,  understand it, and type a response. I assume Turing didn't go for voice transmission, as voice includes other clues to a person's gender.
However, AI is such a complex field that it would have been impractical to implement this until fairly recently. And OCR and robotic movements (typing on a keyboard) are arguably not that relevant to human cognition, so in most actually run Turing-like tests shortcuts are taken.
Update: Also, note that the original Turing test (1950) was based on a party game about distinguishing between a man and a woman (who were not visible). This imitation game was later generalised to a guessing game between a human and a machine.

It's not true that the original Turing test was just about differentiating between a man and woman. It's true that this is how he initially introduces what he calls the imitation game, though. Turing actually writes "We now ask the question, ‘What will happen when a machine takes the part of A in this game?’ Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, ‘Can machines think?’". So, the 1950 paper was really investigating the question: "Can machines think?".

@nbro Thanks for pointing this out; it's been a while since I read that... I clarified this in my answer.

@nbro, Thanks. Yes, basically. But Turing did reject the idea of thinking as too meaningless to deserve discussion. But as a test of (human-like conversational) intelligence, what is the test for, exactly, and what is the evidence which determines pass or fail? The TT tests whether the contestant understand the meanings of the judge's questions. These questions, being text, are made of shapes. The evidence of understanding is that the shapes of the contestant's text answers, as interpreted by the judge, seem human-like. The TT is a test of understanding of the meanings of shapes.

@nbro (cont), the computer (as electronics engineers know) doesn't receive and/or manipulate text. So if the only thing the machine gets from the judge is electronic pluses from the judge's keyboard, then the machine doesn't get the text of the judge's questions. It has no shapes in respect of which to interpret meanings. Even if the machine could interpret the meanings of text, it could never know the judge's questions because it never gets the judge's text. Since the TT tests for understanding of the judge's text, the test as performed can't possibly yield a result.

@Oliver Mason, Comments I've made above might answer some of your comments. But I think the TT isn't really a guessing game. The judge must rely on evidence. The problem I'm trying to raise is that once this evidence is clearly understood - from the correct perspective of interpretation of shape - fairly severe problems arise. It seems the semantic aspects of the TT as performed have been largely ignored. But the really key issue is understanding how a computer could think. Imagine the huge help true AGI would be in mitigating climate change.

There is no form of OCR that assigns "meaning" by processing visual input of letters and words into the computer representation of those same words (e.g. ASCII). The robot with a camera and keyboard does not solve the problem you have raised. You need to look elsewhere for answers, and the state of AI today is that no-one has strong evidence for how meaning arises within an intelligent system. There is plenty of writing on the subject though.
I think you are trying to understand how and where meaning may arise in any system (biological or machine). There is lots of thought around this subject in AI philosophy and research. A good place to start might be with John Searle's Chinese Room argument, which broadly agrees with you that a basic discussion/chatbot program does not prove intelligence, but for different reasons than the "shapes of the text", which is not really an issue at all in my opinion. Searle's argument is by no means the end of the matter, and plenty has been written in rebuttal and support of the argument.
The real issue is the symbol grounding problem, which also applies to visuals of text, or any system of referring one entity by using another entirely different one.
Potentially addressing the grounding problem, there are various philosophies and engineering ideas proposed. These include:

Behaviouralism. It is not important what goes on inside an intelligent system, only that its external measurable behaviours are those of an intelligent system. This matches quite closely to the idea of the Turing Test, but many people find this unsatisfying due to personal experience of self-awareness, subjective experience and consciousness. It is in some ways the "shut up and calculate" of AI.

Embodiment and multi-modal experience. If an agent can experience the world directly and associate symbols with relevant experiences (the word "cat" with seeing and hearing cats), then it would be intelligent in the same way as we are.

Missing components. Humans (and sometimes animals) possess some additional system that cannot be replicated by current computing and robotic devices, even if they were made 1000s of times more powerful. The missing component might be something quantum in our cells (Penrose, The Emperor's New Mind) or "the soul". This is also a common depiction of robots and AI in science fiction, and there is lots of popular support for it as a philosophy, despite weak evidence.

Complexity and power. We can currently replicate the mental power of small insects on computing devices. When we scale up with more powerful computers, larger neural networks, and perhaps a bit of special extra structure (that we don't know yet), then we will hit a level of complexity where true intelligence will emerge. You could view recent very large language models such as GPT-3 as exploring this idea.

But ASCII isn't a computer representation of text words, but rather a standard table printed on sheets of paper which associates mainly letters of the English alphabet with what are called 8-bit strings. So the symbol  "A" is associated with the symbol sequence "01000001". But there are no 0s and 1s manipulated inside computers, so "01000001" can't be a computer representation of "A". I know everyone says computers process zeros and ones, but this simply isn't true. It gives the false impression that computers process symbols when actually they process electrical pulses and switch states.

I agree a robot contestant merely having eyes and fingers doesn't solve the problem of how a system understands the meanings of text. But if a computer will pass the Turing Test, it will have eyes and fingers. So being robotic to that extent is just a precondition. I agree there's no consensus of what intelligence amounts to. But I think we can agree that intelligent systems contain knowledge. So how does the knowledge get inside the system? Through sensory apparatus. So sensory streams contain knowledge in some form. What is the form? Working this out seems a sensible place to start.

CRA. I think basically the CRA shows that no system which manipulates symbols and does nothing else could possibly understand the meanings of what it manipulates (Searle, NY Review of Books, 4 October 2014). He argues that computers are such systems. Hence, they can't possibly understand what they manipulate because, in themselves, those things have no meanings. Searle always comes back to this point. I know there are objections to the CRA, but I've never seen an objection to Searle's fundamental premise that symbols in themselves are meaningless. (But do computers manipulate symbols?)

Symbol Grounding. I distantly recall Harnad's Symbol Grounding Problem in response to which he proposes (something like) analogue processing lower down and symbolic processing higher up. Of course, I've done only only half this - proposed a problem but not offered a solution. The main problem I see with behaviorism is that in effect it's a set of conditionals of the form: If the input = A then the output = B. And B is the response to A not because the machine has applied any intelligence. Rather, the response is a function purely of the programmer's intelligence.

Embodiment and multi-modal experience. You say "If an agent can experience the world directly and associate symbols with relevant experiences (the word "cat" with seeing and hearing cats), then it would be intelligent in the same way as we are". However, we don't associate symbols with experiences. We experience a cat and we experience a symbol. We see both. Both are external. There's no symbol "cat" inside the brain to be associated with an experience of a cat. Similarly, there are no symbols inside computers.

Components of human intelligence that cannot be reproduced in computers (eg soul). Presumably a computer with human-like intelligence could get religion then embrace the belief that it has a soul. But what if there are such things as (immaterial, eternal) souls? But I don't see why an omnipotent omniscient god couldn't create a soul with the owner being a robot, just as a god creates a soul with a human embryo being the owner. For quantum aspects, quantum seems just a current speculation.

Emergent intelligence. All we need is more computer power, processors, memory, etc. This idea has been used since the 1950s to explain failure to achieve milestones. I think if the idea were true, we would have seen successful incremental steps towards AGI. Milestones would have been reached. But the present systems are far removed form anything like human intelligence. The latest approach, Deep learning, has the sever problems of edge cases, catastrophic forgetting, and adversarial attack, which are debilitating.

@Roddus: This is not a discussion forum, and I cannot engage in debate or discussion with you in the comments. If my answer addressed your question as written, please accept it. If it did not, please explain what is missing or what you do not understand.

Thanks for explaining the purpose of ai.se Could you suggest a debate or discussion forum? For ai.se, now I can see the problem I've been having. I thought ai.se was a forum where comments or answers could be challenged. But it's really a forum where the established wisdom is amplified (by the like-minded voting up questions) and any dissent is punished (by voting down questions). I was looking to engage with free thinkers who are prepared to justify their positions in the dialectical style where one can learn through an interchange of ideas. I guess that's just not ai.se.

@Roddus the way to engage with discussion here on contentious subjects would be to write your own answers. Votes may fall in line with current consensus on the subject, but if the site is working well, then differing answers on more philosophical subjects may be equally upvoted. It's not a space for debate though. Sorry I don't know of any and the few I have found in the past for AI have been full of cranks, too much noise for me.