text:
What are the mathematical prerequisites to be able to study artificial general intelligence?
What are the mathematical prerequisites to be able to study artificial general intelligence (AGI) or strong AI?

Most of the answers are oriented towards statistical/probabilistic models. For more 'classic' AI I would say you would need some knowledge of predicate calculus. This is the more symbolic planning approach to AI problem solving.
You could argue it's a bit 'old school', but still relevant for certain aspects of AI.

I always recommend starting with game theory, combinatorial game theory, and algorithmic combinatorial game theory, (but I'm potentially biased;)
Combinatorics is a given--discrete mathematics is heavily utilized in computer science, and, with the advent of Combinatorial Game Theory (CGT), ability to determine if a given choice can be deemed optimal ("perfect play").  CGT arises out of traditional Game Theory, which we sometimes term "economic game theory" to make the distinction.  Out of Game Theory also arises subfields such as Evolutionary Game Theory, which is important in AI.
These fields relate to rationality, which is the basis for optimized decision making. Decision making algorithms seems to be the fundamental distinction of what constitutes an Artificial Intelligence.
From minimax to gametrees, it's probably a good idea to have a basic grounding in these fields, even if the problem you're AI is trying to solve isn't formally defined as a game.
All problems, from a fundamental standpoint, can be regarded either as puzzles--non-competitive context--or games--competitive context.  This distinction is based on whether there is a single agent (puzzles) or multiple agents (games.)

Before proceeding and answering the actual question, it's worth noting that AI and AGI are not the same thing, as was the case at the beginning in 1956, as suggested in the official proposal for the Dartmouth workshop.
Nowadays, people that consider themselves "AI researchers" or "AI practitioners" (e.g. myself) typically are not trying to directly build an AGI, but are focusing on a specific AI approach, such as reinforcement learning, which could, one day, be used to build an AGI. The reason is that we have noticed that directly tackling the "AGI problem" (i.e. creating an AGI) is a lot more complex than was originally thought and some do not think that this is even possible. AGI is a sub-branch of AI that studies how to create an AGI (or human-like AI). Only a few people are still working on AGI.
Ben Goertzel, who's one of the people that is still interested in and attempting to directly create an AGI, wrote a blog post about this topic: AGI Curriculum. If he had to design a curriculum, it would be divided into 6 courses

History of AI
AI Algorithms, Structures and Methods
Neuroscience & Cognitive Psychology
Philosophy of Mind
AGI Theories & Architectures
Future of AGI

He then suggests multiple readings (books) for each of these courses/topics. Below, I will list one book for each of the courses (also based on their free availability online as pdfs). You can find more books in the blog post.

The book What Computers Still Can't Do (1992) by Hubert Dreyfus
The book Artificial Intelligence A Modern Approach (AIMA) by Russell and Norvig, but Goertzel notes that this is not an AGI book, but gives an introduction to multiple AI topics that have been used in many cognitive architectures for AGI
The book Neuroscience: Exploring the Brain by Bear, Connors and Paradiso
The book Being No One: The Self-model Theory of Subjectivity (2003) by Thomas Metzinger
The paper Artificial General Intelligence: Concept, State of the Art, and Future Prospects (2014) by Ben Goertzel
The book Singularity is Near (2005) by Kurzweil

So, to conclude, if you want to study artificial general intelligence, it's not sufficient to just read the typical machine learning or deep learning books, but you also need to have a more solid understanding of other aspects of artificial intelligence and even neuroscience in order to study and do research on AGI. Moreover, it's probably a good idea to also have a good background in all the traditional approaches, what they can or not do, the history of AI (why some approaches have failed or not), and understand the philosophical problems and, last but of course not least, read about the current approaches to AGI, such as universalist (e.g. AIXI) or symbolic ones (all the cognitive architectures such as OpenCog).
To answer your question more directly, if you can read and understand the AIMA book, then you probably have if not all most of the mathematical prerequisites, which will probably include

logic
discrete mathematics
calculus
optimization
linear algebra
probability theory
theory of computation (this will definitely be needed if e.g. you want to learn about AIXI, but you will also need a nice dose of measure theory and algorithmic information theory to understand all the mathematical details of the theory)

Note that, although these subjects (logic, probability theory, or theory of computation) are necessary to understand the current approaches to AGI, they may not be sufficient to develop a full AGI, but this is a different story. Moreover, note that these mathematical subjects are not just required to understand the current approaches to AGI, but they would also be useful to understand any other AI sub-branch, such as machine learning (and that's probably why people may think that this answer is misleading, but it's not: if you have ever tried to learn something about AIXI, you will know that all the subjects above are more than required!)
In the future, if you also want to do serious research on AGI, having a degree in Computer Science, Cognitive Science, Neuroscience, Mathematics, and/or, of course, Artificial Intelligence, may be a good thing. By the way, Ben Goertzel has a Ph.D. in math. Marcus Hutter, the inventor of AIXI, did his bachelor's and master's in CS with minors in mathematics, and one Ph.D. in theoretical particle physics and another Ph.D. in CS during basically the time that he developed AIXI.


text:
Are leaf nodes included in the calculation of average branching factor for search trees?
In the search tree below, there are 11 nodes, 5 of which are leaves. There are 10 branches.

Is the average branching factor given by 10/6, or 10/11?
Are leaves included in the calculation? Intuitively, I would think not, since we are interested in nodes with branches. However, a definition given to me by my professor was "The average number of branches of all nodes in the tree", which would imply leaves are included.

I'd say that the leaves per se count, too, but only if they're real leaves, like e.g., checkmate positions in chess.
Such a node has really no children and no further calculation is needed. Unlike with nodes which weren't expanded yet.
Note that always counting the leaves provably leads to (n-1)/n for every n-node thee!

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

sort: votes


text:
What are good learning strategies for Deep Q-Network with opponents?
I am trying to find out what are some good learning strategies for Deep Q-Network with opponents. Let's consider the well-known game Tic-Tac-Toe as an example:

How should an opponent be implemented to get good and fast improvements?
Is it better to play against a random player or a perfect player or should the opponent be a DQN player as well?

In a two player zero-sum game (if I win, you lose and vice-versa), then you can have a simple and efficient solution learning from self-play.

How should an opponent be implemented to get good and fast improvements?

You don't need to think in terms of agent vs opponent, instead think in terms of coding both the players' goals into a single Q function. Score +1 if player A wins, -1 if player B wins, and zero for a draw. It is then player A's goal to maximise the score and player B's goal to minimise the score.
You can then implement and learn both player strategies in the same self-play learning session and same Q function, using minimax. In practice that means where in Q learning you generally pick the maximising action on next state to bootstrap Q values, in a minimax variant, you pick maximising or minimising action depending on whose turn it is. Otherwise the Q learning algorithm is the same as normal. I have implemented this, but not for DQN, just for tabular Q-learning - feel free to learn from, copy and/or re-use any part of that code.

Is it reasonable to play against a random player, a perfect player or should the opponent be a DQN player as well?

The Q learner will learn to optimise against whichever player you make it play against. Against a random player, it will not necessarily learn to play well, just well enough to defeat the randomness. It may even make deliberate mistakes - such as not blocking a winning line - knowing it has a better chance to win due to random opponent.
Against a perfect player is possible with tic tac toe (because you can construct such a player), although there might be gaps in the trained Q values - game states never seen - which mean that an imperfect opponent could actually defeat the trained agent! In practice this does not scale to more complex unsolved games, because no perfect players exist.
Another DQN player should work fine. You would end up with two agents, each specialising in playing one player's turns. This is less efficient than a single minimax-based player, but no expected problems. It may be a preferred choice for some games, especially if they are not zero-sum.

Thank you very much for your fast and detailed answer. I will give a try to those hints.

Richard Garfield defines luck as "uncertainty in outcomes".  Athough I don't necessarily love this definition for the term (speaking as a deterministic game designer) his point is well taken when he applies this to non-trivial, non-chance, perfect information games such as Chess.  (Essentially it's an extension of the "infinite monkey theorem" applied to unsolved games.  Because perfect play cannot be validated, there is always a chance, however minuscule, of being beaten by an opponent making purely random choices.)

@Neil Slater and do you have some good advices if there are more than two players or even a player in your team?

@murthy10: I am less sure about multi-player scenarios, and do not have much advice. If the team objective is to win as a unit, and players can share information freely, then it could make sense to model the team as an agent (as opposed to each player separately). However, if you want to code a game agent that could play as a team member alongside a human player, that would not work.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What are some tactics for recognizing artificially made media?
With the growing ability to cheaply create fake pictures, fake soundbites, and fake video there becomes an increasing problem with recognizing what is real and what isn't. Even now we see a number of examples of applications that create fake media for little cost (see Deepfake, FaceApp, etc.).
Obviously, if these applications are used in the wrong way they could be used to tarnish another person's image. Deepfake could be used to make a person look unfaithful to their partner. Another application could be used to make it seem like a politician said something controversial.
What are some techniques that can be used to recognize and protect against artificially made media?

Digital Media Forensics (DMF) field aims to develop technologies for the automated assessment of the integrity of an image or video, so DMF is the field you are looking for. There are several approaches in DMF: for example, those based on machine learning (ML) techniques, in particular, convolutional neural networks (CNNs).
For example, in the paper Deepfake Video Detection Using Recurrent Neural Networks (2018), David Güera and Edward J. Delp propose a two-stage analysis composed of a CNN to extract features at the frame level followed by a temporally-aware RNN to capture temporal inconsistencies between frames introduced by the deepfake tool. More specifically, they use a convolutional LSTM architecture (CNN combined with an LSTM), which is trained end-to-end, so that the CNN learns the features in the videos, which are passed to the RNN, which attempts to predict the likelihood of those features belonging to a fake video or not. Section 3 explains the creation of deepfake videos, which leads to inconsistencies between video frames (which are exploited in the proposed method) because of the use of images with different viewing and illumination conditions.
Other similar works have been proposed. See this curated list https://github.com/aerophile/awesome-deepfakes for more related papers.

The techniques you mention use GANs. The key idea of GANs is that you have a generator and a discriminator. The generator generates new content, the discriminator has to tell if the content is from the real data or if it was generated.
The discriminator is way more powerful. It should not be too hard to train a discriminator to detect fakes. Training a model which is able to pinpoint the manipulation and understanding of this is a proof of manipulation is harder. It is impossible to get a proof that something is not manipulated.
About the question how you deal with photoshopped images: you look at differences in compression levels in the image. The keyword to look for is image forensics: http://fotoforensics.com/tutorial-estq.php

I think context is important here. Using tactics like those used by Scotland Yard for over a century is probably the best way. Establishing alibis, realistic time lines, motives. For a legal setting, it would be possible to prove these images were fake using methods like this. From an I.T. perspective, it may be possible to pinpoint an origin for these images. If thousands of duplicitous images came from a single origin, then any images from this origin are suspect.
I think, in general, we should retrain ourselves to not believe everything we see. There are so many methods for faking images, that photography can no longer be considered to be the best evidence of an event occurring. We should not ignore all images, but instead seek outside concurrence of facts before jumping to conclusions. If all facts point to an event happening, then that photograph is likely to be real.

Assuming artifacts and unnatural elements do not exist in the media in question and that the media is indistinguishable to the human eye, the only way to be able to do this is to trace back to the source of the images.
An analogy can be drawn to DoS (Denial of Service) attack, where an absurd number of requests are sent from a single IP to a single server causing it to crash - A common solution is a honeypot, where a high number of requests from one IP is redirected to a decoy server where, even if it crashes, uptime is not compromised. Some research has been done on these lines where this paper spoke about verifying the digital signature of an image or this one where they proposed tampered image detection and source camera identification.
Once traced back to a source, if an absurd number of potentially fake images come from a singular source, it is to be questioned.
The common fear arises when we are dealing with something, on the basis of the analogy, like a DDoS (Distributed Denial of Service) attack where each fake request comes from a distributed source - Network Security has found ways to deal with this, but security and fraud detection in the terms of AI just isn't that established.
Essentially for a well thought out artificial media for a specific malicious purpose, today, is quite hard to be caught - But work is being done currently on security in AI. If you're planning on using artificial media for malicious purposes, I'd say now is the best time probably.
This security has been a concern from a bit now. An article written by a data scientist quotes

Deepfakes have already been used to try to harass and humiliate women through fake porn videos. The term actually comes from the username of a Reddit user who was creating these videos by building generative adversarial networks (GANs) using TensorFlow. Now, intelligence officials are talking about the possibility of Vladimir Putin using fake videos to influence the 2020 presidential elections. More research is being done on deepfakes as a threat to democracy and national security, as well as how to detect them.

Note - I'm quite clueless about network security, all my knowledge comes from one conversation with a friend, and thought this would be a good analogy to use here. Forgive any errors in the analogy and please correct if possible!

It would be nice if you could do some research and provide a link to at least 1 research work/paper which is based on something along those lines (that is, that exploits the source of the potentially fake videos).

Apart from papers speaking about the potential harms, and the ones commonly trying to detect artifacts, fewer papers doing whats stated in the answer such as this one or this one - As said, extensive research has not been done on these lines, but it is being explored. Hope these links helped!


text:
What types of applications qualify as "compound intelligences"?
A question about swarm intelligence as a potential method of strong general AI came up recently, and yielded some useful answers and clarifications regarding the nature of swarm intelligence. But it got me thinking about "group intelligence" in general.
Here organism is synonymous with algorithm, so a complex organism is an algorithm made up of component algorithms, based on a set of instructions in the form of a string.
Now consider the Portuguese man o' war, not a single animal, but a colonial organism. In this case, that means a set of animals connected for mutual benefit.
And physalia physalis are pretty smart as a species in that they've been around for a while, I'm not finding them on any endangered lists, and based on their habitat it looks like global warming will be a jackpot for them. And they don't even have brains.
Each component of the physalia has a narrow function, colony organism itself has a more generalized function, which is the set of functions necessary for maintenance and reproduction.
{Man o' War} ⊇ { {pneumatophore}, {gonophores, siphosomal nectophores,vestigial siphosomal nectophores}, {free gastrozooids, tentacled gastrozooids, gonozooids, gonopalpons}, {dactylozooids}, {gonozooids}, {gastrozooids} }

What types of applications qualify as "compound intelligences"? What is the thinking on groups of neural networks comprising generally stronger or simply more generalized intelligence?

I recognize the underlying problem is ultimately complexity and that "strong narrow AI" is, by definition, limited, so I use "generalized" and omit "strong" because human-like and superintelligence are not conditions.  Compound intelligence is defined as a colony of dependent intelligences.*
Utility software is often a form of an expert system that manages a set of functions of varying degrees of complexity. There's currently a great deal of focus on autonomous vehicles, which would seem to require sets of functions.
Links to research papers on this or related subjects would be ideal.

Portuguese Man o' War (oceana.org)
The Bugs Of The World Could Squish Us All

I don't understand the point of this post and I think it can be simplified a lot. You mention swarm intelligence and there are also multi-agent systems. So, what exactly are you looking for? Your main question is "What types of applications qualify as "compound intelligences"?" and then you define "compound intelligences" as "a colony of dependent intelligences" (whatever a "colony" means, i.e. maybe you should use a more general term). So, are you looking for something like vehicle routing problem solved by swarm intelligence algorithms?

I did work on compound intelligence because that is the direction that Google is trying to go. I couldn't find any basis for it. In other words, having a collection of AI expert systems does not seem to provide any collective intelligence. You would also need some kind of control program that could decide which system to use. Currently, Google relies on the user to choose. If Google was able to create an independent control program it would already be out. This does not seem to be a matter of complexity or code tweeking but a fundamental limit with AI.

Great info.  Thanks for posting!  PS- is there a formal term for this concept?  (in some sense, I was more thinking about integrating neural networks as components in expert systems, but also tried to make the question fairly general.)

I don't know if there is a formal term; I called it the Swiss Army knife strategy. It is possible that neural networks could be components in a future cognitive system but you can't build one using NN alone.

Yes, I did look at social insects. There is no actual intelligence there. I wrote a paper on information storage in non-replicating and replicating systems (like DNA). Viruses do not exhibit any intelligence as a group. You seem to be confusing intelligence with adaptive or shotgun environmental reactance.

Swarm intelligence, compound intelligence, or group intelligence may emerge as an important concept as AI develops toward higher complexity.  Whether these terms should be considered synonymous is doubtful.
Compound features in biology are the result of control in differentiation during the development of an organism from a single cell.  Compounding in biology is a single function performed across like elements.
Swarms are a result of distinct organisms operating in proximity.  Swarming in biology is similarity in complex independent behaviors that appear coordinated but are generally engaged in as a defense from predators.
Group intelligence may be distinct from compound intelligence in that adjacent units may be in agreement or opposition, as can be case with intelligent beings in groups.  The agreement in the group that a variety of models is permissible.  We call them opinions, but they are distinct matches of models to problems that produce different projections and suggest different selections from among the group's options.
The terms general and strong appear in this question and many others and it appears that, mainly for historical reasons, may continue to frustrate clarity.
All intelligence is general in that it learns a generality from specific experiences and then applies that generality to future scenarios to achieve objectives within those scenarios.  What makes the application of the generality intelligent is that it is expected to work because it has been working for similar scenarios.
All intelligence is specific in that it is limited to scope of what generalities have been discovered.  Certain techniques are significantly more general that others because they are allegedly domain independent.  We call this mathematics.
Consequently, the development of artificial intelligence is not a path from specific to general but one of discovering generalities and applying them to specifics.  One could say that the most general thinking of mathematics applied to the computer is the primary activity of applied artificial intelligence, and the capabilities that emerge are a more sophisticated as more sophisticated mathematics is represented in working software and hardware.
The generalities take on greater complexity so that they can apply to a greater number of specific scenarios.  That is not a gain in strength but a gain in the breadth of potential application.
Physalia physalis is a symbiotic colony of organisms of four types, the pneumatophore (or float), dactylozooids (long tentacles), gastrozooids (feeding tentacles), and gonozooids which produce reproductive gametes.  To avoid breaking the historic conception of animals, these organisms are called polyps.  The venom of the colony does not come from any of those organisms but rather from another symbiotic organism, cnidocytes, which attach to the tentacles and are released under strictly controlled conditions.
In addition to all this symbiosis, there are several species of fish that use the colony for shade and protection that the colony lets swim among the tentacles and that have formed partial immunity to the cnidocytes.  These complex symbiotic networks are not fully understood, but they seem to operate well and create sustainable inter-specie systems.
It's not likely that the organisms, within their lifetimes adapt or remember, but the DNA of each organism has involved in a way that resembles intelligence in that the colony and its symbionts have adapted to the ocean's surface and its biology.
The idea that the system of the biosphere is the first example of broad intelligence is likely, in that design excellence has emerged from an evolutionary processes.  It is not altogether ridiculous to propose that human intelligence is a higher speed approach in neurons to the slower speed of DNA replication in larger organisms.  Because of the metabolic requirements of growth slows evolution for larger organisms with a greater cell count, these larger organisms may have needed to develop a way to achieve the nimble adaptivity of their lower ancestors.
Neurology facilitates the approximation of some aspects of evolution and may have been the most attainable natural solution to reacquire nimble adaptivity.
Attempting to apply these various ideas to the current and ongoing development of autonomous vehicles reveals a gap in understanding.  We don't yet have the mathematics developed to understand how compound, swarm, or group intelligence can be used in the laboratory to accomplish well defined problems in controlled execution scenarios.  That is probably a prerequisite to using these ideas in vehicle control.
The system designs of future cars may be like a colony of independent or semi-independent components that each have a role and purpose.  Compound, group, and symbiotic designs are likely to develop.  The current activities of vehicles on the road or in the air near airports is like a swarm, so the application of that idea is obvious: Avoid collisions.

With the increase of both unit capacity turbines and size of wind farm, the safe operatio wind farm has received growing attention. In ma factors that affect the safe operation of wind far being struck by lightning is an important aspect. intelligent lightning monitoring system is used fo surveillance of wind turbine generators, which ca real-time and accurate monitoring of the lightnin current waveforms, amplitude, time of occurrenc number of lightning strokes and all the other imp parameters of lightning thus providing an effectiv monitoring and analysis tool to quickly locate fa location on wind turbine generator and cause of malfunction and a theoretical basis for the desig lightning protection system of wind turbines. Thi examines the principle and main methods of the turbine generator-matching lightning monitoring and combined with the specific research project designs and implements a high-precision and multifunctional intelligent lightning monitoring sy based on the theory of Rogowski coils.

I think you are referencing two different types of control systems in a symbiotic relationship?  If so, could you add a synopsis to clarify?


text:
What is the minimum number of neurons and hidden layers needed to learn a Boolean function that maps $N$ bits to $1$ bit?
Suppose I have a Boolean function that maps $N$ bits to $1$ bit. If I understand correctly, this function will have $2^{2^N}$ possible configurations of its truth table.
What is the minimum number of neurons and hidden layers I would need in order to have a multi-layer perceptron capable of learning all possible truth tables?
I found one set of lecture notes here that suggests that "checkerboard" shaped truth tables (which is something like an N-input XOR function) are hard to learn, but do they represent the worst-case scenario? The notes suggest that such tables require $3\cdot(N-1)$ neurons arranged in $2 \cdot\log_2(N$) hidden layers


text:
Can I train a model starting from another model trained on a subset of the dataset?
I want to create a neural network and train it on some data, however I want to be able to create a new model without retraining it from the start.
An example, I have 1000 data points in my training data

model - trained on 0-99
model - trained on 1-100
model - trained on 2-101
and so forth

So I'm wondering if I can use the first model to train the second model, essentially forgetting the first data point.
You can view it as a sliding window over the 1000 data points, sliding one data point to the right for each new model.
Does it make sense?
Is there any easy way to solve this problem?

As usually the main goal of training an NN is to generalise a function from example data, could you clarify what you mean by "forgetting the first data point"? Are you wanting to purposefully overfit so that predictions against the first data point are no better than random? If not, do you have any criteria for when the training for any specific model e.g. "model 2 trained on 1-100" is complete, or whether its results are acceptable to you? It is clearly possible to take model 1 and re-train it, but what is missing in the question is the goal for doing so . . . and that affects the answer here

The idea is that the first data point is not relevant any more. I dont want overfit, so imagine continuous addition of data points. For a given situation, only the 100 most recent data points are relevant. When a data point is added, the most recent model is trained on a data point that is (in theory) not relevant any more.

Adding to my comment, essentially Im asking if it is possible for a model to only consider the most recent data points, without retraining the model from the bottom on the most recent data - only partially traning it in some way, making it "forget" the no longer relevant data form the previous training set.

If you're afraid of overfitting, there are much better ways, like dropout, l2 regularisation and just stochastic batching.

Actually, your method looks a lot like cross validation (not the SE site, but the actual algorithm) that excludes some of the data for each iteration.

@norflow: Maybe you just want online training, but this is still not clear from your question or comments. How are you testing your network? How will its predictions be used? For instance, how will you know training for items 0-99 is complete and useful for its task? After you have trained it on items 0-99, what is the task you will use it for - predicting item 100?

Thank you so much for all of your time. I should clarify by describing the problem I want to solve instead of wasting your time, sorry.
I want to predict some future event. My hypothesis is that only the last 6 hours are relevant,so when time passes, the model will be trained with "old" data. The idea is that there will be some pattern present in the data that will help predict the next event.
So I need an adaptive model that "forgets" the old (non-relevant) data without retraining it from the bottom every time a time unit passes, because of real-time (every 5th min)

@norflow: It is still not clear from your comment. Are you saying items 0-99 are being used together to predict something about item 100? As you describe the training data, I am expecting each item (e.g. item 25) to be a separate valid input to the network, with a fixed training target. So input data about item 25 into the network, what is the output? Is the output associated only with item 25, and the rules for association changing over time? Or would it be more accurate to say the the output depends on items 0-24 as well, so this is more about predicting outcome of a sequence?

I think some simplified examples will help. Please make it clear how older items become less relevant. In a practical sense, there is a difference between a non-stationary problem (where the rules of induction keep changing) or a sequence prediction problem (where the history of the data has a predictable consequence and is part of those rules).

tl;dr Yes.
Question: Can I train starting with a trained network instead of from a newly initialized network.
Answer: You can, and should, save network weights when it is done training. You can also retain it in memory after writing a copy of it to disc.  If you then train the network some more, you can save it again to a different filename.  You can repeat this for the set of inputs.
At the end of the process you would have only one network in memory, you would have several files of "snapshots" of the weights, and each snapshot would be the result of training on a different parcel of the total data.


text:
What are some alternatives to the book "Artificial Intelligence: A Modern Approach"?
There are two textbooks that I most love and am most afraid of in the world: Introduction to Algorithms by Cormen et al. and Artificial Intelligence: A Modern Approach by Norvig et al. I have started the "AI: A Modern Approach" more than once, but the book is so dense and full of theory that I get discouraged after a couple of weeks and stop.
I am looking for a similar AI book but with an equal emphasis on theory and practice. Some examples of what I am looking for:

The Elements of Statistical Learning by Tibshirani et al. (detailed theory)
An Introduction to Statistical Learning: With Applications in R by
Tibshirani et al. (theory+practical)
Digital Image Processing by Gonzalez et al. (detailed theory)
Digital Image Processing Using MATLAB by Gonzalez et al.
(theory+practical)

As nbro mentioned in his answer, Russell and Norvig is relatively broad, because it is meant as an introduction to all of AI, rather than an introduction to specific subfields (such as statistical/machine learning, etc.).

Why is AIMA dense?
Artificial intelligence is a broad field: that's why Artificial Intelligence: A Modern Approach (AIMA) may look a bit dense to newcomers, given that it covers many different aspects of AI, such as search, machine learning, and natural language processing.

AI is not just ML!
The first book in this answer is a good book, but it focuses on computational intelligence approaches, which are often considered part of AI too. The other books that you mention in your post also focus on subfields of AI, such as machine learning or image processing, so they do not cover all aspects of AI.

Alternatives to AIMA

Title
Author(s)
Year
Topics
Comments

Artificial intelligence
Patrick Winston
1992 (3rd edition)
Search, rule-based systems, machine learning, evolutionary algorithms, etc.
I occasionally consulted this book; Patrick Winston was a professor at MIT and also director of the AI Lab at MIT; you can also find his free course on Artificial Intelligence (which I highly recommend) here.

Artificial Intelligence: A New Synthesis
Nils J. Nilsson
1998
Neural networks, evolutionary algorithms, search, knowledge representation and reasoning, planning, Bayesian networks, etc.
Nilsson also wrote other important books related to AI and the philosophy of AI, such as The Quest for Artificial Intelligence: A History of Ideas and Achievements (2009), among other important contributions to the AI field, such as the robot Shakey and STRIPS.

Artificial Intelligence: A Guide to Intelligent Systems
Michael Negnevitsky
2005 (2nd edition)
Expert systems, fuzzy systems, artificial neural networks, evolutionary computation, hybrid systems, etc.

Artificial Intelligence: Structures and Strategies for Complex Problem Solving
George Lugar
2009 (6th edition)
History of AI, search, knowledge representation, expert systems, rule-based systems, machine learning, evolutionary algorithms, automated reasoning, natural language understanding, etc.
This book is mentioned by Ben Goertzel in his paper Artificial General Intelligence: Concept, State of the Art, and Future Prospects as a "popular AI textbook".

Artificial Intelligence: Foundations of Computational Agents
David L. Poole, Alan K. Mackworth
2017 (2nd edition)
Machine learning, search, planning, reasoning, and knowledge-based systems, etc.
I occasionally consulted this book.

The following 2 books helped me understand the basics and guided me through my first AI / CI implementations.

Computational Intelligence: An Introduction by Andries P. Engelbrecht
It includes the most relevant developments in computational intelligence with good discussions on intelligence and artificial intelligence (AI).

Computational Intelligence: Concepts to Implementation by Russell C. Eberhart and Yuhui Shi
It is a comprehensive textbook, supported with lots of practical examples.


text:
What is the difference between an observation and a state in reinforcement learning?
I'm studying reinforcement learning. It seems that "state" and "observation" mean exactly the same thing. They both capture the current state of the game.
Is there a difference between the two terms? Is the observation maybe the state after the action has been taken?

In the paper Imagination-Augmented Agents for Deep Reinforcement Learning, the authors use the term "observation" instead of "state" on purpose. So, you may want to have a look at it to understand why. Have also a look at the Wikipedia article on POMDPs, which is a mathematical framework that generalizes MDPs, where observations and states are clearly differentiated.

Sometimes observation and state overlap completely, which is convenient. However, there is no reason to expect it in all cases, and that's where interesting problems occur.
Reinforcement learning theory is based on Markov Decision Processes. This leads to a formal definition of state. Most importantly, the state must have the Markov property. Which means that for RL to work according to theory, that knowing the state means that you know everything knowable that could determine the response of the environment to a specific action. Everything that remains must be purely stochastic and unknowable in principle until after the action is resolved.
Systems like deterministic or probability-driven games, and computer-controlled simulations can be designed to have easily observable states that have this property. Games with this trait are often called "games of perfect information", although you may have unknown information, provided it is revealed in a purely stochastic manner.
In practice, real world interactions contain far too much detail for any observation to be a true state with the Markov property. For instance, consider the inverted pendulum environment, a classic RL toy problem. A real inverted pendulum would behave differently depending on its temperature, which could vary along its length. The joint and actuators might be sticky. Rotations and movement will alter temperature and friction, etc. However, a RL agent will typically only consider current motion and position of the trolley and pendulum. In this case, the observation of 4 traits is usually good enough, and a state based on this almost has the Markov property.
There are also problems where observations are not enough to make usable state data for a RL system. The Deep Mind Atari DQN paper had examples of a couple of these. The first example is that a single frame lost data about motion. This could be addressed by taking four consecutive frames and combining them to make a single state. It could be argued that each frame is an observation, and that four observations had to be combined in order to construct a more useful state (although this could be put aside as just semantics).
The second example in Atari DQN is that the pixel observations did not include data that the game was tracking but that was not visible on screen. Games with large scrolling maps are a weakness of the Atari-playing DQN, because its state has no memory of screens other than the four used for movement. An example of such a game, where Deep Mind's player did much worse than a human player is Montezuma's Revenge, where to progress it is necessary to remember some off-screen locations.
There are ways to address knowledge that there is unobserved but relevant state in a problem. The general framework for describing the problem is Partially Observable Markov Decision Processes (POMDPs). Workable solutions include adding explicit memory or "belief state" to the state representation, or using a system such as RNN in order to internalise the learning of a state representation driven by a sequence of observations.

Thanks for the response! So an observation is a subset from the state?

@echo: Not necessarily. You could also observe things that are irrelevant to the state. So it is more accurate to say that state and observation may overlap. Generally, when they overlap a lot, RL is easier to apply, so you try and make this happen - most toy problems in RL are designed so that observation and state overlap perfectly, for convenience. But sometimes interesting problems require you to deal with missing/unobserved state data.

There is a subtle but important difference between observation and state.
The observation is the information that the agent is gathering from the environment. This could be data coming from sensors. It could be noisy, or contain redundant information. It could also be incomplete and not contain enough to capture all the information needed to build the state. (In this case the agent might be able to use what it already knows about the current state and the new observation data to build the new state).
The state is the information that describes all the relevant aspects about the environment that the agent's policy needs to make a decision. We also like to make a distinction between the environment state which could be huge and impossible to capture fully (especially in scenarios where the agent is interacting with the physical world) and the agent state which is the distilled version that only captures the important information the agent needs to make a decision.
In simple settings, the observation could be equivalent to the state. In a fully-observable board game (e.g. checkers) the position of the pieces is the observation and also the state. In more complex scenarios this is not the case.
When using Deep RL or most of the modern RL variants, the state is often not explicitly encoded. It is a feature of these modern algorithms to use function approximation to bridge the observation inputs directly into the policy or value function. This is especially useful if the number of unique states is very huge, and you also want your policy to be robust enough to behave correctly even for unseen states that are similar to ones (but not exactly the same) it was trained on.
This is why in frameworks such as Gymnasium you see observations and not states, and probably one of the sources of this confusion between both.


text:
What are the minimum computing resources needed to train a machine learning algorithm?
For a school project, I would like to investigate a paper on either reinforcement learning or computer vision. I am particularly interested in DQN, RNNs, CNNs or LSTMs. I would eventually like to implement any of these. However, I also need to take into account the computing resources required to train and analyse any of these algorithms. I understand that, in computer vision, the data sets can be quite large, but I am not so sure regarding the resources needed to implement and train a typical state-of-the-art RL algorithm (like DQN).
Would a "standard PC" be able to run any of these algorithms decently to achieve some sort of analysis/results?

Short answer: yes, state of the art with many many layers probably not but enough to tinker and learn and make a project.

How do you plan to implement it? using some language from scratch, using python and some of the deep learning packages like Tensorflow? I am not an expert at all, but my standard PC seems not good enough to install and run Tensroflow, for instance.

I believe I will be using Python along with a few packages. However, as it's my first time trying these algorithms I have no advice to give you. After trying it out I'll try and remember to get back to you.

It might be hard to implement deep reinforcement learning algorithms, especially considering your previous experience and the computing resources you have. They require almost the same (even more) GPU power. Deep reinforcement learning algorithms use deep neural networks for learning the optimal policy. Even if you are given appropriate resources, it would be tough to replicate the results of the paper, if you are a novice.


text:
Can YOLO detect large objects?
I have a rather basic question about YOLO for bounding box detection.
My understanding is that it effectively associates each anchor box to an 8-dimension output.
During testing, does YOLO take each anchor box and classify on it alone? What happens if the object is big and spans over several anchor boxes (e.g., covering 70% of the image)? How can YOLO classify and detect objects spanning over many anchor boxes?

If my memory is correct, I have seen objects larger than 70% of the image be detected.  Have you tried to test on your own?

@BrianO'Donnell Yes I have, but I'm just getting low f1 score for large objects.

The example given based on the yolov1 paper:

The last layer has a tensor of the dimension 7x7x30. but the dimension of the last tensor is not in every case 7x7x30.
let be:
S: the number of grid cells in X and Y direction
C: the number classes to train
B: the number of bounding boxes in every grid cell
The dimension of the output tensor is calculated with this formula: SxSx(5*B+C). The given example in their paper has the following values:
S:7
B:2
C:20
(7X7X(5*2+20) = 7x7x30)
With this configuration you can detect at most 96 object (SSB -> 7*7*2) and 2 objects per grid cell. (e.g. it's not possible to detect many small object in a few grid cells)
now lets consider what are each feature map of the last tensor used for:
The data of every bounding box is stored in 5 feature maps:
- 1 Feature map for the bounding box center in x direction
- 1 Feature map for the bounding box center in y direction
- 1 Feature map for the height of the bounding box
- 1 Feature map for the width of the bounding box
- 1 Feature map for the "confidence score" of the feature map
(confidence score = P(Object)*IoU(BoundingBox,Object))

P(Object): the probability that in this cell is an object
IoU: Intersection over Union. A ration between the overlapping area and the union area:

additional there are a feature map for every class. In every cell the following probability is calculated: P(Cass|Object)
Long story short:
If the bounding box is bigger than one grid cell, then some neurons share the same values (they share the bounding box center, the bounding box size and the confidence score, because they refer to the same bounding box).
Source:
https://pjreddie.com/media/files/papers/yolo_1.pdf
edit no Bounding boxes on the image
if there are no bounding boxes on the image, then the confidence score would be zero, because there are no overlapping area. you can configure the threshold of the confidence score. look at this sample picture with threshold 0:
additionally the feature maps of the classes would be zero because every neuron represents this probability P(Class|Object) because P(Class|Object) = P(Class And Object) / p(Object); (P(Class) = 0 -> P(Class and Object) = 0)
Is adding images without bounding boxes to the training set useful?
I think it helps to lower the number of false positive matches. But that is just a guess. I have trained yolo to detect this plant https://en.wikipedia.org/wiki/Rumex_obtusifolius and I added a lot of picture without bouding boxes, becuase a low number of false positive matches was important. The result: the specificity was >99%.
Hope this helps

Thanks for the explanation. Indeed, there's a feature map for every class. But is it useful to have images with no bounding boxes at all in the training set? Suppose there's only one class, does YOLO treat the images with no bounding boxes as negatives?

I edited my answer. I hope it answers your questions.

Thanks again. Just for comparison, do you happen to know what the specificity was without throwing in images without bounding boxes?

If my memory is correct: the specifucity was 1-2% lower but the sensitivity was 0.5% higher.


text:
If we model the game "2048" using a max-min game tree, what is the maximal path from a start state to a terminal state?
If we model the game '2048' using a max-min game tree, what is the maximal path from a start state to a terminal state? (Assume the game ends only when the board is full.)
This is one of the sub-questions that should prepare us to actually modeling the game as a max-min game tree. However, I'm failing to understand the question.
Is it actually the path to receiving 131072 as an endgame?

This is an old question, but you should specify why you fail to understand the question. What is "131072" here? What do you know about minimax? Is this a homework? If yes, you should add the tag [tag:homework]. These are questions that people reading your post (like me) might have.

To model 2048 (or any problem) for search, you need a only a few pieces of information.
Note first though, that 2048 is not suitable for minimax, because there's only one player! Instead, you can treat this as a Markov decision process. The techniques to solve it are pretty similar though. Basically, you'll do search for one player, and insert "chance" nodes at each ply of the search. The value of a chance node is the expected value of its children. Note that this will reduce the effectiveness of pruning, so it might mean the problem is not tractable for search-based approaches.

What does an end state look like? Usually you have some function G(s) that accepts a state s, and produces true if and only if it's an end state. In 2048, end states would be states where the player loses (no moves possible), or where the player wins (a 131072 tile is present), so this should be fairly easy to write.
What are the payoffs? This is usually given by a utility function U(e) that accepts an end-state e, and produces a numeric value indicating the utility the player will receive.
What actions can the player take in each state? In 2048, these are always the same (up, down, left and right)
How are new states generated from old states and player actions (in this case, the tiles slide according to the rules of the game, and then a new tile is inserted at a random empty location.

Although search might work here, since 2048 is a relatively simple MDP, you might be happier using techniques from reinforcement learning, which were specifically designed for this kind of problem. Russell & Norvig have a good set of chapters on both approaches (14-17).

You can model 2048 as a two player game, just with a random opponent (how the game chooses new tiles) instead of one playing adversarially.

2048 is still a two player game, just not in the minimax sense. The computer of the original game just plays moves randomly rather than adversarially (it is possible to program a much harder variant of 2048 that does play adversarially).
We can therefore use a variant of minimax called exptectimax that considers expected value of all possible computer random moves instead of minimizing the score in adversarial play. Since we cannot evaluate the entire state space, we must limit the depth of our search and estimate expected value using heuristics described below instead of computing it exactly.
http://azaky.github.io/2048-AI/paper.pdf for more explanations and experiments results
https://stackoverflow.com/a/22498940/3163618


text:
Which evaluation metrics should be used in training, validation and testing of a model?
Which specific performance evaluation metrics are used in training, validation, and testing, and why? I am thinking error metrics (RMSE, MAE, MSE) are used in validation, and testing should use a wide variety of metrics? I don't think performance is evaluated in training, but not 100% sure.
Specifically, I am actually after deciding when to use (i.e. in training, validation, or testing) correlation coefficient, RMSE, MAE and others for numeric data (e.g. Willmott's Index of Agreement, Nash-Sutcliffe coefficient, etc.)
Sorry about this being broad - I have actually been asked to define it generally (i.e. not for a specific dataset). But datasets I have been using have all numeric continuous values with supervised learning situations.
Generally, I am using performance evaluation for environmental data where I am using ANNs. I have continuous features and am predicting a continuous variable.

Thanks for that. Sorry about that - I will try to explain it more. I am actually after deciding when to use specifically correlation coefficient, RMSE, MAE and others for numeric data (e.g. Willmott's Index of Agreement, Nash-Sutcliffe coefficient, etc.)

Since I actually need to explore all ways performance evaluation is used in training, validation and testing, I would be happy to know how it used to train the network. Thank you

@DuttaA Thank you for your reply and sorry that this is not clear. Actually in my course I am doing it is all about using ANN to model data.  The question I have been asked to answer is about discussing which stages out of training, validation and testing should the performance metrics correlation coefficient, RMSE, MAE, Willmott's Index, Nash-Sutcliffe coefficient, Legate and McCabe's Index and Percentage Peak Deviation be used in? We haven't been given any more information or data. I think what I would like to know is probably when do we use error metrics and when to use non-error metrics?


text:
How would you design a neural network that generates the positions of comparators in a sorting network given a set of numbers?
How would you design a neural network that generates the positions of comparators in a sorting network given a set of numbers?
I've tried to modify some already implemented networks that given a set of numbers it sorts the number. My goal is, given an unsorted sequence of numbers, to generate a sorting network that will sort those numbers. I am not asking for the complete solution, just a starting point.

Did you see this: https://github.com/mayank26saxena/deepsort


text:
How can I determine whether a car in a video is moving or not?
How can I classify a given sequence of images (video) as either moving or staying still from the perspective of the person inside the car?
Below is an example of the sequence of 12 images animated.

Moving from the point of the person inside the car.

Staying still from the point of the person inside the car.

Methods I tried to achieve this:

A simple CNN (with 2d convolutions) with those 12 images (greyscaled) stacked in the channels dimension (like Deepmind's DQN). The input to the CNN is (batch_size, 200, 200, 12).

A CNN with 3d convolutions. The input to the CNN is (batch_size, 12, 200, 200, 1).

A CNN+LSTM (time-distributed with 2d convolutions). The input to the neural network is (batch_size, 12, 200, 200, 1).

The late fusion method, which is taking 2 frames from the sequence that are some time steps apart and passing them into 2 CNNs (with same weights) separately and concatenating them in a dense layer As mentioned in this paper. This is also like CNN+LSTM without the LSTM part. The input to this net is (batch_size, 2, 200, 200, 1) -> the 2 images are first and last frames in the sequence

All the methods I tried failed to achieve my objective. I tried tuning various hyperparameters, like the learning rate, the number of filters in CNN layers, etc., but nothing worked.
All the methods had a batch_size of 8 (due to memory constraint) and all images are greyscaled. I used ReLUs for activations and softmax in the last layer. No pooling layer was used.
Any help on why my methods are failing or any pointers to a related work

CNNs are translation invariant.
You are over complicating the problem.  The easiest thing you can do is define a region of interest (ROI) of the hood.  In the first case where the car is moving and the reflections are dynamic.  In the second case they are static.  Just do frame-to-frame image subtraction of the hood.  If the vehicle is moving you will have lots of 'edge energy'.  If it is not moving it will be just noise.
You can apply the same method to the whole image too.  In the static case the image subtraction method may become messy as the clouds are moving along with vehicles and pedestrians.  For this case use the image subtractions as input to your method.
Another approach is to run an image stabilization algorithm.  OpenCV has one.  Look at the transformation outputs (translation, rotation, scale, rigid, similarity, affine, etc.).  If you can't make a simple filter on them to determine the two cases, train a classifier.

Do you mean translation equivariant? pooling layers are responsible for the invariance, right? and I didn't use pooling in my methods.


text:
Back-of-the-envelope machine learning (specifically neural networks) calculations
There is a popular story regarding the back-of-the-envelope calculation performed by a British physicist named G. I. Taylor. He used dimensional analysis to estimate the power released by the explosion of a nuclear bomb, simply by analyzing a picture that was released in a magazine at the time.
I believe many of you know some nice back-of-the-envelope calculations performed in machine learning (more specifically neural networks). Can you please share them?

Yes, this question is not fully clear. This answer attempts to give you a "back-of-the-envelope" calculation (as defined by the linked Wikipedia article), i.e. a rule of thumb to approximately calculate something, although I don't think that answer is clear (or even correct). Is this type of calculation that you were looking for, or were you looking for something else? Please, edit your post to clarify that, otherwise, I will close it as "needs details or clarity".

I think the question is clear, and so do people who have provided answers. I think both submitted answers look like what I might expect. Frankly, I modeled this question based off of https://mathoverflow.net/questions/8846/proofs-without-words. So, if this question is closed, then I would expect that question also be closed. I'm open to improving the question somehow if you have suggestions.

I think a nice back-of-the envelope calculation is the intuition for exploding/vanishing gradients in RNNs:
Simplifications

diagonalisable weights $U$ and $W$
no non-linearities
1 layer

This gives a hidden state $h_t$ at timestep $t$ for input $x_t$:  $h_t = W\cdot h_{t-1} + U\cdot x_t$
Let $L_t$ be the loss at timestep $t$ and the total loss $L = \sum_t L_t$. Then (eq. 3 -> 5 in the paper)
$$
\frac{\partial L_t}{\partial W} \sim =
\sum_{k=1}^{t} \frac{\partial L_t}{\partial h_t} \frac{\partial h_t}{\partial h_k} \frac{\partial h_k}{\partial W} = \sum_{k=1}^{t}\frac{\partial h_t}{\partial h_k}\times\alpha_{t, k}
$$
Let's not care about terms regrouped in $\alpha_{t, k}$:
$$
\frac{\partial h_t}{\partial h_k} = \prod_{k<i\leq t} \frac{\partial h_i}{\partial h_{i-1}} =  \prod_{k<i\leq t}  W =  \prod_{k<i\leq t} PDP^\top = PD^{t-k}P^{\top}
$$
So you can easily see$^1$ that if the eigen values of $W$ (in the diagonal matrix $D$) are larger than $1$, the gradient will explode with time, and if they are smaller than $1$, it will vanish.
More detailed derivations in On the difficulty of training recurrent neural networks

$^1$ remember $\lim_{n \to +\infty}|x^n| = +\infty$ if $|x|>1$ and $=0$ for $|x| < 1$

You say "you can easily see that", but I cannot easily see it (so I guess many others can't easily see it too). Maybe you should provide more intuition behind that and, more importantly, define your matrices and other symbols.

I didn't think I had to explain that x^n goes to inf for |x| > 1 and to 0 for |x| < 1 as n goes to inf. Also notations are standard rnn notations and there is a link to the paper for the details you asked for.

I have one to share. This is no formula, but a general thing I have noticed.
The number of neurons + neurons should be proportionate, in some way, to the complexity of the classification.
Although this is fairly basic and widely known, it has helped me in many times to consider one thing: how many at a minimum does it need?

It looks like my question may be a bit vague. I’ll try to add more constraints this afternoon if I have time.

What does "The number of neurons + neurons" mean? Maybe you mean the number of parameters is proportional to the complexity of the neural network?

One neat thing is to estimate the GPU usage when training an ML model. This can be relevant for training or fine-tuning large models. If inference with one instance takes $N$ GB of vRAM, then one training step with batch size $B$ requires $2 \times B \times N $ GB of vRAM.

Note. The factor of 2 is there because forward pass computes activations and stores them, while the backward pass needs to store both activations and gradients for computing parameter updates through backpropagation. The exact memory usage can vary based on the network architecture and the specific operations involved, but a rule of thumb is that it's approximately 2x the memory of the forward pass.


text:
How can genetic programming be used in the context of auto-encoders?
I am trying to understand how genetic programming can be used in the context of auto-encoders. Currently, I am going through 2 papers

Training Feedforward Neural Networks Using Genetic Algorithms (a classific one)

Training Deep Autoencoder via VLC-Genetic Algorithm

However, these papers don't really help me to grasp the concept of genetic programming in this specific context, maybe because I'm not very familiar with GP.
I understand that autoencoders are supposed to reconstruct the instances of the particular classes they have been trained on. If another fed instance is not reconstructed as expected, then it could be called an anomaly.
But how can genetic programming be used in the context of auto-encoders? You are still required to create a neural network, but, instead of a feed-forward one, you use autoencoder, but how exactly?
I would appreciate any tutorials or explanations.

The first classic paper your present uses a genetic algorithm (GA), while you mention genetic programming (GP). In short, a GA uses an evolutionary process to evolve a fixed-length vector, normally a bit string, while a GP evolves variable-length decision trees, which could be a computer program or mathematical function.
An autoencoder is something that automatically encodes or decodes something. For example, a noisy image is decoded to the original image (very simplified I know), the point here is automatically, so, without any defined process, an autoencoder can "learn" how to remove the noise from the image.
Generally, an autoencoder is made using a neural network (NN). The noisy image is read in as input and the output is a clear image. To train the NN, backpropagation is used. For backpropagation to work, an error is needed to calculate the weight changes. Alternatively, the weights could be encoded as a vector. A fixed-length vector is then evolved using a GA to evolve the weights of the NN. This is what the "classic paper" mentioned does.
Using a GA to evolve the weights of a NN is not really efficient, because weights affect each other which means each time crossover happens the weights are totally messed up and the GA can easily get stuck in a local optimum. Gradient descent is probably more efficient.
The structure of a multi-layered network is important. Here, gradient descent is still used to update the weights, but a GP could be used to evolve the structure of the NN.
Alternatively, the GP itself could be used to evolve an autoencoder without using a NN. Here, the chromosome will contain nodes such as mathematical operations or computer code that would alter the input to produce the output.

The second link should now work. So, you may want to address that part of the question too.


text:
What is the significance of this Stanford University "Financial Market Time Series Prediction with RNN's" paper?
Researchers at Stanford University released, in 2012, the paper Financial Market Time Series Prediction with Recurrent Neural Networks.
It goes on to discuss how they used echo state networks to predict things such as Google's stock prices. However, to do this, once trained, the network's inputs are a day's stock price, and the output is the day's predicted stock price. The way the paper is worded is like this could be used to predict future stock prices, for example. However, to predict tomorrow's stock price, you need to give the neural network tomorrow's stock price.
All this paper seems to show is that the neural network is converging on a solution where it simply modifies its inputs a minimal amount, hence the output of the ESN is just a small alteration of its input.
Here are some Python implementations of the work shown in this paper:

https://github.com/kimanalytics/Recurrent-Neural-Network-to-Predict-Stock-Prices
https://github.com/europa502/RNN-based-Bitcoin-Value-Predictor

In particular, I was playing with the latter which produces the following graph:

If I take the same trained network, and alter the 7th's day's "real" stock price to say something extreme like $0, this is what comes out:

As you can see, it basically regurgitates its inputs.
So, what is the significance of this paper?
It has no use in any financial predictions, like the network shown in the paper Classification-based Financial Markets Prediction using Deep Neural Networks.

Without looking too deeply: The first paper's network is naive about the system it is trying to model, while the second one has more assumptions about the system it is trying to model.

After a very brief look at the paper, I think that they are predicting the stock price for the next day, and not for the current day, which is quite common and reasonable: see equation (1), where they predict $x(t+1)$.
So I don't see any issue with this paper.
But I've only quickly looked at it, so I may have missed something, of course.


text:
Can a neural network work out the concept of distance?
Imagine a game where it is a black screen apart from a red pixel and a blue pixel. Given this game to a human, they will first see that pressing the arrow keys will move the red pixel. The next thing they will try is to move the red pixel onto the blue pixel.
Give this game to an AI, it will randomly move the red pixel until a million tries later it accidentally moves onto the blue pixel to get a reward. If the AI had some concept of distance between the red and blue pixel, it might try to minimize this distance.
Without actually programming in the concept of distance, if we take the pixels of the game can we calculate a number(s), such as "entropy", that would be lower when pixels are far apart than when close together? It should work with other configurations of pixels. Such as a game with three pixels where one is good and one is bad. Just to give the neural network more of a sense of how the screen looks? Then give the NN a goal, such as "try to minimize the entropy of the board as well as try to get rewards".
Is there anything akin to this in current research?

I don't think you would directly feed it the image and have it find the distance... Rather you would feed it a set of numbers that represent distance, vertical distance, horizontal distance, etc.

@Pheo yes, but you would have to feed it different values for every type of "game". Whereas what I'm saying is, could we have some global type of value that is high when pixels are grouped together and low when pixels are spaced apart?

"The next thing they will try is to move the red pixel onto the blue pixel." might to do not will "red" and "blue" are most times are enemys so you will start to increase distance before the blue pixel notice you.

We can break down the problem as follows:
First, if you have two points on a plane and feed the coordinates of those points to a neural network (e.g., a vector $< x_0, y_0, x_1, y_1 >$) and and train it on a label thats the actual distance (e.g., $ \sqrt{(x_0 - y_0)^2 + (x_1-y_1)^2} $), it should be able to learn this relationship with arbitrarily close accuracy.
Next, if you have an image similar to what you describe, and feed that through a different neural network (e.g., a CNN), and as labels you used the the points of the two dots (once again $< x_0, y_0, x_1, y_1 >$), then it should be able to learn that relationship with arbitrarily close accuracy once again.
Of course, there's no reason to do this in two separate neural network, so we can just combine the two end-to-end have a model that takes the image as input and the distance as output.
This model would need to be trained on labeled data, however, so you'd either need to generate the data yourself or label images.
But if you wanted it to learn the notion of closing a distance in a less supervised way, you'd need to use reinforcement learning. In this case, you'd have to setup an environment that incentivises the agent to reduce the distance. This could be as simple as gaining reward if an action reduces the distance.
Another approach would be to incentivise the agent using future reward. That is, it's reward doesn't just come from the results of the next immediet state, but there's also contributions from the next possible state, and the one after that, and so on. This is the idea behind Deep Q-Learning, and I implement a simple example (very similar to what you're describing) in this notebook.
So, now the question is: has this implementation done something other than randomly moving around until it follows a path to success?
In your example, you talk about rewarding the agent when it lands on the goal. But in what I described, it gained reward by moving closer to the goal (either through the Q-Function or directly from the environment). It is able to do so by learning some abstract idea of distance (which can be illustrated in the supervised version).
When a human learns this, it's for the same exact reason: the human is gaining a reward for moving in that direction through a sense of future rewards.
I'd say that, given enough training and data, reinforcement learning could learn this concept with ease. As far as other rewards being present on the board (e.g., "minimise the entropy of the board as well as try to get rewards"), you need to think about what it is you're asking. Would you rather the agent minimize distance or maximize reward? Cause, in general, it can't do both. If you're looking for some balance between the two, then really you're just redefining the reward to also consider the distance.

Answer
I'm going to take your question at face value, and go really deep into this topic.
Yes, they can. The typical human mind can. But consider the human mind. Millions, if not billions, of neurons. In fact, one can consider distance as a human concept, simply a theory developed from interactions with the world.
Therefore, given a year or two, with a ton of neurons on your hand, you could replicate this scenario. That is if your computer is as parallel as the human mind. The short explanation is that the human mind is very parallel.
However, it would be simpler to calculate the distance with a program, not an AI, and simply feed the result to the AI that would make the decisions.
Consider the amount of time you have spent looking at a screen. If you can tell the (approximate) distance between two pixels, so can a Neural Network, as you are one. However, add the amount of time you have spent alive and learning into the equation, and it becomes a disaster.
Further reading
The human brain is parallel
This is a result of the fact that all of the neurons in the human brain are independent of each other. They can run true simultaneous actions, thus making the action of interpreting images and such much easier, as blocks of neurons can "think" independent of the operations of the others, limiting what would be "lag" to a minuscule amount.

You can create AI to "see" as a human. As you said, giving the human the keys, he will click randomly. He just needs to know which keys he presses that brings him closer to other objects on the screen. I think the basics of an AI is object recognition. I would try to create a script to map the screen objects of the game. There are legal examples in Python.
I would try to follow a path like this:

Make the AI understand that by clicking the arrows or the WASD and it is in the context GAME, the object that move pixels according to the direction, represents the main author (the player).
In parallel: map all boundaries of the region and index different objects within that region to automatically have the coordinate domain and object distance. AI needs to SEE (stream) the game and through images to categorize objects. Do you understand what I mean?
In parallel: The AI needs to be aware of all texts and information that is on the screen (all mapped, remember?). You need to understand when a text changes or something different happens. For example: whenever he returns to the initial position of each phase, whenever he has a count, what happens when the cout reaches zero or a common number that generates another type of change.
He needs to understand what is repeated at every "respawn". You also need to understand what "respawn" is. Maybe a certain map position on every map it returns whenever a count on the screen ends. Or when it comes up against a certain type of object (mapped object)

To be honest, if you want to create a super intelligent robot, you can go following all the steps that go through the heads of different humans, or the best humans, or the rules of each game. But sometimes it's easier to build specific bots to perform specific tasks. It depends on what you want to do

He was not asking how you would do it, but rather can you do it.

It is possible to do it in several ways. I passed the way I would take to create the template. It is not a theory, it is a process that can encompass other processes according to the evolution of AI.

What you mention there is the perfect example for path-planning, which is extensively researched in AI.
Please look for A-star algorithm and how to enhance it with neural networks :)


text:
Why does C++ seem less widely used than Python in AI?
I just want to know why do machine learning engineers and AI programmers use languages like Python to perform AI tasks and not C++, even though C++ is technically a more powerful language than Python.

You don't need a powerful language for programming AI. Most of the developers are using libraries like Keras, Torch, Caffe, Watson, TensorFlow, etc. Those low level libraries are highly optimized and handle all the tough work. They are built with high-performance languages, like C, C++. Python is just there for high level task like describing the neural network layers, load data, launch the processing, and display results.
Using C++ for high level task instead of Python would give barely any performance improvement, but it would be harder for non-developers as it requires to care for memory management. Also, several AI people may not have a very solid programming or computer science background.
Another similar example would be game development, where the engine is coded in C/C++, and, often, all the game logic scripted in a higher-level language.

C++ is actually one of the most popular languages used in the AI/ML space. Python may be more popular in general, but as others have noted, it's actually quite common to have hybrid systems where the CPU intensive number-crunching is done in C++ and Python is used for higher level functions.
Just to illustrate:
http://mloss.org/software/language/c__/
http://mloss.org/software/language/python/

Good point, however the links do not work anymore: they both forward to the same list of results, not filtering per language. Next time, please provide a textual list of examples for both cases, and only use the link for historical reference. Answers on SO/SE should not strongly rely on external links.

It depends how flexible it needs to be: if you have a fully-fledged system ready for production, which is not going to need much adjusting, then C++ (or even C) might be fine. You need to put a lot of time into building the software, but then it should run pretty fast.
However, if you're still experimenting with settings and parameters, and maybe need to adjust the architecture, then C++ will be clumsy to work with. You need a language like Python which makes it easier to change things. Changing the code is easier, as you can generally code faster in languages like Python. The price you pay is that the software does usually not perform as well.
You need to decide how that trade-off works best for you. It is usually better to spend less time on coding, and not worry too much about longer run-time. If you take a day less to get your code done, that's a lot of time the C-coded version needs to catch up. Most of the time it's just not worth it.
A common approach seems to be hybrid systems, where core libraries are implemented in C/C++, as they don't need much changing, and the front-end/glue/interfaces are in Python, as there you need flexibility and speed is not that critical.
This is not an issue specific to AI, by the way, but a general question of interpreted vs compiled languages. With AI a  lot of systems are still focused on research rather than application, and that is where speed of development trumps speed of execution.

You claim that

C++ is technically a more powerful language than python.

But that claim is wrong (or does not mean much). Remember that a programming language is a specification (often some document written in English). For example, n3337 is a late draft of the C++ specification. I don't like Python, but it does seems as powerful than C++ (even if C++ implementations are generally faster than Python ones): what a good Python programmer can code well in Python, another good C++ programmer can code well in C++ and vice versa.
Theoretically, both C++ and Python are Turing-complete (on purpose) programming languages.
And Python is as expressive as C++ is. I cannot name a programming language feature that Python has but not C++ (except those related to reflection; see also this answer and be aware of dlopen - see my manydl.c program -, of LLVM, of libgccjit, of libbacktrace, and consider some meta-programming approach with them, à la Bismon or like J.Pitrat's blog advocates it).
Maybe you think of a programming language as the software implementing it. Then Python is as expressive as C++ is (and seems easier to learn, but that is an illusion; see http://norvig.com/21-days.html for more about that illusion). Python and C++ have a quite similar semantics, even if their syntax is very different. Their type system is very different.
Observe that sadly, many recent major machine learning libraries (such as TensorFlow or Gudhi, both mostly coded in C++) are in practice easier to use in Python than in C++. But you can use TensorFlow or Gudhi from C++ code since TensorFlow and Gudhi are mostly coded in C++ and both provide and document a C++ API (not just a Python one).
C++ enables multi-threaded programming, but the usual Python implementation has its GIL, is bytecoded, so is significantly slower than C++ (which is usually compiled by optimizing compilers such as GCC or Clang; however you could find C++ interpreters, e.g. Cling). Some experimental implementations of Python are JIT-compiled and without GIL. But these are not mature: I recommend investing a million euros to increase their TRL.
Observe also that C++ is much more difficult to learn than Python. Even with a dozen years of C++ programming experience, I cannot claim to really know most of C++.
Sadly, most recent books teaching AI software engineering (e.g. this one or that one) use Python (not C++) for their examples. I actually want more recent AI books using C++ !
BTW, I program open source software (like this one, or the obsolete GCC MELT) using AI techniques, but they don't use Python. My approach to AI applications is to start designing some DSL in them.
Some AI approaches involve metaprogramming, e.g. generating some (or most, or even all) the code of a system by itself. J.Pitrat (he passed away in October 2019) pioneered this approach. See his blog, his CAIA system, read his Artificial Beings, the conscience of a couscious machine book (ISBN 978-1848211018) and the RefPerSys project (whose ambition is to generate most -and hopefully all- of its C++ code).
On operating systems such as Linux you could in practice generate C++ (or C) code at runtime and compile it (using GCC) into a plugin, then later dlopen(3) that generated plugin, and retrieve function pointers by their name using dlsym(3). See the manydl.c example (on a powerful desktop in 2020, you would be able to generate and load half a million of plugins, if you run that example several days). With dladdr(3) and Ian Taylor's libbacktrace, you can also inspect some of the call stack.
AFAIK major corporations such as Google use C++ internally for most of their AI-related code. Look also into MILEPOST GCC or the H2020 Decoder project for an application of machine learning techniques to compilers. See also HIPEAC.
Of course, you can code AI software in Haskell, in Common Lisp (e.g. with SBCL), or in Ocaml. Many machine learning frameworks can be called from them. Number crunching libraries could use OpenCL.


text:
Will artificial intelligence make the human more rational?
With so much innovation, with so much previous human manual labor being performed in minutes or seconds by an artificial intelligence, one day man will put the survival and propagation of his species above his ideologies and cultures.
I am worried because we are living the fourth industrial revolution, and this will generate millions of unemployment, even if new jobs are created in the future. The problem is that a lot of humans worry about their own job, and not about their own children's future. This is completely retrograde.
Will, one day, Artificial Intelligence be able to direct us towards an intelligent path as a propagation of the species, or else center the focus of humanity on something that it adds?

Quite salient.  I keep seeing serious scholars making the point that it's not superintelligence we need to be worrying about atm, but "strong narrow AI" replacing humans in the laborforce in the very future.  Numerous other scholars have mentioned human "short term thinking" as the source of many of our current woes.  You see it manifest concretely in computing in the form of technical debt, but that's just a minor symptom.

a bit of a random comment, but measures of rationality and measures of intelligence are not super well co-related, which upshots that the two are not the same.

AI in my opinion is an enabler and provides a better path for everyone, most people are pushing it away because it seems to be a journey into the unknown so this sense of not really having complete control of our destiny brings about a bit of a sense of unease. So yes AI will bring enlightenment to the human race

I would counter this question with another one:
Is rationality all we strive for?
Intelligence is a waste ground, and we have not even defined which intelligence we're talking about: Emotional Intelligence (empathy), Rational/Intellectual Intelligence (IQ), artistic intelligence, etc. etc. etc.
Something which demarks us as humans is that sometimes we're able to deal with to counteracting intelligences. Logic sometimes dictates one thing, but logic is bound to what parameters are taken into account, and mostly it's about quantifying. an intelligence which does qualitative logic behaves sometimes different, and suggests sometimes other actions.
As humans we weight and counterweight these factors.
Politics is a good example. In most countries there are 2 or 3 rational approaches to problem solving (to make it easy let's call them left, center, right - though I know in fact it's more positions and in an at least two dimensional space, not linear).
But what I mean is all those positions are rational, nevertheless it's other types of intelligence who define their the way they use their ratio.
So in the end this fourth step is just a continuity and I assume that with time theoretically at least the shift should be away from blue collar jobs to more white collar jobs until there are no blue collar jobs.
So far the theory I guess, but then again? Would this be smart? Or intelligent? A world where machines make all the dirty work and we're just supervisors?
My thought to this is we will still need mechanics, probably with more technology always more, just that their focus will shift. 20 years ago mechanics were teached analogue technology, nowadays they learn more digital technology (best example would be car mechanics), but there you see: They're still needed, we also need people with analogue knowledge to mantain older systems which cannot or will not be converted to digital era. And I think that's OK!
So from my point of view, nobody has to fear to loose it's job, but instead embrace, that when you keep your knowledge up to date, there will always be a job for you.
Nowadays for example IT-professionals who are able to code older computer languages are better paid than ever, because their scarce...

This is a very good answer. Politics is very relevant to it I think. Specifically I think meritocracy, transhumanist politics and fully automated luxury communism are the only political ideas and sometimes even political parties in some countries are the closest political dynamics steered towards artificial intelligence and future technologies. I like Fully automated luxury communism and the idea that jobs will be literally non existent but humans still get to live a good life.

Artificial intelligence may exercise human rationality in some conditions, but if, after a time, all the thinking is delegated to computers, humans are likely to fall backward into superstitious times.
===============
Since the text of the question unrelated to the title question, I'll treat that as a separate entity.
The fact is that most people do manual labor because they feel they must.  If intellectual skills are also replaced by machines, those jobs will disappear, but not because the work is left undone.  The farm and factory automation replaced the skilled workers in many cases, but the farming and manufacture continues.  The same would be true of office work.  The cubes and once coveted window offices will be empty and decrepit, like the depleted oil fields of Pennsylvania, Texas, California, and the U.K., but the office work will continue to flow.  The computers are doing it.
Theoretically, the standard of living will remain the same if there is a one to one replacement.  If the computers improve above the skill and throughput of people, the human standard of living should increase.
Humans can play sports, watch movies, blog, and garden for fun.  A world where there is 100% unemployment but a high GNP is idyllic.
I'm not sure we need to be directed toward an intelligent path in those regards.  Now if AI could be used to teach children not to solve problems by beating the crap out of each other, betraying each other, gossiping behind each other's backs, or scheming and scamming then that would be a plus.
Perhaps AI could someday teach people not to drop bombs on each other or point thermonuclear weapons at each other too.


text:
Why use semi-gradient instead of full gradient in RL problems, when using function approximation?
Semi-gradient methods work well in reinforcement learning, but what is there a reason of not using the true gradient if it can be computed?
I tried it on the cart pole problem with a deep Q-network and it performed much worse than traditional semi-gradient. Is there a concrete reason for this?

Just so that this could be useful for people who refer to this post later on: Please refer to Sutton's reinforcement learning book (2nd edition) example 11.2. It provides an example for why full gradient wouldn't work.

Semi gradient methods work well in Reinforcement Learning, but what is the reason of not using the true gradient if it can be computed?

Just complexity and extra computation, in many cases for a marginal benefit.

I tried it on the cart pole problem with a deep Q-Network and it performed much worse than traditional semi gradient, is there a concrete reason for this?

It is hard to tell, without exploring the implementation in detail. However, DQN is an inherently unstable learning technique that needs care in choosing hyper-parameters that control this instability and offset against learning rate:

size of minibatch to train from experience replay on each step
number of training steps between taking frozen copies for estimation*
whether or not you use double-learning to avoid maximisation bias (more important if you have fine-grained discretisation of continuous action space)

There is a chance that the optimal choices here are different between true gradient and semi gradient approaches.
* The frozen estimator could be a big clue here in your implementation. If you are using this frozen copy technique, it has a big impact on how you should calculate the true gradient, because changing the parameters would no longer change the current TD target - which is what the true gradient approach fixes. However, getting rid of this stability-improving addition in order to get true gradients might on balance make the algorithm less stable - you could try to fix that by taking larger mini-batches.


text:
Do individual dimensions in vector space have meaning?
Word2vec assigns an N-dimensional vector to given words (which can be considered a form of dimensionality reduction).
It turns out that, at least with a number of canonical examples, vector arithmetic seems to work intuitively. For example "king + woman - man = queen".
These terms are all N-dimensional vectors. Now, suppose, for simplicity, that $N=3$, $\text{king} = [0, 1, 2], \text{woman} = [1, 1, 0], \text{man} = [2, 2, 2], \text{queen} = [-1, 0, 0]$, then the expression above can be written as $[0, 1, 2] +  [1, 1, 0] - [2, 2, 2] = [-1, 0, 0]$.
In this (contrived) example, the last dimension (king/man=2, queen/woman=0) suggests a semantic concept of gender. Aside from semantics, a given dimension could "mean" a part of speech, first letter, or really any feature or set of features that the algorithm might have latched onto. However, any perceived "meaning" of a single dimension might well just be a simple coincidence.
If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information? Or is this purely a "random" artefact of the algorithm, with only the full N-dimensional vector distances mattering?

Do individual dimensions in vector space have meaning?

IIRC, some dimensions are interpretable, but in general this is not the case. Also it is debatable as to wether it is actually learning the actual representation or just an approximation of it. But in any case its not very reliable outside from some edge cases.

If we picked out only a single dimension, does that dimension itself convey some predictable or determinable information?

Yes, but as to what that information entails in terms of "meaning" is lesser clear. You could say that if in a certain dimension the distance between two vectors is 0, then you have am estimate of the real distance that is better than guessing.

IIRC means: if I recall correctly


text:
How to perform gradient checking in a neural network with batch normalization?
I have implemented a neural network (NN) using python and numpy only for learning purposes. I have already coded learning rate, momentum, and L1/L2 regularization and checked the implementation with gradient checking.
A few days ago, I implemented batch normalization using the formulas provided by the original paper. However, in contrast with learning/momentum/regularization, the batch normalization procedure behaves differently during fit and predict phases - both needed for gradient checking. As we fit the network, batch normalization computes each batch mean and estimates the population's mean to be used when we want to predict something.
In a similar way, I know we may not perform gradient checking in a neural network with dropout, since dropout turns some gradients to zero during fit and is not applied during prediction.
Can we perform gradient checking in NN with batch normalization? If so, how?

You should be able to do gradient checking as long as you fix the randomness by fixing the random seed, on python you might want to look at numpy.random.seed.
From http://cs231n.github.io/neural-networks-3/#ensemble :

When performing gradient check, remember to turn off any non-deterministic effects in the network, such as dropout, random data augmentations, etc. Otherwise these can clearly introduce huge errors when estimating the numerical gradient. The downside of turning off these effects is that you wouldn’t be gradient checking them (e.g. it might be that dropout isn’t backpropagated correctly). Therefore, a better solution might be to force a particular random seed before evaluating both (f(x+h)) and (f(x-h)), and when evaluating the analytic gradient.

This link from Stanford is by far the best resource on gradient checking that I have encountered so far:
http://cs231n.github.io/neural-networks-3/
I am sure it will help you a lot.
Pro Tip: make sure that you use "centered/central difference" formula for the derivative calculations and also use "relative error" (not absolute) to compare the two gradients.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Is it possible to learn to estimate the minimum value in a table?
Is it possible to classify or learn to estimate the minimum value in a table if the values are integer and represented 32 bits (and we can input all variables at the same moment, like in system on a chip (SoC))?

I'm not sure what you're asking. To find the minimum in a list, you have to look at every element once, and the simple solution (a for loop scanning the table) is a theoretically optimal O(n). If you get to design circuitry that can read every input at once, you can get that down to O(1) time, though the cost in terms of silicon and complexity would be formidable. Or something like O(log n) pretty simply. In any case, I'm not sure what you're hoping for from a learning algorithm. Can you elaborate?

@mico why is that ?

@deong you absolutly right, but to answer your question i was looking for resolve a minimum search problem as a learning problem, and try to design a parallel solution just to figure out if any problem can be considered a learning problem

I know there has been a lot of work on things like learning parallel sorting algorithms, for example. I've seen some work on genetic algorithms for that I know. That might provide you a starting place to jump off from.

There are several ways to take this question:

can a learner ingest a table and give an exact minimum in less compute than the O(n) method?
can a learner ingest a table and give an exact minimum in NOT less compute than the O(n) method?
can a learner find patterns in tables and predict an approximate minimum in less compute than the O(n) method?
can a learner find patterns in tables and predict an approximate
minimum in NOT less compute than the O(n) method?

Answers:

If there is no larger scale pattern then the best the classical learner is going to do would be the O(n) method; none of the current ones should beat it.
It is possible to get the exact value with much more compute overhead.  This is like using a backhoe to plant a carrot, you can do it, but its overkill.
If the table has patterns that are matched to the learner, then it can beat the search.  Think of a classification and regression tree model (CART), which can be reduced to a few if-then's and give an answer in O(1).
Because of 3, if the pattern exists but the learner is slow like FFNN which is O(n^5) to predict, it can be slower and if done right might also be as good.