text:
How long it takes to train face recognition deep neural network? (rough estimation)
If I use a desktop PC with a GPU, how long it might take to train face recognition deep neural network on let's say dataset of 2.6 million images and 2600 identities? I guess it should depend on various properties (e.g., type of the DNN). But I am just looking for a rough estimation. Is it a matter of hours/days or years?
Thanks!

Very much dependent on your processing power and the computational complexity per DuttaA's comment.

Training time depends on a lot of parameters. Some of them are:

Size of each image (resolution)
Color/Monochrome image (color image has 3 times data if you consider RGB image)
Like you mentioned on the type of DNN.
No. of layers of DNN.
No. of neurons in each layer.
Total no. of images in the dataset. (2.6 million here)
GPU you are using (you didn't mention which GPU you are using. There are GPUs with wide range of capabilities, to predict the time, you need to know the exact specs of GPU).
Training time also depends on RAM on your machine and also how fast host PC can transfer data to GPU for processing.
Since you mentioned face recognition, i am assuming you are using CNN, but if you again use fully connected network, training time will change and will obviously increase manifold.

Your task of classification and your database is mostly similar to ILSVRC that uses imagenet database.
Making some reasonable assumptions for the parameters you didn't mention, i feel your task is similar to ILSVRC and i am predicting the training time will be a few days.
Below are the links which mention time of training for ILSVRC.
https://mxnet-tqchen.readthedocs.io/en/latest/tutorials/imagenet_full.html
Below are the details of imagenet database for your comparison
https://en.wikipedia.org/wiki/ImageNet

YELP Dataset (200k images) used to take 5 hr for training to identify Five (5) classes on GPU - Nvidia 1080 Ti with 11 GB RAM. So I guess in your case it will take days. Again it will depend on the type of your GPU configuration and type of Architecture you will be using.


text:
What are criteria for ML model to be satisfactory for commercial use?
I often read "the performance of the system is satisfactory" or " when your model is satisfactory".
But what does it mean in the context of Machine Learning?
Are there any clear and/or generic criteria for Machine Learning model to be satisfactory for commercial use?
Is decision what model to choose or whether additional model adjustments or improvements are needed based on data scientist experience, customer satisfaction or benchmarking academic or market competition results?

I find this answer useful for my question as well: https://ai.stackexchange.com/a/15391/28605

The answer is "when it works well enough to perform the task that you have set it".
It is a good idea to set your performance criteria in advance so that you can clearly identify the goal that you are trying to achieve and also so that you will know if the model is likely to be successful or not.

From what I have observed, the ability to scale an ML model is key.
Real time inference must be quick, and cause no delays from the provider side. Being able to deploy the model also carries enormous weight - that is, how easy would it be to build the data pipelines and how easy would it be to integrate it in a web application from the server perspective.
Apart from the obvious achievement of set metrics and performance criterion, speed and ease of deployment also carry a very important role. There have been scenarios of brilliant solutions being denied (from what I have seen) because they exceeded the limits set for time and compute in an application scenario.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Current research on indoor localization and navigation in changing environment?
I'm trying to get up to speed on the latest research regarding indoor localization, scene classification, navigation in changing environment, etc.
Any advice would be appreciated, but I'm especially interested in recent research papers from vetted sources.

I would kindly request you to check out community guidelines for your insightful knowledge.

@quintumnia I don't understand the request. What should I do and what should be the outcome of such action?

we have guidelines,which we follow here.

@quintumnia That doesn't answer any of my question. Is something wrong?

I would start with:

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization
Image-based localization using LSTMs for structured feature correlation
Image-based Localization using Hourglass Networks
UAN: Unified Attention Network for Convolutional Neural Networks
Scene Coordinate and Correspondence Learning for Image-Based
Localization

You can also search for articles using the 7-scenes dataset (Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images) which is pretty standard

Welcome to SE:AI and thanks for contributing!  (I've edited the question to include the titles of the papers, with links embedded.  Please follow this protocol in future, since links can expire, and posting web addresses reduces readability.)

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How have robots developed and how sophisticated might they be in the future?
We all know how robots are getting more and sophisticated and more interesting
What is the future of robotics in relation to AI, how and how will AI work with robotics in improving to affect our world in a positive way.

Welcome to AI! I've slightly modified the question to focus on the AI aspects of robotics, so that it will be on-topic for this forum.

The future of robotics

Companies like Google are doing restless research in robotics whereby machine learning algorithms are applied to empower self driving cars well equipped with computer vision, etc.
The future is so exciting in that it will be an era where artificially intelligence machines can rewrite their own source code and constantly improve upon themselves at an exponential rate -- ASI: artificial super intelligence.
Right now, robots like Sophia are using machine learning algorithms to answer or respond to human interactions.

The next to last paragraph (source code editing and ASI) seems like subjective speculation. Could you provide sources to that? Also to the last paragraph?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What is a model and how is it designed?
I read these things on the internet like

My model determines the future scope..."

or

My model gives accurate readings about what the score would be..."

What are these models? How are they designed?

Welcome to AI!  It would be helpful if you could provide more context, ideally links to where you read it.

@DukeZhou This question lacks artificial intelligence context

A model is exactly what the name suggests.  A simplified representation of a solution to a real life problem.
For example, if you think of a simple formula for a falling object you may not take into account real life variables such as the imperfections on the surface of the object, atmospheric conditions, the exact composition of the air the object is falling through and how it may affect friction.  you just use a simple model to "model" the fall and get a reasonably accurate result.  its a model, it doesnt have all the details.  just like a model car wouldnt have all the details of a real car.
Here's a good read to get you started:
https://en.wikipedia.org/wiki/Mathematical_model

The word "model" may not suggest anything, if you don't know the meaning of the word.

"A simplified representation of a solution to a real life problem.": why is a model a representation of a solution? Are you sure?

In the context of what the poster is asking it is a solution or a problem or just anything.  I tried to explain in the rest of the post but it doesnt matter as I have a feeling this whole thread will be marked off-topic soon.

These links might strengthen this answer: Model-based reasoning, Bayesian Networks, and Statistical Model


text:
How to use MOPSO to align characters vertically?
I need to efficiently align characters vertically using Multi Objective PSO. Alignment is achieved by adding spaces in between a given set of characters.
a b c d e f
b b d h g
c a b f

Might be
- a b - c d e f - -
- - b b - d - - h g
c a b - - - - f - -

Now this is a multi objective solution.
I need to maximize the characters that get aligned vertically and minimize the amount of spaces in between the characters.
I wanted to focus firstly on how to get a set of characters to represent a position of a particle. This would mean that I need to somehow transform a possible set of characters into a position of a particle. If I can somehow achieve this then the rest should fall into place.

How do I transform these set of characters into a position of a particle?
Also is this the best approach or are there better ways to approach this problem?

Welcome to AI!  I did a slight edit to highlight the questions, and added the "combinatorics" tag, which seemed appropriate for this problem.  Feel free to re-edit or update as you see fit.

Your problem appears to be multiple sequence alignment. That problem is well studied, and depending on your application, there are fast algorithms that can perform quite well without recourse to AI or optimization, and which are detailed in the provided link.
If your problem has a lot of local minima, then a PSO might be a good approach. Certainly local search has been widely used for this kind of problem in the past.
Your state should be represented by a list of sequences, each with some blanks inserted. Movement between states can consist of adding or deleting blanks from one or more sequences. The distance between two states can be computed as the sum of the number of operations needed to transform each sequence to its corresponding sequence in the other state, which you can compute in linear time (just sum up the needed additions or deletions between each pair of letters).
Hope that gives you a good starting place!


text:
Detect observations under certain conditions
I have a customer purchasing dataset and the data set is from a retailer having an online store and offline stores. So, customers have two options in their shopping channel, online or offline. In an online shopping, there is a shipping fee however if a basket size is larger than $50 there is no shipping fee.
I found pieces of evidence that customers are trying to add some of items to make their basket size larger than $50 when their baskets are near and a little bit below the $50, because their shipping fee can be waived by doing that.

In this situation, I am trying to identify and characterize items that were purchased only because of the shipping threshold by using a machine learning algorithm.

If there is no shipping threshold, $50, the customers would not purchase the items, but they purchased some items to make their basket size larger than $50. I have not observed those kinds of items (added items because of the shipping threshold).

Is there any machine learning algorithm that I can identify those kinds of items?

I think I need to use some of unsupervised machine learning algorithm.
Another challenging part is that each customer has different characteristics so I probably need to consider it as well. How can I detect those kinds of items??


text:
How to implement exploration function and learning rate in Q Learning
I'm trying to implement Q-learning (state-based representation and no neural / deep stuff) but I'm having a hard time getting it to learn anything.
I believe my issue is with the exploration function and/or learning rate. Thing is, I see different explanations in the sources I am following so I'm not sure what's the right way anymore.
What I understand so far is that Q-learning is TD with q-val iteration.
So a time-limited q-val iteration step is:
Q[k+1](s,a) = ∑(s'): t(s,a,s') * [r(s,a,s') + γ * max(a'):Q[k](s',a')]

Where:
Q = q-table: state,action -> real
t = MDP transition model
r = MDP reward func
γ = discount factor.

But since this is a model-free, sample-based setting, the above update step becomes:
Q(s,a) = Q(s,a) + α * (sample - Q(s,a))

Where:
sample = r + γ * max(a'):Q(s',a')
r  = reward, also coming from percept after taking action a in step s.
s' = next state coming from percept after taking action a in step s.

Now for example, assume the following MDP:
    0    1    2    3    4
0 [10t][ s ][ ? ][ ? ][ 1t]

Discount: 0.1 |
Stochasticity: 0 |
t = terminal (only EXIT action is possible)
s = start

With all of the above, my algo (in pseudo code) is:
input: mdp, episodes, percept
Q: s,a -> real is initialized to 0 for all a,s
α = .3

for all episodes:
    s = mdp.start

    while s not none:
        a  = argmax(a): Q(s,a)
        s', r = percept(s,a)
        sample = r + γ * max(a'):Q(s',a')
        Q(s,a) = Q(s,a) + α * [sample - Q(s,a)]
        s = s'

As stated above, the algorithm will not learn. Because it will get greedy fast.
It will start at 0,1 and choose the best action so far. All q-vals are 0 so it will choose based on arbitrary order on how the qvals are stored in Q. Asume 'W' (go west) is chosen. It will go to 0,0 with a reward of 0 and a q-val update of 0 (since we don't yet know that 0,0, EXIT yields 10)
In the next step it will take the only possible action EXIT from 0,0 and get 10.
At this point the q-table will be:
0,1,W:      0
0,0,Exit:   3 (reward of 10 averaged by learning rate of .3)

And the episode is over because 0,0 was terminal.
On the next episode, it will start at 0,1 again and take W again because of the arbitrary order. But now 0,1,W will be updated to 0.09. Then 0,0,Exit will be taken again (and 0,0,Exit updated to 5.1). Then the second episode will be over.
At this point the q-table is:
0,1,W:      0.09
0,0,Exit:   5.1

And the sequence 0,1,W->0,0,Exit will be taken ad infinitum.
So this takes me to learning rates and the exploration functions.
The book 'Artificial Intelligence: A Modern Approach' (3ed, by Russell) first mentions (pages 839-842) the exploration function as something to put in the val update (because it is discussing a model-based, value iteration approach instead).
So extrapolating from the val update discussion in the book, I'd assume the q-val update becomes:
Q(s,a) = ∑(s'): t(s,a,s') * [r(s,a,s') + γ * max(a'):E(s',a')]

Where E would be an exploration function which according to the book could be something like:
E(s,a) = <bigValue> if visitCount(s,a) < <minVisits> else Q(s,a)

The idea being to artificially pump up the vals of actions which have not been tried yet and so now they'll be tried out at least minVisits times.
But then, in page 844 the book shows pseudo code for Q-learning and instead does not use this E in the q-val update but rather in the argmax of the action selection. I guess makes sense? Since exploration amounts to choosing an action...
The other source I have is the UC Berkeley CS188 lecture videos/notes.
In those (Reinforcement Learning 2: 2016) they show the exploration function in the q-val update step. This is consistent with what I extrapolated from the book's discussion on value iteration methods but not with what the book shows for Q-Learning (remember the book uses the exploration function in the argmax instead).
I tried placing exploration functions in the update step, the action selection step and in both at the same time.. and still the thing eventually gets greedy and stuck.
So not sure where and how this should be implemented.
The other issue is the learning rate. The explanation usually goes "you need to decrease it over time." Ok.. but is there some heuristic? Right now, based off the book I am doing:
learn(s,a) = 0.3 / visitCount(s,a). But no idea if it is too much or too little or just right.
Finally, assuming I had the exploration and learn right, how would I know how many episodes to train for?
I'm thinking I'd have to keep 2 versions of the Q-table and check at which point the q-vals do not change much from previous iterations (similar to value iteration for solving known MDPs).

I think that very good exemple in Python is http://amunategui.github.io/reinforcement-learning/

Your main problem is that you need to separate out what is driving the behaviour policy from the Q-table.
Q Learning is an off-policy algorithm. The Q-table that it eventually learns is for an optimal policy (also called the target policy). In order to be able to learn that policy, the agent needs to explore. The usual way to do this is to make the agent follow a different policy (called the behaviour policy). For efficient learning, you generally want the behaviour policy to be similar to the target policy. So it is common to also drive the behaviour policy from the Q-table, but not absolutely necessary.
You do not need an exploration function, but it is one good way to drive exploration.
The simplest behaviour policy, and one that will work in your case, is to behave completely randomly - ignore the Q-table and select actions at random. With your simple toy problem, that should work reasonably well.
A more common approach is to behave $\epsilon$-greedily. For some probability $\epsilon$ (e.g. $\epsilon = 0.1$), behave randomly. Otherwise take the argmax over a of $Q(s,a)$.
The exploration function approach is similar to what you have so far, you are just missing the separation of behaviour policy from the Q-table updates. The Q-table update ignores the exploration function, and maximises over estimated next values:
$$Q(s,a) = Q(s,a) + \alpha (r + \gamma \text{max}_{a'}Q(s',a') - Q(s,a))$$
The behaviour policy for picking the actual next action to take can be decided by using the exploration function:
$$a' = \text{argmax}_{a'} E(s',a')$$
Note that in stochastic environments (where the chosen action may lead randomly to multiple different states), that you may be able to get away without any separate behaviour policy, and always act greedily with respect to the Q-table. However, that is not a generic solution - such a learning agent would do badly in deterministic environments.

Right, my exploration function was meant as 'upgrade' from a strictly e-greedy strategy (to mitigate thrashing by the time the optimal policy is learned). But I don't get why then it won't work even if I only use it in the action selection (behavior policy). Also the idea of plugging it in the update step I think is to propagate the optimism about exploring not only to the state in question but to states leading to that state. As for separating behavior and learning, shouldn't the behavior be driven from what has been learned so far?

@SaldaVonSchwartz: "the idea of plugging it in the update step I think is to propagate the optimism about exploring" - that sort of works with the function as you have described it, because it cuts out after certain number of actions. Eventually the Q-table will forget the large optimistic starting values due to the fixed learning rate. But really this is the same as behaving randomly to start with (e.g. for 100 steps) then behaving closer to optimally later. You still need some exploration after the exploration function is exhausted, thus need a non-greedy action selection.


text:
Data extraction from medical reports
I am new in Machine Learning. I have taken a course in vision and we are required to do a project.
I am thinking of  data mining medical lab report images. My code must take an image and jpg file and then extract important information from it like lab where test has been done, patient name, test type and more important various data like heamoglobin, RBC, etc in case of blood test report.
I can build an OCR, but, problem which I am stuck at is in case of data which generally forms a table like structure. So, I want to find that tabular structure on which I can just apply matrix extraction to find various datas.
I'm looking for assistance with two basic things:

Is my approach of finding tables and then extracting data is correct? If yes, then can you point out some good papers or implementation to find tabular structure. (P.S.- Don't mention tabular)
Any approach which is state-of-the-art or good? (Paper or implementation)


text:
Is it fair to compare AlphaGo with a Human player?
A human player plays limited games compared to a system that undergoes millions of iterations. Is it really fair to compare AlphaGo with the world #1 player when we know experience increases with the increase in number of games played?

Define fair. :)

Is it fair to compare AlphaGo with a Human player?

Depends on the purpose of the comparison.
If we are comparing ability to win a game of Go, then yes.
If we are comparing learning ability, then maybe. It depends on the task. AlphaGo and systems like it are capable of learning only in well-described limited domains. There may be an analogy with sensory learning (it might even be possible in theory to take a small piece of brain tissue and run an algorithm similar to AlphaGo's learning process on it).
In general, the approach used by AlphaGo and other reinforcement learning successes is "trial-and-error plus function approximation". It seems analogous to perception and motor skills, such as object recognition or riding a bike, as opposed to reasoning skills and games as humans play them, which goes through many more cognitive and conscious layers that have no real analog in a RL system like AlphaGo.

A human player plays limited games compared to a system that undergoes millions of iterations

This is an advantage of a machine to learn this kind of task. It would equally apply in other simulated environments with simple rules. If your goal is to have the most skilled and optimal navigation of such a domain, the implication now is that you would not train a human expert through years of study, but to write the simulator and train an AlphaGo-like machine.
This is no different a comparison than deciding cars and roads are better solutions to long distance travel for the general population than walking or horses and carts. It doesn't matter what underlies the advantage of one over the other, the assessment is cost/benefit, which resolves to a single comparable number.
It would, however, be wrong to assess AlphaGo as a better general-purpose learning engine than a human. The fact that humans do not have to work fully through millions of simulations in full detail is important. It means that something about how humans learn is still not covered by learning machines. Some of these things are understood and being discussed - such as the ability to focus intuitively on important aspects of what to learn, the ability to reason about the environment, learning analogously or transfer learning from other domains.

As a chess player and a AI/ML engineer, I can say yes, why not. I'm not sure why it isn't fair to compare anything if you give each side it's just due and do a 'fair comparison'. Obviously, what that encompasses is extremely subjective, but there are philosophical and logical measures of fairness.
Now speaking on the comparison, AlphaZero and a Human's learning styles are much more similar than that of a Human's and Stockfish. This is mainly due to the fact that human's in some capacity use RL, mainly in the dopaminergic neural pathways. While human behavior can certainly be modeled as an alpha/beta tree-search, it is not anything like the way we make decisions.
As far as the top humans, who cares? We we've been worse than computers for years.

Yes it is. If we ever compare computers to humans we should take into acount the fact that computers can work 24 hours a day every day and faster than humans. That is bigest advantage of computers over humans.

If you read through the abstracts of Chess AI papers, it is often pointed out that humans "search" the Chess game tree much more efficiently than computers, which was why it was so hard to beat the top humans in Chess for so many years.  (The human efficiency may have to do with intuition and judgement, which are difficult to replicate. "Confidence levels" for AI evaluations is one method of addressing these issues, as is "monte carlo".  But it's also important to note that humans are far more limited in the depth and breadth of their "searches", which is why, now that we have the right algorithms, humans can no longer win.)

Is it fair?

Perhaps the more salient question is:

Is it useful to compare AlphaGo to a human player?

It most certainly is, because it tells us that we have is sometimes termed a "strong-narrow AI" that can outperform a human in a single task.
Why AlphaGo beating Lee Sedol was a big deal is the complexity of Go, the intractability of the Go game tree, and the fact that computers were previously ineffective against high-level human Go players.
This human vs. AI evaluation doesn't strictly fall under the "Turing Test" (Imitation Game), it does fall squarely under the maxim of Protagoras that "Man is the measure of all things."
This is critical because intelligence is a spectrum, and gauging strength of intelligence, in the context of intractable problems (problems that cannot be fully solved due to their size) is a function of relative strength of two agents, whether human or AI.
This relative assessment is all we have, and all we may ever have for certain sets of problems.
The problem with humans is not that we're not clever, but that our minds have cognitive limitations.  So to tackle certain problems, intelligent machines are useful.

There is no such thing as fairness when comparing. You define a measure for performance and then compare the values of the measure.
One sensible measure for playing the game of GO is the 'Number of games won', regardless of any investment in the development of the system, computational or sample efficiency. AlphaGo is currently at the top by this measure.
Another sensible measure could be 'Number of games won under a restriction on sample efficiency during training'. As others pointed out, such a measure could be much more favorable for humans.


text:
Stacked softmax layers before output
I have seen people using stacked softmax layers right at the output of neural networks designed for classification. I'm trying to understand this. Does it give any additional value? I think this could "sharpen" decisions on the boundaries.

model.add(Dense(10, activation='sigmoid'))
  model.add(Dense(1,
  activation='sigmoid'))

Seen here.

could you give an example? i dont think this does anything


text:
Can the first emergence of consciousness in evolution be replicated in AI?
At some point in time during the evolution, because of some factors, some beings first started to become conscious of themselves and their surroundings. That conscious experience is beyond some mere sensory reflexive actions trained. Can that be possible with AI?

Current limitations in our knowledge mean that the question is not directly answerable:

There is no scientific consensus on what consciousness is. Therefore any device designed to "be conscious" is necessarily going to be built on the premise of unsupported, maybe fringe, theory.
There is no robust measure of consciousness. If any AI system was built in order to exhibit conscious behaviour, there would be no way to prove it is conscious. There is no general agreement or theory on whether any particular animal species is conscious for example. This is often limited by communication. Of the few animals smart enough to be trained in communication with humans, there appears to be conscious behaviour. Researcher opinion ranges from "all non-humans do not possess consciousness" to "all animals have some degree of consciousness".
There is incomplete understanding of what the components of consciousness are. A bottom-up build of a conscious machine requires a baseline theory of what those components are.
We may be able to ignore lack of knowledge and take a very high level of abstraction, such as A-life or evolutionary approach where nothing is assumed and the hope is that consciousness would spontaneously emerge from a complex enough simulation (as we assume it has done with organic life in the real world). However, this would seem to require many orders of magnitude more computing power than is currently possible.

To answer the question as written:

Can the first emergence of consciousness in the Evolution be replicated in AI?

Despite the many books, articles and posts written on this subject over many years, the answer is two-fold:

We do not know of any fundamental reason why AI could not be conscious.
We have no theory or experimental proof that AI can replicate consciousness.

I would go further than this, and say that anyone who tells you otherwise on these two points has already subscribed to some unproven theory of consciousness.
As well as well-thought-out peer-reviewed theories and experiments by scientists and researchers, there is a lot of pseudo-scientific junk published on the internet on this subject. So take care if researching reading material.

@Bobs: Potentially. However, that is already a statement from "some unproven theory of consciousness". There is absolutely no evidence that consciousness cannot be achieved physically via evolution. In that regards, all we have is a big question: How did life on Earth result in conscious systems? The answer to date: We don't know.

It partly depends on the framing of the question, in terms of how you are defining consciousness.
Neil Slater's answer is comprehensive, and his warnings about pseudo-science and junk publication should be heeded.
However, since you frame this in the context of "Can the first emergence of consciousness in evolution be replicated in AI", I feel like I can provide an answer.

If we define rudimentary consciousness as simple awareness of the environment, distinct from higher functions such as self-consciousness, then yes.

Under this definition, any algorithm that takes input is "conscious".  This in no way represents human-level consciousness, or even the consciousness of higher animals, but is more akin to simple organisms such as microbes.

I personally have no doubt human reactions are largely mechanistic, and would surely be deterministic if quantum theory didn't open the door to true randomness in nature (validity of free will tbd.)  But thank you for recognizing my reduction of the question.  This is a complex problem, so starting small is likely the way to go.

I see in all the answers a conflation of consciousness with self-consciousness.  I don't think anyone would disagree with the statement: "all living organisms exhibit some degree of awareness of their environment" -- even a mosquito is "aware" of the air pressure changes caused by the hand attempting to smack it.  That would correspond to consciousness.  The issue is really the attainment of self-consciousness, that is, the "awareness of one's own awareness", i.e., that "self-awareness" that homo sapiens is the only one to have manifested up to the present. In this view a definition of "self-consciousness" arises as follows: self-consciousness is attained whenever consciousness takes itself as an object. In this view, the evolution of consciousness is seen as reaching the level above mere "awareness" (shared, as said before, by all living organisms) into awareness of the awareness of the self. The background exposition of this definition of self-consciousness was first first proposed in the late 90s by the "modern teleologues", an anonymous group that produced a substantial output of theory in France and Canada. As regards AI, the jury is still out as to whether it will be able to become self-conscious and I consider the question an open-ended one.


text:
State representation of position in 2D plane for Reinforcement Learning (Q Learning)
I recently finished Course on RL by David Silver (on YT) and thought about trying it out on simple application in Unity Game Engine, where I've built simple labyrint with ball and want to teach the ball to get from point A to point B in there while avoiding obstacles and fire (the place where you'll get burnt so big negative reward)
The problem I encountered while designing the whole thing (programming-wise) is: What is the correct (or at least good) way of representing the position in 2D space? It is continuous so I thought about representing it as feature vector consisting of [up, down, left, right, posX, posY] where direction is whether I am pressing button of moving in that direction in binary (or actions if you want) and pos are floats (0-1) representing normalized position from one corner on the plane where the whole map is. That would be accompanied by vector W that would represent the weights adjusted using Gradient Descent.
Question is: will this work?? I am asking for 2 reasons. One is that I am not so sure about that posX and posY since it can be 0 and if I multiply it by the weights vector then how could be resulting reward anything but 0? Second reason is that I am not sure if the actions should be part of the features. I mean, it makes sense to me but I could easily be very wrong since I am a beginner.
Thanks a lot guys in advance. If you have any more questions or think the problem is not described deeply enough just ask in the comments and I'll edit the question. :)
PS: I could just code it the way I think is right, but I also want to get gasp of designing applications on paper before coding them (project management).

Shouldn't the action be the output not the input? If you also learn a bias vector you mitigate the pos==(0,0) problem.

@BlindKungFuMaster That's the second reason there. To explain my thought process - The vector there is intended to be used as input to Q(s, a) which will be a function with parameters W (vector) that will be learned. The exact action then will be extracted from there finding max a in that Q(s, a) where S will be extracted from actual position of the ball and A will be found in some sort of loop.
What do you mean by the bias vector? Is it that W vector I mentioned or something else? Thanks.

Ah, ok, so Q predicts the expected reward. A bias vector would be added to the product of input and W. So it would B would contain additional parameters for Q.

So in the end it would be something like: Q(Sa, B) = W*Sa + B where B would be just some value so the Q of state (0, 0) won't be always 0 and therefore everything will be shifted by B which will not matter because in the end as W gets learnt the Q will converge to the actual values anyway, just W will be different then as if I did it without B. Do I understand correctly?

Yes, though I would suspect that you'll need more layers anyway, which would look like Q(Sa, B) = W2f(W1Sa + B1)+B2, except if your labyrinth is extremely simple.

I think your net should have the various actions as outputs, but I am not an expert in Deep Nets. I just think that that light form of multi-task learning might be better. The idea of multi-task learning is that a predictor predicting multiple variables (in this case the various Q(s,a1), Q(s,a2), ...) using mostly the same structure (varying only the output weights) will learn more sensible things. Though I admit applying this here might be a bit of a stretch.
As for the real question, a popular technique in Reinforcement Learning is Tile Coding.
The basic idea is to discretize the (2-dimensional, in your case) state-space - imagine a grid laid over the 2D space - and assigning an input feature to each cell; all of these variables are set to zero except for the one your continuous variables fall into. For example, if your grid is 20x20, you will have 400 variables, 399 of which are set to zero, and 1 set to one.
Tile Coding takes this one step further and repeats this using slight offsets for the grid. Imagine you create an identical grid but you move it slightly to the right by 1/10 of the width of a cell: you will have another set of 400 variables like before, but it is possible that the cell set to one is not the same. Then you repeat this moving the grid by 2/10 and you have another set of 400 variables, again, only 1 of which is set to one. In total you have 10 sets of 400 variables (if you repeat more than that, you get the same grids as before); of your 4000 variables, only 10 are set to one.
Now you repeat this by adding a 1/10 of a cell offset in the Y axis and obtain another 4000 variables. Repeat with 2/10 and you get another 4000. By the end of it, you have 40000 variables, 100 of which are set to one.
Now your net can more easily learn different weights for different positions. I recommend you to follow the link above for a better explanation than mine (and figures!)
My suggestion is to feed all of these variables to your net and have it predict the Q-value for all of the actions. But, again, I am no expert in deep nets so I may be wrong.
Also, according to Andrej Karpathy, "most people prefer to use Policy Gradients, including the authors of the original DQN paper who have shown Policy Gradients to work better than Q Learning when tuned well.". This means that you may be better off not using Q-learning (as they did in the original DQN formulation) to train your net. Have a look at Andrej's blog and the paper he points to.

So basically you are suggesting to divide continuous space into tiles so effectively making it discrete and therefore simplify the problem. Yeah that would work. I just wanted to try doing continuous space to see if I understand the point. But still, the answer is great so thanks. I may try it if the continuous thing suggested by BlindKungFuMaster will fail. About Policy Iterations, I am just beginning so first I want to try classic Q-Learning and then move on Policy methods, still thanks for the suggestion.


text:
Is transistor the first artificial intelligence?

Artificial Intelligence is any device that perceives its environment
  and takes actions that maximize its chance of success at some goal.

I got this definition from Wikipedia that cited "Russell and Norvig (2003), Artificial Intelligence: A Modern Approach".
A transistor is a device that amplifies or switches electronic signals when it received an input signal.
Could one say the transistor is the AI?
It is certainly a basic building block of every AI out there, but is it an AI itself, albeit the most basic one?
I'm looking at it from a technological and economic point of view, leaving philosophy aside. From an economic perspective it seems to be an AI because transistor does useful work that it took an intelligent human to perform less than a century ago. And it does it completely on its own.

I think it might come down to whether the transistor is making a decision.  If the transistor is being used as a switch, that would seem to qualify as a decision, even though it's an extremely rudimentary decision.
Intelligence, in reference to Artificial (or Algorithmic) Intelligence, is not restricted to high intelligence.  A brute force Tic-Tac-Toe AI has extremely low, narrow intelligence but still constitute AI.  An automatic switch, which makes the most simple decision possible, a binary choice, would seem to be the most basic form of intelligence.
Norvig's definition seems rooted in game theory, which is important in terms of utility of intelligence.  But in a condition of intractability one is only assuming one's decision is more optimal than other choices. Outcomes can be evaluated, and a determination made as to whether the algorithm was "smart" or "dumb", but these terms refer to relative positions on a spectrum.
It's worth noting this fundamental definition of AI opens up a can of worms in that pre-transistor, automated mechanical switches such as the Strowger switch would also probably qualify.
And automata do not have to be electrical.  The History of Automation wiki suggests the first feedback control system was for a water clock invented by Ctesibius.  This device dates to the 3rd Century BCE, and water clocks were said to be the most accurate time pieces until Huygens.
This type of intelligence I tend to think of as autonomic, in the sense of involuntary, and distinct from higher functions, which are more commonly associated with "intelligence".

Note: The characteristics of an autonomic system in a computing context are quite interesting and include self-optimization, self-learning, and self-awareness.
ADDENDUM: After much thought, in reference to @JT's answer, I can't see how decisions can be separated from goals—there needs to be an intent, or the decision is merely random. This might prompt the question "can a simple switch be said to have a goal?"

Replacing my previous ill conceived answer with this definition of Intelligence from Richard Sutton (a founder and leader Reinforcement Learning) should answer your question.

John McCarthy long ago gave one of the best definitions: "Intelligence
  is the computational part of the ability to achieve goals in the
  world”. That is pretty straightforward and does not require a lot of
  explanation. It also allows for intelligence to be a matter of degree,
  and for intelligence to be of several varieties, which is as it should
  be. Thus a person, a thermostat, a chess-playing program, and a
  corporation all achieve goals to various degrees and in various
  senses. For those looking for some ultimate ‘true intelligence’, the
  lack of an absolute, binary definition is disappointing, but that is
  also as it should be.
The part that might benefit from explanation is what it means to
  achieve goals. What does it mean to have a goal? How can I tell if a
  system really has a goal rather than seems to? These questions seem
  deep and confusing until you realize that a system having a goal or
  not, despite the language, is not really a property of the system
  itself. It is in the relationship between the system and an observer.
  (In Dennett's words, it is a ‘stance’ that the observer take with
  respect to the system.)
What is it in the relationship between the system and the observer
  that makes it a goal-seeking system? It is that the system is most
  usefully understood (predicted, controlled) in terms of its outcomes
  rather than its mechanisms. Thus, for a home-owner a thermostat is
  most usefully understood in terms of its keeping the temperature
  constant, as achieving that outcome, as having that goal. But if i am
  an engineer designing a thermostat, or a repairman fixing one, then i
  need to understand it at a mechanistic level—and thus it does not have
  a goal. The thermostat does or does not have a goal depending of the
  observer. Another example is the person playing the chess computer. If
  I am a naive person, and a weaker player, I can best understand the
  computer as having the goal of beating me, of checkmating my king. But
  if I wrote the chess program (and it does not look very deep) I have a
  mechanistic way of understanding it that may be more useful for
  predicting and controlling it (and beating it).
Putting these two together, we can define intelligence concisely
  (though without much hope of being genuinely understood without
  further explanation): Intelligence is the computational part of the
  ability to achieve goals. A goal achieving system is one that is more
  usefully understood in terms of outcomes than in terms of mechanisms

The question is based on two concepts:

First artificial intelligence (AI)
The transistor is an intelligent device.

Let us talk about the first AI, why transistors, the same definition of intelligence can be applied to Vaccum tubes, and they definitely existed before transistors. So no matter what definition you decide for intelligence, transistors are not the first AI.
Now we come to the next part, what is artificial intelligence, like intelligence, the definition of artificial intelligence has undergone changes, in the last 6 decades.
If you use the definition loosely, almost any device can be intelligent even an electric bulb.

It's instructive to know the definition of the transistor.
Transistors are electronic components that are used for amplifiers, as circuit breakers, as connectors, as voltage controllers, for signal modulation and others functions.
(An analogy is that the transistor functions as an electric "faucet" that regulates input and output voltage.)
The definition of AI by Andreas Kaplan and Michael Haenlein defines artificial intelligence as:

"The ability of the system to interpret external data correctly, to learn from that data, and to use that learning to achieve certain goals and tasks through flexible adaptation."

[Citation Needed]
Under this definition, transistors are not part of AI, because transistors do not learn and cannot adapt.  Transistors are just electronic devices for regulating current.

Transistor is very similar in its function to single neuron and because of that one transistor could be considered to be a very tiny neural network - from this perspective it could be considered to be a form of artificial intelligence on their own. But transistor is not the first building block of AI, the first building blocks are the smallest particles that are possible in phisics - those are the building blocks of any machine.
Also the definition is somehow hard to accept, as there is no reason why thermostat can be considered AI on their own and transistor not. Since thermostat needs an environment in which it works in a way that fulfills the definition of AI, so the transistor could be put in such environment - there's no difference. We consider that something is AI when it maximizes chnace of some goal, but who is defining  this goal? We do. So if you say that the goal of some AI system is to have the output current equal to input current multiplied by factor of x, then you can say that bipolar junction transistor is the AI that does exactly that. Of course you can build a processor and memory from such transistors, then write software for eg. face recognition for a computer builded from them. It is all about connecting algorithms together to perform more sophisticated functions. The first "logical gates" (and builing blocks of AI) are the smallest particles possible, the smallest particles that can interact with each other to produce some output available to be used in next interaction - the whole universe works bacause of that - the matter just follows the algorithms - laws of phisics. On the side note: The only thing that for me is not algorithmical is consciousness, I think it is impossible to be created by just algorithms working together. Algorithms doesn't have any feelings and any number of them will not have feelings either - this will only process information without any consciousness inside. Intelligence is computational, consiciousness somehow reaches beyond phisical/algorithmical world - it is worth consideration - I personally think that we are like players in a game and there is other dimension/world from which our consciousness comes from, good question is: what are the rules of this game?


text:
Neural Network to Interpolate Matrices
I am considering some possibilities to improve a Variable Gain nonlinear control system.
One of the drawbacks of the current technique is that the change of the gains is discrete and the switching effect is highly undesirable...
The gain matrices are 4x16 and are calculated for some values of a bounded variable and then selected trough a lookup table.
So, what my idea was to try and do this with an Neural Network, that would receive as input the control variable and give the gain matrix as output, allowing my control to be continuous or almost continuous.
Is this idea possible/feasible?
My knowledge on neural nets is very poor, so I hope you can enlighten me somehow.

Welcome to AI!  (I took the liberty of editing your tags, as "control problem" has a slightly different meaning than I think you intended;)


text:
How much power might the creator of the first Artificial General Intelligence obtain?
If Artificial General Intelligence is possible, whoever creates it is likely to have a lot of power.

What might this look like in practical terms, and what might be limiters?

Good answers will include aspects such as historical precedents related to technology, and cover subjects such as potential societal and economic impacts, and limiters such as regulations and physics.

@nbro do you have a rationale for re-opening?  (Answers would likely be primarily opinion based.  I'm not opposed, since I study the mythology of AI, but my sense is community consensus would be that this is an unsuitable question.)

@DukeZhou Sorry, I wanted to "save" this question. I wanted to leave just the question "How much power will the creator of the first AGI obtain?", which I think could be accepted. However, I realised that the current only answer attempts to address also the other question "Who will be the first creator of true artificial general intelligence?". So, maybe, at this point, we should leave it closed (but I can't remove my vote to re-open it), even though we have a lot of similar questions whose answers will be speculative.

@nbro no worries.  It seems to me that the second part of the question, "how much power with the creator of the first AGI obtain" might be solid--good answers would discuss the intersection of regulations & technology, the history and potential issues with enforcement.  Physics would also come into play, per your recent answer on "intelligence explosion"...

"Artificial Intelligence" has existed for many, many years.  Decades even.  Go back to Arthur Samuel and his Checkers playing program (if not before).  That's AI.
The thing is, as soon as computers can do something, people want to quit treating it as AI.  AI is almost, by definition, something that can't really exist, because it has to be about something humans can do, that computers can't.   So all progress made in the field is reclassified as "not AI" as soon as it does anything useful.
A more salient question might be "who will be the first to announce that Artificial General Intelligence" has been created.   AGI will, if it ever happens, be fundamentally different in terms of being able to learn to do anything, as opposed to, say, a Checkers program which can only play Checkers, or a Go program that can only play Go, etc.
And the answer to that question is -- nobody knows.  Nobody even know for sure that AGI will ever happen (though a lot of us presume that it will).  Anyway, there isn't a ton of research on AGI going on, relative to the amount of research in Machine Learning / Narrow AI.  But somebody working in their garage could make a huge breakthrough tomorrow for all we know.  I think a lot of people assume that creating this will take Nation-State level resources, but that hasn't really been proven.  So it might be some kid in a garage in Omaha tomorrow, or it might be the Chinese government in 2074.  There's just no way to really know.


text:
Book recommendations on deep learning (convolutional neural networks)
I am working on software which deblurs the motion blur created by camera movement.
 I've surveyed some research papers and determined this process requires deep learning and CNN. Now I'm looking for some books that would be useful in getting a more complete picture of the process.

If you are new to neural networks, I recommend studying the free online book Neural Networks and Deep Learning. It teaches you the basic concepts and the underlying math - a great starting point to dig deeper.
Once you understand the basic concepts, I recommend watching the Stanford Lecture CS231n about Convolutional Neural Networks for Visual Recognition. It consists of 16 lectures and teaches you most of the things you'll need to know for your project. The also provide all the slides - check out the links below each video.
Analyzing existing projects and their documentation is a good next step for deeper understanding of what's state of the art in CNN. You could start with this article about Q-Learning based on a CNN architecture, but there are many more out there.
More specific questions will come during your project. You'll find more information in scientific papers and blogs from researchers. Feel free to ask here as well, if you don't find what you are looking for elsewhere.


text:
AI efficiency KPI
There should be some Key Performance Indicators designed for measuring AI performance. For example, the number of entities examples you have to feed it in order to obtain single task on a testing entity with repeatable 97% accuracy.
Is there any of such measure constructed?
Motivation: you cannot learn AI to be good at playing a game released 1 month ago because there is not enough data on how to play. It doesn't matter how clever you are, present brute force paradigm in AI just doesn't fit in such circumstances.

Welcome to AI!  I feel this needs a slight qualification in that game complexity is a determining factor.  If complexity is low, brute force is more than adequate. If the game is non-trivial, solution and AI performance becomes an issue. Great question though.

The KPI's would probably be different for each methodology.  For example an NB classification algorithm can easily reach %97 and above (and it would be very easy to measure this) but it would be difficult to measure say a game playing algorithm.  Would you suggest counting the wins/losses or another way to measure them?

Take a 4,5 years old child for example. It takes 2 or 3 times to point on a cat and a child is able to recognise it. Take Google for example. 10^7 cat pictures is probably nearly enough to learn Google how cat look like. I would like to measure this gap.

It would entirely depend on what algorithm you chose to use.  I'm a huge fan of NB (personal choice) as it is easy to construct and works surprisingly well compared to more sophisticated methods.  Try using Weka and play around with the differences in algorithms, it will also provide the statistics you are looking for.  You can download it for free from the Waikato Uni website.


text:
Strategic planning and multi dimensional knapsack problem
I'm trying to find a planning approach to solve a problem that attempts to model learning of new material. We assume that we only have one resource such as Wikipedia, which contains a list of articles represented as a vector of knowledge it contains and an effort to read that article.
Knowledge vector and effort
Before we start, we set a size for the vector, depending on the number of different types of knowledge. For example, we can define the items in the vector to be (algebra, geometry, dark ages), and then 'measure' all the articles from this point of view. So, a math article will probably be (5,7,0), since it will talk a lot about algebra and geometry but not about the dark ages. It will also have an effort to read it, which is simply an integer.
Problem
Given all the articles (represented as knowledge vectors with an effort), we want to find the optimal set of articles that help us to reach a knowledge goal (also represented as a vector).
So, a knowledge goal can be (4,4,0), and it's enough to read an article (2,1,0) and (2,3,0), since, when added, it adds up to the knowledge goal. We want to do this with minimal effort.
Question
I've tried to some heuristics to find an approximation, but I was wondering if there is any state of the art strategic planning method that can be used instead?

It might help to divide the knowledge vector by the effort - this way you know how much knowledge per effort an article gives you.

Very well structured and interesting question.  Welcome to AI!

Is addition of the vectors so they sum to the knowledge vector the only criterion? If so, you're problem seem to be a multi-dimensional case of the coin problem https://en.wikipedia.org/wiki/Coin_problem  https://msp.org/involve/2011/4-2/involve-v4-n2-p07-p.pdf

Could you clarify that you don't want critique of the "learning of new material" model (which IMO seems an unusual way to model acquiring knowledge, whilst the goal of attaining an arbitrary score within the model is more well-defined)? I don't think you do from what is written, but now this has been bumped up to the top, it is possible that someone will respond to that, and not to the knapsack problem presented

Here is a speculative cast of the problem to a travelling salesman problem, which would lead to shortest-path algorithms.
Please note this idea suggests different constraints to explore.

Given the knowledge vectors and efforts, build a acyclic directed graph (acyclic, as we are not supposed to unlearn). A vertex is an article, represented by its knowledge vector. An edge links two articles, weighted by the effort to "move" to the target article/vertex (i.e. acquire the knowledge of that article).
Assign a zero vector to a new participant. That is the starting point on the graph is vertex V0 = (0, ..., 0).
Define a learning objective as a vector V.
Use a shortest-path algorithm to find a (V0, V) plan.

This procedure is insufficient, as there are many ways to build the graph (in other words, the above is completely pointless as is). Extra constraints are necessary to make it practical. For example, we can order the vertices by ordering them along each dimension. Such setting would lead learners to start with "easy" articles (V[i] is low), and move step by step toward more complex topics ((V[i] gets higher).
The graph construction depends on the data available. For example, are knowledge vectors "absolute", or can they be relative? Relative can help in creating a path, as moving from V to W requires an effort that depends on your learner's initial conditions (V0 may not be 0 everywhere, afterall).

Is it an AI question? Definitely.


text:
How to detect when human voice / speech appears in an microphone stream?
I want to build a personal assistant that listens to me continuously..
The flow looks like this:

continuously record voice
stream it to google speech api.
get back the text in real time -> parse for intent etc..

Problem is, google speech api gets expensive if you record for hours. A better way to do it is to only submit the parts where I'm actually speaking to it.. Then the cost of running this full time (17 hours a day, every day) becomes very accessible.
Now my question is:
How can i detect that a voice is present in the microphone stream?
I have a lot of noise in the background, dumb increase in volume detection is not a very good solution. I need something more intelligent. It doesn't need to be very accurate - just good enough to not break my cloud computing budget. I'm thinking that human voice sounds distinct enough that is not such a big problem to detect when is there.
What do you recommend me to do, given this is a real time stream - not an audio file.
The audio will be generated in chromium browser (electron) - with getUserMedia API, and using node.js i plan to handle the streaming logic.
Note: there is a built in speechRecognition api in electron - but from my experience currently it doesn't work (not even after i give it my API key), and even if would have worked, i think it has the same cost problem. So this is why i'm trying to provide my own implementation.
I don't know what i'm doing, any insight is welcomed:) Thank you.

Welcome to AI.  I've added a "voice recognition" tag since this is in that general field.

Your problem is an old one. There are many methods, referred to as Voice Activity Detection (VAD) methods, which detect speech from an audio signal.
The typical design of a VAD algorithm follows one of more of these 3 approaches:

A noise reduction stage, e.g. via spectral subtraction.
Some features or quantities are calculated from a section of the input signal.
A classification rule is applied to classify the section as speech or non-speech – often this classification rule finds when a value exceeds a threshold.

There may be some feedback in this sequence, where parameters (like the noise threshold or the classification threshold) are tuned to improve the estimate or to adaptively vary the threshold(s). These feedback operations improve the VAD performance in non-stationary noise (i.e. when the noise varies a lot).
Check out this wiki page on Voice Activity Detection for more information.

Ah, that does look like a good start! In the interest of keeping answers as self-contained as possible, could you maybe include an outline of the architecture? That would be great.

Good point, updated.


text:
Handling varied-size input with fixed-input network
I'm running A3C (Asynchronous Actor-Critic Agents) to learn a game where an agent needs to catch 3 rewards. The input of my network, among other things, is the relative position of the 3 rewards against the agent.
However, as the agent catches these rewards, my network can't just decrease its input size. What are ways of handling this?
My current solutions have been:

set remaining features to zero - have tried, bad results
just show the closest reward - loss of information, might not be optimal
repeat existing reward positions to remaining features - we are giving more emphasis to some rewards (the ones that are repeated) over others

Have you tried adding an additional, binary feature indicating for each reward if it has already been caught?

@user12889 Great idea, actually, I hadn't thought of that


text:
Help with implementing Q-learning for a feedfoward network playing a video game
I want to train a feedforward neural network to play a video game called Puyo Puyo 2, using reinforcement learning. More specifically, I'm trying Q-learning but I'm open to better alternatives.
In this video game, you start with an empty 12x6 board which you fill with falling pairs of colored blobs called puyos. When 4 or more puyos of the same color are connected horizontally and/or vertically, they disappear and the puyos above them fall down, potentially creating new groups of puyos that disappear and so on, creating a chain (here is a chain example : https://puyonexus.com/chainsim/image/5SoqU.gif).
My goal is to teach the neural network to build the longest chains possible given a random sequence of puyo pairs.
The game's information is incomplete though, as you only see the current and the next two pairs at a given time.
What is the best choice for the output layer: a layer with 22 neurons, one for each possible way to place a pair at a given time, and a softmax activation function, or a layer with a single output node, with a linear activation function? Or something else? Also what do you think is a good number of hidden layers and neurons per layer and what activation function should I use for the hidden layers?
What should my target Q-value be when I evaluate a move, so that I can train the network on each game situation? Right now I'm thinking about using the maximum chain possible after the move is done, divided by half of the number of moves played so far. This is because if you play optimally and you are lucky enough, you can increase your maximum chain by 1 every 4 puyos you place, that is every two moves. Thus my Q-value would stay between 0 and 1.
What is the best update formula I can use? Is the one on wikipedia correct?
If I use a softmax output layer, how do I update the target Q-value vector? The update formula gives me an update but then the vector doesn't represent probabilities anymore. Can I add the update then take the softmax of the vector again just to make it probabilities again?


text:
Is the traditional meaning of "strong AI" outmoded?
Traditionally, "strong AI" refers to Artificial General Intelligence, the human mind understood as an algorithm (Searle, Chinese Room) and Artificial Consciousness.
But recent advances in Artificial Intelligence, most notably breakthroughs involving Machine Learning and Neural Networks, have led to strong AI being used in the context of an algorithmic intelligence that can outperform the top humans in specific tasks.  (This is sometimes qualified as "strong narrow AI", but the qualifier seems less used time goes on.)

Because the traditional usage of strong AI are based on theoretical concepts, where the current usage is based on practical, real-world results, does the new usage of "strong AI" supercede the traditional usage?

If this is the case, at what point does the Wiki need to be updated?  Is a major paper focused on re-coining the term necessary to formally reference the new usage of the term, or are numerous, informal uses sufficient?

Related: "Strong" has a precise, analytic, formal usage in the context of solved games.  This game theoretic usage is similar in spirit to new usage of strong AI in that it connotes results that can be validated.

This is an old question, but I just stumbled upon it. Can you provide 1-2 links to sources that use the expression "strong AI" to refer to systems like AlphaGo and the expression "strong narrow AI"?

@nbro I'll do some google searches for references.  (I asked this question during the honeymoon period after Alpha Go when I saw the term starting to pop up.)

This "strong narrow AI" is marketing buzzword; we used to call them "weak AI", defined in your terms as "algorithmic intelligence that can outperform the top humans in specific tasks".

The meaning of strong AI has changed as you correctly indicated, and the term "strong narrow AI" is more appropriate as now people are shifting towards the practical uses.
This also bring to light the number of breakthroughs that AI is having and literature and other various related resources will often(unlike in other fields) need to be changed constantly.


text:
What are the (general) sizes of AlphaGo and Deep Blue?
I'm trying to get a gauge on just how big the programs and databases are these automata.  I understand that this is a changing number, particularly in regard to Machine Learning.
Q: How large was Deep Blue when it beat Gary Kasparov?
Q: How big was AlphaGo when it beat Lee Sedol?

Let's get the concepts correct. AlphaGo uses Monte-Carlo Tree Search, it doesn't prune. Also, we don't prune a database, we search from it. We prune a search tree, like the alpha-beta algorithm.

@StudentT thanks for the correction.  (I've amended the question to remove the problematic language.)

AlphaGo used data from the KGS Go Server, which had 160,000 games and 29 million board/next-move pairs. But crucially, after it was trained on the dataset, AlphaGo was trained through self-play, so its competence shouldn't be measured strictly in terms of its database.
I'm not 100% sure how Deep Blue worked, but I think it was a mix of 1. a "book" of opening theory 2. explicitly coded board evaluation functions 3. a "book" of endgames. So there isn't a "database" in your traditional "ML by big data" sense. But in any case, I would assume the bulk of the work is done by the evaluation function, so again its strength cannot be measured in terms of if its database.

Thanks for answering!  I should clarify that my project involves non-networked automata, restricted by the resources of a single device, and the allocated volume of the gamespace (the app) which ideally is trivial.  (The goal is semi-strong AI that for a metagame with complexity < Chess to > Go that runs on a mobile phone.  Metagame here is defined by a set of core mechanics that can be extended in numerous ways, including board configuration and value adjustments which do not require additional mechanics, but alter equilibria.) Thus I'm interested in raw bytes.

Great point about Deep Blue--I now want to ask a question about the size of the programs, which would include the evaluation functions, which I assume are fairly weighty in the IBM case, from what I know about that program and it's development.  Game books I definitely consider database content--it's still data stored in memory for access.

That Im not sure about. I recall IBM was very secretive about this (going as far to disassembling the machine not long after the match), and a quick search shows nothing regarding the specifics of the algorithm asides from the very high level concept.
Regarding AlphaGo, though Google has been quite open with the research itself, I don't think they have released any code (at least not the trained version).

Off handed note. I don't recall from where I've read this, but the fact that chess programs are better than GMs now has more to do with better computing power than better algorithms.


text:
What are bottleneck features?
In the blog post Building powerful image classification models using very little data, bottleneck features are mentioned. What are the bottleneck features? Do they change with the architecture that is used? Are they the final output of convolutional layers before the fully-connected layer? Why are they called so?

What are "bottlenecks" in neural networks?

In the blog post Building powerful image classification models using very little data, bottleneck features are mentioned. What are the bottleneck features?

It's clearly written in the link you gave the "bottleneck features" from the VGG16 model: the last activation maps before the fully-connected layers.

Do they change with the architecture that is used?

Sure. The author most likely used a pre-trained model (trained on a large data and now used only as a feature extractor)

Are they the final output of convolutional layers before the fully-connected layer?

Yes.

Why are they called so?

Given the input size to VGG, the feature maps of HxW dimensions are getting twice smaller after every max-pool operation. HxW is the smallest on the last convolutional layer.

First, we need to talk about transfer learning. Imagine you trained a neuronal network over a dataset of images to detect cats, you can use part of the training you have done to work over another detecting something else. That's known as transfer learning.
To do transfer learning, you will remove the last fully connected layer from the model and plug in your layers there. The "truncated" model output is going to be the features that will fill your "model". Those are the bottleneck features.
VGG16 is a pretrain-model over ImageNet catalog that has very good accuracy. In the post you shared, is using that model as a base to detect cat and dogs with a higher accuracy.
Bottleneck features depends on the model. In this case, we are using VGG16. There are others pre-trained models like VGG19, ResNet-50
It's like you are cutting a model and adding your own layers. Mainly, the output layer to decide what you want to detect, the final output.


text:
Who manufactures Google's Tensor Processing Units?
Does google manufacture TPUs? I know that google engineers are the ones responsible for the design, and that google is the one using them, but which company is responsible for the actual manufacturing of the chip?

Google has not released the manufacturing details for their TPUs. However, it's suspected that they're produced by either Taiwan Semiconductor Manufacturing or GlobalFoundries, as these are some of the largest companies in the industry.
Source


text:
How can I use a trained CNN to predict a new image label?
I was applying this CNN fine-tuning example from Matlab.
The example shows how to fine-tune a pre-trained CNN on letters to classify images of digits. Now I would like to use this new fine-tuned CNN on new images of digits that i have on my computer. How can I do that?

Are you basically asking how to do what apparently is already explained in the link you provided above?

I want to use the newly trained CNN on digits for predicting the label of an image he has not seen before. An image that is not among the test images it was fine-tuned on. I tried using the predict function but it gives me an error.

@F.Lin There is a misunderstanding I think: "An image that is not among the test images it was fine-tuned on" - Test images are by definition images that were NOT used for training the CNN. So, your images should go in place of the variable testDigitData in the example from the link you give. The CNN is trained on trainDigitData.

Thank you Eskapp for the reply. The problem is that i don't have a dataset, actually i only have veriy few images.

As the question has been left unanswered, for future readers of the questions:
The documentation you link gives the answer to your question.
Given the fact you have a pre-trained model as you say:
YPred = classify(netTransfer,testDigitData);

where netTansfer is the pre-trained model and testDigitData is your test image that you want to predict the label.
Your image(s) need to be loaded following the instruction also given in the documentation:
digitDatasetPath = fullfile(matlabroot,'toolbox','nnet','nndemos', ...
    'nndatasets','DigitDataset');

digitData = imageDatastore(digitDatasetPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');

(digitData will become testDigitData if the images that you load are for testing purpose only, which seems to be your case)


text:
Perfect play in information incomplete games
As titled, is there such thing as perfect play (or at least "perfectly optimal") in a game with incomplete information? Or at least a proof as to show why there cannot?
Naively (and seemingly obviously), the answer would be a resounding no, since the agent would be likely be forced to pick between "lottery events".
But in practice (using competitive video games as an analogy), we'd see that players would stick to a meta-game that is well equipped to defend against a majority of events that might happen, given incomplete information. Of course the response to that would be that there probably exists a "hard-counter" for any given meta-game, but if it is indeed the case that the meta-game is the "most-optimal" it probably is the case also that such a hard counter puts the player in an unfavourable position most of the time, thus the "hard-counter" itself is not optimal. Thus we'd likely see that any given first encounter players would still stick to their "optimal meta-game" rather than a hard counter of their optimal play.
A more rigour analogy would be to ask: "Under Hofstadter's notion of superrationality, how would agents play information incomplete games", but I couldn't find any readings on trying to import the notion of super-rationality into information incomplete games.
Alternatively: is there such thing as a "perfectly optimal meta-game"?

How deeply have you researched Game Theory?  (I'm asking for context.)  Even in games with incomplete information, whether "deterministic" (Battleship) or involving "quantum indeterminacy" (Prisoner's Dilemma), there are optimal strategies (for instance, binary search extended to a 2D model in Battleship, and minimax in simultaneous games.)  Even Rock, Paper, Scissors seems to have an optimal strategy: https://arstechnica.com/science/2014/05/win-at-rock-paper-scissors-by-knowing-thy-opponent/

Great question, btw!  This is a central issue in my own research.

I have little experience in game theory asides from reading a few things here and there. Regarding your examples, I was curious if there was any approaches to the problem in a more general sense

Ok. I'm going to attempt a formal answer.  Mega-props for bringing up Hofstadter, btw.

Id read this paper out of the University of Alberta: http://science.sciencemag.org/content/early/2017/03/01/science.aam6960.full

@JadenTravnik Thank you for the reference, but I am asking in regards to perfect play in a philosophical sense. In a sense the paper you mention is "better than human" but not "super-rational".

Super-rationality is distinct from perfect play and connotes a meta-strategy that may be counter to optimal strategy from a minimax perspective. Right now, this is still in the realm of metaphysics, but that does not mean it will always be so (if we're lucky, that is, b/c it may have implications regarding the survival of our species in a physical, material sense, i.e. "cooperate or betray?" when dealing with AGI much smarter than ourselves, and also in regard to human vs. human "play" in the sense of competition i.e. partisan contexts)

For context, I guess perhaps this paper would be example of "super-rational play" under perfect information? https://intelligence.org/files/ProgramEquilibrium.pdf

@colourincorrect I've only had a chance to scan this paper, but it look really interesting.  Thanks for linking.  I'll let you know if I have anything useful to say about it.  Also, I've made a small correction in my answer, and tried to explain why I take the route I do.  PS respect for choosing such a mind-bendingly difficult problem to contemplate. Part of the reason I focus on non-chance, perfect information games is to not have to deal with this subject.

Ok, I think I have the concise answer for you.  (Lots of convolutions to get there, including creating an analyzing the equilibrium in an iterative Dilemma;)  I think the tension here is between objectively optimal outcome in terms of max benefit, and limiting downside in a condition of indeterminacy, which might be though of as subjectively optimal.

4 years later: perhaps any game that can be equivalently expressed as an optimization task over some probability function (discrete or not) can be claimed to be "maximally solved" under conditions of incomplete information, as you are literally maximizing expected value. the only question then, would be on how to formulate it (and computationally tractably)

This may be an evolving answer, because the question is, in some sense, a (useful) rabbit hole. I apologize if I don't go deeply into meta-games per se, as it's a little outside of my scope, which is non-chance games of perfect information, but I think it's worthwhile to think about the underlying problem of indeterminacy in relation to games in general.
Bounded Rationality* is a useful concept because it pre-supposes a condition of computational intractability. Computational intractability can be introduced into games in several forms:

Complexity
Hidden Information
Randomness ("quantum" indeterminacy)

[For more details on my use of "quantum" in regards to randomness, see Deterministic Games.]
The underlying purpose of game theory is to determine "optimal" strategies for any given problem.  I put optimal in quotes because optimality is a spectrum, and subjective in a condition of computational intractability.
Thus, we cannot know if AlphaGo plays optimally, only that it played more optimally than Lee Sedol in 4 out of 5 games.
This is distinct from strongly solved games such as tic-tac-toe, where we can know with total certainty that a choice is optimal, because the problem of tic-tac-toe is computationally tractable.
Part of the confusion may be semantic, because the concepts are subtle and profound, and require language, what TS Eliot might have called "the intolerable wrestle with words and meanings." (For instance, I used hidden information above to avoid having to distinguish between incomplete and imperfect information.)

Perfect Play is generally defined as a strategy that leads to the best possible outcome for a participant, regardless of the choices of the opponent.

Thus minimax is of central importance, and provided the foundation for game theory.
Even in games with incomplete information, whether "deterministic" (Battleship) or involving "quantum indeterminacy" (Prisoner's Dilemma), there are optimal strategies. For simultaneous games such as Dilemma and all of the numerous extensions minimax is used.  In Battleship, there are at least three strategies of increasing optimality, and although there doesn't appear to be a strategy that can yield P > .5, if one player employs a more optimal strategy, they will win in aggregate. Even Rock, Paper, Scissors seems to have an optimal strategy, which blows my mind, and carries the caveat that I need to look into it more.

Thus, perfect play, as defined, is certainly achievable, but does not necessarily connote (objectively) optimal choices, which is a little confusing, because "perfect" implies objectivity, a condition which is only possible in regard to tractable problems.

It is also important to note that there may not be a "winning" strategy in the sense of being better off than the opponent, and in this condition, perfect or optimal play is mitigation of loss.

*In terms of incomplete information games specifically, I think there's a case for extending the concept of Bounded Rationality is extended to include information that cannot be observed or "known".
Colloquially, this would include the "unknowns" (both known and unknown) and the "unknowable" (quantum indeterminacy and superpositions).

I see.... correct me if I am wrong, but are you basically saying that right now we do not even have an rigour analytical framework to trace the problem?

@colourincorrect Quite the opposite. The framework has been rigorously defined and continues to be validated on problems of increasing intractability. Perfect play is not objectively perfect, just more optimal than the opponent's play. But superrationality is still new and requires more research and validation from a mathematical (as opposed to metaphysical) perspective.  Superrationality is being studied in the context of games of imperfect information, such as Prisoner's Dilemma, and if validated, that validation will be mathematical.  Right now it is an hypothesis.

"...we do not even have an rigour analytical framework to trace the problem" as in in terms of super-rationality, and an "objective utility function" to "correctly evaluate" situations I ask

@colourincorrect Game Theory is that framework. There are absolutely strategies for dealing with different levels of indeterminacy, such as P < .5  In simultaneous games, P can never objectively be > .5 , because of hidden information (i.e. the most basic form is a binary Schroedinger box) Thus no rational agent plays the lottery to win, unless the odds are flawed in a favorable way, which does rarely occur and has been, famously, exploited by mathematicians.

@colourincorrect It is actually a good idea to look at Blackjack, and more recently, Poker for an understanding of how strategy changes if probability can be raised above .5.  Even 50.000000000000000000001% probability will lead to gain given a sufficient number of hands.  The trick is to determine if probability of a given event is truly above this threshold, which also involves confidence levels (analysis of the dataset to reduce margin of error for a sufficient confidence level, such as with a point estimate.)  Recent winning strategies for poker are a function of mathematics.

This second answer attempts to address perfect play in relation to incomplete information specifically.
An element in the difficulty in answering this question may be that the concept of perfect play is widely applied to solved games in the domain of Combinatorial Game Theory as opposed to strictly economic Game Theory.
In relation to games with incomplete information:

Perfect play, defined as the best possible choice, without regard to the opponents choice, may be achieved in games with incomplete information

It's important to note that perfect play may not result in a win.  In tic-tac-toe the result is a draw.  In certain games, for a disadvantaged player, it may result in the "best" possible loss.

Perfect play in classic Prisoner's Dilemma is the minimax strategy.

The conundrum is that in this model, it does not lead to the optimal outcome, only the optimal outcome without regard to the other agent's choice.
In classic Prisoner's Dilemma, the supperational strategy is more risky because there is no information on the other agent (probability for either choice is always 50%) and it doesn't limit downside.

Superrational strategies can be shown to be mathematically supportable by extending Prisoner's Dilemma to iterative and cyclic variants. This is partly because in iterative variants, choice are a form of communication between agents. However, the superrational strategy may not be a winning strategy, as the motive of the superrational agent may be said to be maximization benefit, as opposed to limiting downside exclusively. In in iterative Prisoner's Dilemma, the superrational agent may have to sacrifice a couple of iterations (turning the other cheek) in order to incentivize the rational agent to change strategy and cooperate, and to determine if the other agent is irrational, in which case the superrational agent may switch to the rational strategy of minimizing maximum downside and maximizing minimum benefit.
In classic iterated Dilemma, choices are the exclusive form of communication between agents, and each choice becomes part of a dataset on the other agent's decision making. Information is still incomplete, but less incomplete with each iteration.
Superrational strategies for games of incomplete information then become viable via statistical analysis.

It depends on the game. In zero sum non cooperative games yes, there's always a GTO strategy.
The easiest example is Rock, Paper, Scissors, where playing 1/3 of each randomly would be the only optimal strategy. In this case a break even one too, in some games though GTO has a positive expected value against any strategy that's not GTO itself.
Usually online video games strategies and metas are heavily based on adaptation to population tendencies though, which in of itself it's not perfect play, but it can have a better expected value than perfect play against a non optimal opponent.

apologies for the late response: i didn't see it. is there a source for this? i think i agree with your claim, but wouldn't information incompleteness also imply that the payoff matrix is unknown? wouldn't that affect play?

two years later: if the payoff matrix is unknown, we can still use maximum likelihood estimation to calculate the optimal strategy, and bayesian techniques to compound (multiply) different distributions together. however the computational difficulty of this technique compounds /very/ quickly, and many games do not lend itself well to modelling like this.


text:
Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning?
Learner might be in training stage, where it update Q-table for bunch of epoch.
In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by random action rate.
After some epoch, when reward is getting stable, let me call this "training is done". Then do I have to ignore these parameters(gamma, learning rate, etc) after that?
I mean, in training stage, I got an action from Q-table like this:
if rand_float < rar:
    action = rand.randint(0, num_actions - 1)
else:
    action = np.argmax(Q[s_prime_as_index])

But after training stage, Do I have to remove rar, which means I have to get an action from Q-table like this?
action = np.argmax(self.Q[s_prime])

Epsilon (your rar parameter) is used to have the possibility of a random action which handles the problem of exploration and exploitation.
If you have a deterministic world and are certain that your learner has learned a policy with performance to your liking, removing it will ensure that only the greedy policy being learned will be taken and that you can expect the same performance afterwards.
The agent will still be learning at this point because alpha (learning rate) is not set to 0. Because of this, if the world is stochastic instead and possibly changes slowly over time (e.g. motor speed due to battery power), then having epsilon non zero still allows the agent to explore and overcome these changes when they happen before it's performance gets low enough to change is behaviour (e.g. q values dropping).
If instead you simply want to test the performance of the policy learned up until a timestep, setting alpha to 0 in a compares the epsilon-greedy policy, setting both alpha and epsilon to 0 compares the greedy policy.


text:
Successful methods for optical character recognition?
Alright, I want to write a mobile app that lets you take a photo of your equation, detects the equation, transforms it from pixels to text and then solves if it's possible.
Right now, I am doing the part where I receive a photo (2D array of pixels) and what to output a resized part of this photo (the part has a rectangular shape) that just contains the equation and some pixels around it.
Basically, I need a model that takes a 2D array and returns a resized version of it.
Since I am new to AI, I don't know what techniques have been successful in optical character recognition.
Any suggestions?

I suspect you were down voted because the question is so broad, in the sense that you "don't know what to use and how to use it".  You might try to rephrase to ask something like "What techniques have been successful in optical character recognition" and break out advice on where to learn about those techniques into a separate question.  Welcome to AI!

@DukeZhou Thanks a lot, changed my question!

This is not meant to be a direct answer, as OCR is not an area I have experience in, but I want to present a deeper problem, that of mathematical notation.
Mathematics has "exploded" in the last few centuries, and notation is not standard across various sub-fields.  Check out this question on notational change with integrals. In some cases, individual sub-fields have alternate forms of notation.
My suspicion is that this task will be enormously difficult without a great deal of human guidance, so I'd point you towards AI methods that utilize "human-assisted machine learning".

Well, since it is kind of my first 'real' AI project, I only want it to recognize and solve some basic equations like polynomials of degree 1-3, do you think tesseract can do this task?

I can't really give you advice on that b/c my own AI endeavors are quite distinct.  From what I've read, Tesseract seems to have a good reputation as a "bare bones" engine and it's free.  For more specific advice, you'll need to wait until someone with direct experience weighs in. But it never hurts to just jump in if you're passionate.  You'll probably fail a number of times before you succeed, and you'll learn from every step.


text:
Negative counterfactual regret
I am reading the paper Regret Minimization in Games with Incomplete
Information on CFR algorithm.
On page 4, the paper defines $R^{T,+}_{i,\text{imm}}=\max\{R^{T}_{i,\text{imm}}, 0\}$ after equation (5). I am confused why it is necessary? It seems to me that since in the definition of $R^{T}_{i,\text{imm}}$ the regret is computed with respect to the optimal action.

As everything is in expectation, is mixed-action going to make any difference?

Isn't $R^{T}_{i,\text{imm}}$ always nonzero already?


text:
Translate product names with AI
I frequently need to translate product names for hundreds of similar products -- and I have a list of past product names. Is it possible to train AI to review past translations and translate? It doesn't have any special grammar, simply the name (with some industry-specific usage that a general machine translator can't do.) What would I need to do to get started?

One of the big problems here is if the product names are idiomatic or involve wordplay. This feels like a very difficult NLP problem.  By contrast, technical names would be fairly easy.  It seems to me that translating product description accurately would be more important than translating product names.

They are unique. i.e. One word in Chinese always translates to one word in English. However, the way the words are ordered and splitting up Chinese text into 'words' require some learning (not a straight-forward find/replace) that's why I thought some kind of pattern-matching and learning can solve this.


text:
How to predict a preferred route based on weather and distance
I want to train a neural network to predict what my favourite home-work route will be for a particular day. I have these features for routes on a day:
temperature, humidity, congestion, distance, duration

I have come up with this concept of training/testing a network:
//
// training features result in route 1,2 or 3
//
Network.train([30,10,12,20,12] , 1)
Network.train([20,10,22,20,14] , 3)
Network.train([23,10,2,20,10] , 2)
Network.train([20,10,22,20,12] , 2)

//
// On a new day, predict which route the user is most likely to take:
//
var route = Network.test([25,8,12,22,12])

My question is: is this a viable approach? Can I make relevant predictions this way if I have enough training data?
Can I generate an outcome between 1 and 3 this way?

Thinking out loud: perhaps distance and duration shouldn't be taken into consideration since they are actually part of the route