text:
Effective algorithms for OCR
I am using Google's OCR to extract text from images, like receipts and invoices.
Whare examples of techniques used to make sense of the text? For example, I would like to extract the date, name of the business, address, total amount, etc.
Before marking this question "too broad", if someone can please direct me to the right set of algorithms the industry uses for machine learning will be great.

This is an old question, but it's unclear (to me at least) whether you're asking for 1. OCR techniques or 2. techniques that "give a meaning" to the extracted text (which was extracted with some OCR technique). From the title, it seems that you're looking for algorithms that solve the OCR problem (so it should be 1), but from your comment below under one answer it seems that you don't just want to perform OCR but you somehow want to associate a meaning with the extracted text. So, can you please clarify exactly what you're asking here?

To make things even worse, you say that you're using Google's OCR, which makes me think that you're not looking for OCR techniques, because, otherwise, why would you care about specific OCR techniques if you already have one? You should probably have also linked to the tool/library you're referring to when you say "Google's OCR"

Last semester I, along with my team, made a project on OCR.
Note: I am assuming the data set for your pictures has white
background with black (or some other dark text) on it.
These are the overview steps we followed:

Pre-processing includes grayscale conversion, noise reduction,
binarization and skew detection.
Next step was segmentation. This process extracts the individual
characters from the image. Histogram taken along the y-axis divided
the image into lines. This is followed by histogram along the x-axis
which divided them into words and further into characters.
At the end of the step, we used Savgol filter to smooth the curves of the histogram.
Next step was feature extraction. This is the most important step.
The accuracy of your code depends on how well your features are.
We used the following features:

Crossing: Counting number of transitions between foreground and background. We used
two diagonal lines, two horizontal and one vertical line. You can used any number you want.
Zoning: Whole character region is divided into 16 zones, and density of each zone is measured.
Projection Histogram: Each character has unique (almost) vertical and horizontal histogram signature.
Other features include number of endpoints in the character, number of loops and horizontal/vertical line count.

We used three different classification algorithms for our project. They were KNN (K-Nearest Neighbours), Artificial Neural Network (ANN) and Extra Tree classification. Their F1 score was 0.84, 0.82 and 0.77 respectively.
For training, you will need to find datasets. Many data sets for OCR was available online. Make sure you are using good ones.

Remember we are talking about finding relation between data we extracted from a document.

An interesting question, I think the algorithm used for OCR is "Logistic Regression" or "Decision Tree" in multiple steps.
The steps can be

Image Classification - In this step, the images are classified into
"with or without" text.
Text Detection - In this step, the images with text are taken
divided into blocks and the blocks are classified into "with or
without" text.
Character Detection - In this step, the blocks with text are taken
and divided into smaller boxes of single characters and compare with
a database of characters.

The database is built using the crowdsourced "captcha" project.

Can you please explain with an example.


text:
Do off-policy policy gradient methods exist?
Do off-policy policy gradient methods exist?
I know that policy gradient methods themselves using the policy function for sampling rollouts. But can't we easily have a model for sampling from the environment? If so, I've never seen this done before.

An extension to this would be - Can we leverage experience replay concept in policy gradient algo? I understand the answer is No because we are not talking of an off policy update, but are there alternatives or workarounds

Absolutely, it’s a really interesting problem. Here is a paper detailing off policy actor critic. This is important because this method can also support continuous actions.
The general idea of off-policy algorithms is to compare the actions performed by a behaviour policy (which is actually acting in the world) with the actions the target policy (the policy we want to learn) would have chosen. Using this comparison we can determine a ratio ($0 \leq \rho \leq 1$) which can scale the update to the target policy by the probability of the target policy taking that action. A higher $\rho$, the more alike the two policies are, and this increases the magnitude of the learning update for the target policy for that step. A $\rho$ of $0$, and the update is ignored.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Is robotic process automation related to AI/ML?
I am trying to understand if robotic process automation (RPA) is a field that requires expertise in machine learning.
Do the algorithms behind RPA use machine learning except for OCR?

In the context of IT systems, "Robotic Process Automation" (RPA) is a term often used to describe a technique where software systems are integrated or work processes are automated through the existing user interface of the applications rather than writing new software to provide integration points.
In that context, RPA has nothing to do with AI or machine learning. In most cases, it does not even require OCR.
For an example of a common use case, let's say you have an old mainframe IT systems for tracking your subscriptions and a new website to let people order subscriptions from their phone.
In this case you might create an "RPA" job that opens the list of new subscription requests from the website and for each of them  opens the old application, clicks of the "new subscription" button, click on the "Customer Name" field, pastes the name, clicks "Customer address" and pastes in the address etc.
In some cases, the RPA job will be exposed as a service with an API that can be called by the new application, so it can dump data directly into the old application. The benefit is that it can do this without any changes to the old application.
It is attractive because defining the steps of copy this, click that can often be defined in visual tools by non-programmers very quickly and at much lower cost than setting up a systems integration project to connect the two systems and because integration through the existing user interface does not require any changes to the application.
In this way, it is similar in spirit to how Excel allows non-programmers to automate calculations by writing formulae and thus automating their spreadsheets.
You will often see RPA proponents putting some AI buzzwords into their presentations but from what I have seen in industry RPA is mostly just a visual scripting technique that is easy to learn and easy to apply.

Yes, rpa is AI. In particular, applied AI.
The definition of rpa in wiki is:

Robotic process automation (or RPA) is an emerging form of clerical
  process automation technology based on the notion of software robots
  or artificial intelligence (AI) workers.

Moreover, look at this characteristics of rpa:

The paradigm, in summary, is that a software robot should be a virtual
  worker who can be rapidly "trained" (or configured) by a business user
  in an intuitive manner which is akin to how an operational user would
  train a human colleague.

If we talk about training or learning, we talk about AI.

Robotic Process Automation (RPA) is linked to but separate from AI and ML. RPA is the process of automating repetitive, rule-based processes with software "robots." While it lacks actual intellect and learning ability, it can mimic human activities. AI and ML, on the other hand, are more sophisticated, including the creation of algorithms that can learn from data and make judgments. RPA may be improved with AI/ML to execute cognitive tasks and manage unstructured data. RPA is a subset of AI/ML, concentrating on process efficiency rather than cognitive thinking and adaptability.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How to implement an "unknown" class in multi-class classification with neural networks?
For example, I need to detect classes for MNIST data. But I want to have not 10 classes for digits, but also I want to have 11th class "not a digit", so that any letter, any other type of image, or random noise would be classified as "not a digit". Similarly, with CIFAR-10, I want to have the 11th "unknown" class to classify any image that contains something out of the available 10 classes of CIFAR-10.
So, how to implement such a feature? Maybe there are some examples somewhere, preferable with Keras.

hello, did you get solution for this?

future readers may find zero shot learning interesting

The usual way to implement this would be to add the new class with data examples.
Some things you need to address:

Sourcing new data for your "other" class.
Ensuring the amount and variation of data in "other" class examples matches how the predictor will be used.

Code examples for this are not necessary, as you would just use the same network design as you already have and just add another output. This is a data and model definition problem.
Logically you have another option: As well as outputting the predicted class, you predict separately whether there is any detectable object at all as a true/false value. This still requires the additional data, but is for example how the YOLO algorithm works for object detection. Object detection has a specific meaning - it involves finding the co-ordinates and class of possibly multiple objects in an image. This goes beyond the wording of your question, but is a typical end goal if you are asking this kind of question.
YOLO predicts the presence of an object separately from the class of object. The additional data for YOLO training comes from segmenting the source images, so many parts of the target image are background with no objects. In that case the additional data you require is due to more detailed labelling within each image example.
YOLO is quite complicated architecture, so you might want to look at this example using Keras on a Github project for more details, if object detection is your goal.

"As well as outputting the predicted class, you predict separately whether there is any detectable object at all as a true/false value." Thanks for this! Combining this as a first step to my existing model has decreased the noise significantly!

If you are using a softmax distribution for your classification, then you could determine what your baseline max probability is for correctly classified samples, and then infer if a new sample doesn't belong to any of your known classes if its max probability is below some kind of threshold.
This idea comes from a research paper that does a much better job of explaining the process than what I just said: A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks


text:
Is it a good idea to train a CNN to detect the hydration value (percentage) in skin images and evaluate it with the MSE?
I have a large dataset of skin images, each one associated with a hydration value (percentage).
Now I'm looking into predicting the hydration value from an image. My thinking: train a CNN on the dataset and evaluate the model with a mean square error regression.
First, does this sound like a sensible way to try this?
Second, I'd like to run the model on mobile. Can you recommend any examples with Caffe2 (or alternatively TensorFlow) or diagrams that might explain a similar task?

Your initial idea seems about right. Before creating your own classifier you might want to try transfer learning, using some pretrained network like VGG16 that is incorporated in most of machine learning frameworks.
As to inference on mobile devices, TensorFlow offers some tutorials in this subject:
https://www.tensorflow.org/mobile/tflite/


text:
Why do non-linear activation functions that produce values larger than 1 or smaller than 0 work?
Why do non-linear activation functions that produce values larger than 1 or smaller than 0 work?
My understanding is that neurons can only produce values between 0 and 1, and that this assumption can be used in things like cross-entropy. Are my assumptions just completely wrong?
Is there any reference that explains this?

This discussion might be helpful

Why wouldn't they work?
Each neuron's output is equal to a function over the sum of all its weights multiplied by their corresponding neurons. If that function is the Sigmoid function, then the output is squashed from $[0,1]$. If the entire layer uses a SoftMax function, then the output of all neurons is squashed from $[0,1]$ and their sum equals 1. In other others, they represent a set of probabilities, where you can then use cross-entropy to optimize their values (cross-entropy measures the difference between two probability distributions).
ReLU and ELU are simply other types of functions, whose output is not limited to the range $[0, 1]$. They are differentiable, like other activation functions, and so they can be used in any neural network.

I think the OP was confused about the fact that the cross-entropy may require a probability vector as input, so, by using activation functions that do not have values in the range $[0, 1]$, the output of the neural network may not be a probability vector. You partially address this by mentioning the softmax, but I think this answer could definitely be improved by explaining more in detail the cross-entropy, and it's relation to the softmax and other activation functions of the hidden neurons.

Christopher Olah's blog post describes it better that I ever could.
Basically, most data we come across can't be separated with a single line, but with some kind of curve. Non-linearities allow us to distort the input space in ways that make the data linearly separable, making classification more accuarate.

This does not really answer the question. This answers another question, which is "Why do we need non-linearities?". Please, read the actual question again.


text:
What are the connections between ethics and artificial intelligence?
What are the connections between ethics and artificial intelligence?
What are the issues that have arisen, especially in the business context? What are the issues that may arise?

Welcome to AI!  I would suggest searching this Stack for questions related to ethics.  You may also be interested in this recent question: Few implementations in Machine Ethics?  See also: https://ai.stackexchange.com/questions/tagged/ethics

The connections between ethics and artificial intelligence can be divided into five major categories, and other categories may form over time.

Correlations between ethics and artificial intelligence
Existential impacts of artificial intelligence on the human experience
Threats to current ethical social, economic, and legal standards arising from artificial intelligence research and application
Uses of artificial intelligence to breach ethical standards without detection
Uses of artificial intelligence to detection of breaches of ethical standards and assist in remedial action

Since the most important in the long term is the most likely to be dismissed by those with normal perspectives about ethics and AI, the four will be addressed in reverse order
Automatic Detection and Remedial Action
The pattern recognition capabilities of existing AI systems and sub-systems is already employed to detect a variety of ethical breaches.

Securities misconduct, including insider trading
Breaches of anti-trust law, including conflicts of interest
Employer misconduct, including inequality in hiring
Tax evasion
Misuse of non-profit funds

Remedial actions may be the opening of a case with the automatic generation of a notice to those in potential breach.
Smart Organized Crime
Although much detail could be included here about detection avoidance in crime using AI, it may not be socially responsible to include such in a global public facing site.
Threats to Economies and Individuals
As with any high impact technology, disruption is a possibility.  This was true of fire, irrigation, the wheel, bronze smelting, gun powder, typesetting, steel-working, engines, textile automation, alternating current power distribution, aeronautics, petroleum refining, electronics and radio transmission, pharmacology, terrestrial nuclear reaction, and the Internet.  Genetic engineering and artificial intelligence are next in line.
What ethical conventions will likely be impacted?

Distribution of employment roles, the change of which may not match distribution of educational preparation
Distribution of wealth, favoring prowess in highly automated business
Mutual exclusivity of personal privacy and the use of technology
Obscurity of totalitarian control (such that common citizens may be more like cogs in a machine than during industrialization)
Changes in the balance of world power
New forms of asymmetric war, such as cyber-war and autonomous combatants

All of these have either direct or direct impact on the viability of business options and how business may be conducted.
Existential impacts
Some may consider dominion over the earth as an ethical grant to humanity.  Others may equate the soul with the cognitive and self-aware aspects of only one species on earth.
We already have a world that is sufficiently disconnected on an existential plane that many consider their cats or dogs as more important than any human.  When people are more connected to their intelligent agents than their pets, family, and friends, that may qualify as an ethical impact.  Others may see it as a psychological impact.
Realistically, it is ontological.  What is a human to think of her or his only purpose when it becomes questionable whether homo sapiens is simply a link between DNA based intelligence and some more capable species the reproduction of which has been decoupled from DNA coding.
Replacement of jobs have caused changes in what families wish for their children.  What will be the impact when few job roles (or eventually none) exist where artificial employees don't exceed their human counterparts in effectiveness?
If humans cannot adjust to the idea that the sole purpose of life has nothing to do with practical provision of water, food, shelter, clothing, and essential products and services, there may be systemic depression.  Conversely, leisure may become the reality for all humans, leaving ethics and the finite nature of DNA based life the only two concerns of humans.
Correlations Between Ethics and AI
This is the most unpalatable of categories to examine when the examiner is human.  It is possible that artificiality may be an ethics progenitor.  The limitation of humans as ethical beings is well documented.  It is possible that AI may be more ethical than its developers.
Will a group of AI systems be able to arrive at method for distribution of power and a standard for global trade that is as good as or better than humans have been able to negotiate and then police each other in a way that leaves no possibility of undetected breach of treaty?

In a business context, there are issues surrounding the implementation, the implementors, the other employees, the business entity itself, and the customers. These stem from data used, risks inherent in an implementation, like unknown errors bugs or algorithms without human checks, behaviour change of impacted stakeholders, job losses, reputation impacts on the company, etc.
There's a lot to think about AI and ethics. It could be seen as a combination of a broad set of topics including computer science, humanities, economics, and philosophy.
I run a podcast on some of these issues http://machine-ethics.net/ let me know if there's anything you want to be discussed or someone you would like to hear.

Great answer.  (Jealous I didn't think of this angle;)  This is a massive issue as data mining becomes ubiquitous and a major source of revenue for many companies.  Welcome to AI!

This is a good related read from Nature: There is a Blind Spot in AI Research

Fears about the future impacts of Artificial Intelligence are distracting researchers from the real risks of deployed systems

I'd strongly recommend looking into Game Theory's relationship and impact on AI. Prisoner's Dilemma is a good place to start, because optimality can have repercussions.
With computing in general, optimization is a major goal.  For AI, optimal decision-making is what it's all about.  But sans humanity, this may prove to be problematic.

(Apologies for the brevity--I'll be returning to elaborate since this is a subject of personal preoccupation--but I wanted to leave you with a few tidbits in the meantime. :)

There's an interesting new paper here: Algorithmic Decision Making in Financial Services
This paper highlights the correlation between corporate ethics and interests with the normative issues arising in respect of algorithmic decision-making.

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.


text:
What is the relation between back-propagation and reinforcement learning?
What is the relation between back-propagation and reinforcement learning?

Backpropagation is a subroutine often used when training Artificial Neural Networks with a Gradient Descent learning algorithm. Gradient Descent requires the computation of the error gradient, i.e. derivatives, of a cost function with respect to the network parameters. BP allows you to find this gradient a lot faster than using naive methods.
Reinforcement Learning refers to inferring "optimal" behavior, i.e. a strategy, of an agent maximizing some goal in an environment. Depending on the specific RL algorithm, BP may be employed to adjust the parameters of a function approximator used to represent aspects of the environment and or the agent.


text:
Should the mutation be applied with the hill climbing algorithm?
As far as I understand, the hill climbing algorithm is a local search algorithm that selects any random solution as an initial solution to start the search. Then, should we apply an operation (i.e., mutation) on the selected solution to get a new one or we replace it with the fittest solution among its neighbours?
This is the part of the algorithm where I am confused.

In general, hill climbing algorithms select a random initial solution, then takes the best move available after evaluating all possible operations available. The possible operations are determined by the search operators (which you determine and is dependent on the problem setting). So by your words, you should 'replace it with the fittest solution among its neighbours'.
Mutation can be applied as a search operator, but this varies by degree and context. For example, in structure learning for bayesian networks, a random sequence of search operators are sometimes applied to escape local optima during the search process.
See https://www.cs.helsinki.fi/u/bmmalone/probabilistic-models-spring-2014/StructureLearning.pdf - slide 9 for random restarts


text:
Is there an AI system that automatically generates classes and methods by giving it voice commands?
I want to develop (in Java) a voice plugin for Eclipse on a Mac that helps me jot down high-level classes and stub methods. For example, I would like to command it to create a class that inherits from X and add a method that returns String.
Could somebody help me point out the right material to learn to achieve that?
I don't mind using an existing solution if it exists. As far as I understand, I would have to use some Siri interface and use nltk to convert the natural text into commands. Maybe there's some chatbot library that saves me some boilerpate NLP code to directly jump on to writing grammar or selecting sentence patterns.

While you can use NLTK for analyzing and parsing the text obtained from the speech to text interface (e.g. Siri), there are higher level APIs available for this. The class of problem you are trying to solve in NLP is "intent detection".
There are several open source and commerical APIs available for this including Amazon Alexa, Google Cloud Natural Language, Azure, as well as libraries like RASA NLU, etc.
The high level flow of your program will be:

Record/receive spoken audio
Convert audio speech to text
Detect intent of the text command using an intent detection library
Use the intent to feed a script/automation that generates the code in your IDE

you could implement a simple TTS system that can translate your voice line by line to code , but it would of no use . you cant express code in a line-by-line manner. Coding is a highly iterative process at first you come up with a rough sketch for which you add details later on , and from NLP point of view this is a highly ambitious project .
At the heart almost all of
ai techniques (neural networks) are functions that map one domain to another , you cant map natural language sentences to instructions in code.
However you can implement a tts system for a small language like LOGO.

TTS = text-to-speech; you presumably mean ASR (automatic speech recognition) or STT (speech-to-text).


text:
In YOLO, what exactly do the values associated with each anchor box represent?
I'm going through Andrew NG's course, which talks about YOLO, but he doesn't go into the implementation details of anchor boxes.
After having looked through the code, each anchor box is represented by two values, but what exactly are these values representing?
As for the need for anchor boxes, I'm also a little confused about that --
As far as I understand, the ground truth labels have around 6 variables :

$P_o$ checks if it's an object or background,
$B_x$ and $B_y$ are the center coordinates
$B_h$ and $B_w$ are the height and width of the box
$C$ is the object class, which depends on how many classes you have, so you can have multiple $C$

As for creating the bounding box,
$B_h$ is divided by 2, with one half from the center points ($B_x, B_y$) to the top, and the other half to the bottom.
If we train our classifier, wouldn't the prediction boxes be close to the ground truth labels as training progresses? So, if our ground truth label has a high height, small width as boxes for some images, and low hight and large width for other images, wouldn't our classifier automatically
learn to differentiate between when to use one over the other, as it is being trained? If so then what is the use of anchor boxes? And what are those numbers representing anchor boxes representing?

Please, next time ask only 1 question per post. If you have multiple questions, even though they are about the same model or topic, split them into multiple posts, so that people can focus on 1 question/problem at a time.

Not an pro but I think I know some answers to your questions.

If we train our classifier, wouldn't the prediction boxes be close to the ground truth labels as training progresses

I think that's what YOLO v1 did. According to Andrew NG's video the bounding boxes are introduced to solve multiple objects inside the same grid cell. And according to this post anchor boxes assignment ensures that an anchor box predicts ground truth for an object centered at its own grid center, and not a grid cell far away (like YOLO may)

what are those numbers representing anchor boxes representing?

They are just width and height (shape). In YOLO v2 it is used to compute IOU assuming all boxes are placed at the same location (ignoring the location), you could think of it just tries to match the shape. And it uses (1-IOU) as the distance when applying the K-means clustering.


text:
Can I do deep learning with the 1060 or the 1070 ti?
Before I start, I want to let you know that I am completely new to the field of deep learning! Since I need a new graphics card either way (gaming you know) I am thinking about buying the GTX 1060 with 6GB or the 1070 ti with 8GB. Because I am not rich, basically I am a pretty poor student ;), I don't want to waste my money. I don't need deep learning for my studies I just want to dive into this topic because of personal interest. What I want to say is that I can wait a little bit longer and don't need the results as quickly as possible.
Can I do deep learning with the 1060 (6GB seem to be very limiting, according to some websites) or the 1070 ti? Is the 1070 ti overkill for a personal hobby deep learner?
Or should I wait for the new generation Nvidia graphics card?

Tim Dettmer's wrote a few articles about which hardware he recommends to buy for which purpose, see his recent one here: https://timdettmers.com/2019/04/03/which-gpu-for-deep-learning/ . If you don't want to read the entire article, look at the "TL;DR advice" section.

Regarding specific choices I can't recommend, but if you are completely new, you should probably learn/code some more until you get a GPU. There is a lot to learn in machine learning before GPU speedups make a significant difference, and until then doing the computations on any old CPU would be just fine, especially if you are just starting since you won't be doing anything too complex. You will know when computational resources are your main bottleneck, and until then it shouldn't really matter too much.
Or, you could also rent computing power from say, AWS or Google

Given that you're a student doing this out of personal interest and wanting to do some gaming on the side, I'd suggest the GTX 1060 6GB since at present the GTX 1070Ti is overpriced due to crypto miners (this will date the answer, but for reference the 1060 is going for ~GBP340, the 1070Ti for ~GBP600; two other options are the 1050Ti 4GB for ~GBP160 or the vanilla 1080 at ~GBP650).
'Which GPU...' by Tim Dettmers is very helpful, as is 'Picking a GPU...' by Slav Ivanov, especially the summaries at the end for different use cases. As you're not looking at spending a huge amount of money, the 1060 seems like a good compromise as the 1050Ti might just leave you with a disappointing gaming experience. Finding a used 1070 is also suggested, but you'd need to be comfortable with that.
Other answers have mentioned the cloud, but that doesn't help with your gaming. If you want to save some cash while you're waiting for the next gen of cards, take advantage of your student status on AWS educate or Azure on MS Imagine - the GitHub student dev pack is a good package.

I don't think you need to invest in any kind of GPU unless you're familiar with the computations required for the task you want to achieve using deep learning.
Also, by the time you've sufficiently mastered Deep Learning to a point where you can actually make the most of your GPU, there will be new products in the market.
So until then I suggest you use your CPU for doing little tasks such as Regression etc.  You can always use the free credit offered by the various cloud companies for your tasks

You make a good point about holding off on hardware purchases b/c newer, more powerful products continuously emerge, which has the effect of reducing the price of previous products.  The longer you wait = more bang for your buck!


text:
Is there a limit to the increase of intelligence?
Some argue that humans are somewhere along the middle of the intelligence spectrum, some say that we are only at the very beginning of the spectrum and there's so much more potential ahead.
Is there a limit to the increase of intelligence? Could it be possible for a general intelligence to progress infinitely, provided enough resources and armed with the best self-recursive improvement algorithms?

To have a "maximum achievable intelligence", first of course you have to define "intelligence" well enough to be able to rank things by intelligence. There is no widely-supported theory that is able to do so.
You might like to look into AIXI as described by Marcus Hutter in a video lecture. It is an attempt to formalise intelligent agents that attempt to make optimal decisions mathematically. Here is another written introduction. Of course this is only one of many possible frameworks to describe intelligent agents.
One interesting implication is that AIXI implies intelligence - in terms of ability to learn from and exploit an environment - is upper bounded. In principle there is a ceiling due to uncertainty about what the data that a rational agent possesses might infer.
However, this ceiling refers to only specific abilities to extract actionable information from data that the agent has access to in order to solve decision problems. There is an open question about how much data can be collected, stored and processed by any entity, and this ability to acquire and retrieve relevant knowledge would be viewed by many as part of the "intelligence score" when comparing agents.
There are theoretical limits to computation from physics, e.g. some are based on the fact that it fundamentally requires energy, the energy has a mass equivalence, and enough concentrated mass would form a black hole. This sets a high upper bound, and it is likely that real-world structural and design issues will set in way before this limit. However, combined with the above limits on decidability and practical access to data, it does seem there should be a ceiling.

Interesting. Even though the resources may be limited. Won't self-recursive algorithms be repeatedly able to improve themselves infinitely with the higher intelligence being able to find more avenues for improvement? There are also theories of manipulating the radiation from black holes resulting in the most effective computation possible within the limits of current human intelligence.

@Jeevan: Intelligence is still bound by the laws of physics. Yes it is possible that a higher intelligence than humans discovers that our knowledge about those upper bounds is in fact wrong. But no recursing can "beat" the true laws of physics. Your link about manipulating the radiation from black holes I think leads to the same upper bound as considering the maximum computation before forming a black hole (the two seem linked, this is part of the holographic principle in physics that constrains activity within a volume to be equivalent to a transformed version on the surface of same).

Have you considered the possibility of manipulating the laws of physics, say for eg: to network billions of black holes by clumping together all available matter via E-R bridges to compensate for the distance problem. A higher intelligence might think of a more efficient solution than that. But won't rewriting and editing the source code(laws of physics) of the universe be possible with a higher intelligence?

@TransPlanetaryInjection: The idea of super intelligences "manipulating the laws of physics" is outside science. It is science fiction, and more suitable for worldbuilding.stackexchange.com than here. We cannot say whether it is possible or not any more than we can claim to have proof of god's existence or non-existence.

Absolutely, regardless of how you define "intelligence".

If intelligence is merely information, as in "a piece of intelligence", as in data, or an algorithm, the structure is finite.  (Structure, here, refers to the information, which may be reduced to a single string in either case.)

See: Turning Machine.

If intelligence is the rational capability of an automata, it is likewise bounded by the tractability of the decision problem, the structure of the algorithm, and the time available to make the decision.

See: Bounded Rationality
Both answers are really the same, because "intelligence" in the first sense is limited by physical constraints on information density, sophistication of the algorithm, and time.
See also: Computational complexity of mathematical operations, Computational complexity, Time Complexity

The answers previously given are correct for AI which can indeed process more information with more computational power. However, actual reasoning ability like humans have is not defined by Church-Turing. AIXI has nothing to do with human reasoning. A pretty good clue to this fact is that AIXI has been around since 2005 and to date there are no machines based on it that have human-level reasoning. For example, an interesting topic in AI is natural language processing (NLP). I can speak into my Android phone and it will transcribe my speech into text. It seems like an amazing advance. However, this is what a human would do if they heard a foreign language and then did a phonetic transcription of what they heard. Then they looked up a phonetic chart to match the sounds with words. This would take place without any actual understanding of what they were hearing. This is how it works on my phone, much like Searle's Chinese Room.
Humans are quite different because they actually understand words. The equivalent to this in AI would be natural language understanding (NLU). No AI today has NLU and no theory within AI explains how to construct it. There isn't any research on AI NLU because there is no starting point. A fact that most AI enthusiasts don't like to admit is that even the smartest AI systems are routinely outclassed by rats and even six month old babies in terms of comprehension. AI systems have no comprehension or understanding and without this they have no actual reasoning ability. Human-level comprehension falls under a completely different theory from the computational derivatives of Church-Turing.
Can you make a human-level machine agent smarter by giving it more computational power? No, because you'll run into all sorts of problems which would take a few book chapters to explain. There are enhancements you can make but these have limits. If you go by a standard deviation 15 chart for IQ like Wechsler or the 5th edition of Stanford-Binet, the chances of having an IQ of 195 is 1 out of 8 billion. So, this roughly sets the upper bound of human ability. We could probably see machine agents with an IQ of 240 but not 500 or 1,000. I do understand the confusion concerning computation since exhaustive routines in AI are time limited. For example, our dim-witted chess programs play by laborious trial and error. They don't actually get smarter with more computational power, they are just able to eliminate bad moves faster. Let me give a human example. Let's say that I could do 5 math problems of a given complexity per hour using pencil and paper. So, I add a slide rule and my rate changes to 10 problems per hour. Then I switch to a calculator and my rate increases to 20. Let's say I then start using a spreadsheet and I hit 30 per hour. I am not actually 6x smarter than I was when I used pencil and paper.
So, to answer the question, it is not possible to continuously increase intelligence even with unlimited computational power. However, it should be possible for machine intelligence to exceed human intelligence. One final thing that I should mention though is that this type of theory is quite good at organizing knowledge in a way that current big data methods do not. So, it is probable that the same theory that would allow a machine IQ of 240 would also provide enough assistance to a human to function at the same level.

Actually, these aspects is part of some books I am working on right now.
Like what Jeevan say it is bound by laws of physics. I see that too, but in case of the Human, in the way i look at it, we are at the very lowest step/beginning of an AI that can do self reflecting and question its own ability to do rational thinking and how far it can go and develop.
And I also use the Black Hole - maximum density, maximum energy concentration as a 1'st upper limit. A intelligence who can operate at Plank Level, can store all information in this world in like a few sand corns size. So given enough density, optimization in information exchange/methods, the upper limit is far far away from what humans process.
But I take the analyze further and go beyond the Black hole theory. Of course it is speculation, but we still don't know all. So what is possible in more than 4 dimension (human brain, 3 dimension and time to think).
As I see it, in real world, upper limit is so far away from our understanding, that we still can't handle it.

Welcome to AI!  Thanks for bringing up information density in the context of black holes.  Could I ask you to provide some reliable links, so those not familiar with these concepts can read more about them?

It depends on the specific definition of intelligence, but with a definition based on proficiency on a set of arbitrary tasks the answer is yes. For instance, if we take intelligence as multiplicative function
$$ I = f(Accuracy) * g(Resource \, Consumption) * h(Generalisation \, Abilities) $$
there is an inherent limit given by the maximum values obtained by the individual functions.

Accuracy on the tasks is limited by the maximum obtainable score on those tasks.
Minimal resource consumption is limited by physics and information theory (entropy of the tasks, limits of computation, quantity of matter-energy in the universe)
Generalisation abilities are capped at intelligences able to solve arbitrary tasks.

Moreover this limit is not simply the product of the max of the individual functions, since there are inherent tradeoffs (to increase accuracy more resources are needed, etc.)
The definition of intelligence above is inspired by the Pragmatic General Intelligence Index.
Nevertheless, if in our definition of intelligence we disregard resource consumption, our universe is infinite and we allow for tasks with arbitrary high scores, then there is no upper bound on intelligence.

No. There is no ceiling to intelligence. However, I am applying this loosely.
When you consider the intelligence of a person, you generally think of some baseline IQ that ranks that person on a scale.
Per the definition of our "Salable" IQ, 200 is considered being a (nearly) unbelievable intelligence.
However, when you look at "Entities", the scale disappears. Google, NSA, Universities - they could all considered super-intelligence. There's no possible metric or IQ to assign to these entities that we know of because they aren't necessarily comparable. However, no single person is smart enough to invent all that was necessary to bring an entity and it's "intelligence" into fruition.
When you bring AI into the equation, your limited by your resources. AI is proving more and more capable as we evolve the technology and we understand our data better.
We may find in the future that GO isn't all that infinite and there is a "Best" strategy (thus we solve the game). However, we keep on making machines that are smarter than previous ones.
I wouldn't be surprised if AI starts making paper companies soon per business tactical advantages.


text:
How can computers beat humans at image recognition, if humans may incorrectly label the images?
For supervised learning, humans have to label the images computers use to train in the first place, so the computers will probably get wrong the images that humans get wrong. If so can computers beat humans?

When researchers claim "better than human accuracy", they are demonstrating that a computer can beat an individual human on a test. And that is because the ground truth labels are actually higher accuracy than a single human could label the images individually.
There are at least two major ways that ground truth labels can beat an individual human on image tasks.

Additional information is available from the same source as the image. For instance many pictures of pets in the ImageNet database are labeled with a specific breed of animal, due to how they are sourced. Most people
who are not experts at pet breeds will score quite badly on a test to identify dog breeds at the fine grained level that ImageNet presents.
Ground truth based on expert opinion can be sourced from multiple experts and their opinions combined. This approach can independently be shown to be more reliable than the opinion of a single person.

So in short yes computers can beat humans when they have had access to better original ground truth, and that is possible, even if that ground truth is generated by humans.
However, in general your concern stands. Ground truth data is a limiting factor. It might be possible in theory for a computer model to have an even better accuracy at the "real" task than the ground truth for a supervised learning task. However, this is next to impossible to prove, and other concerns, such as changes to distribution of real data as opposed to training data, are usually more important at that level of accuracy.

I seem to recall that "human crowdsourcing" is/was a solution in captcha identification, in the sense that a single human is not reliable, but a large sample yields correct results.  (Makes me wonder if the same technique can be applied to algorithms, where any one algorithm may not be reliable, but an expanding group of unique algorithms might become more reliable.)

@DukeZhou This is used in theoretical computer science. When you can't solve a problem efficiently, often instead you generate a random number and use that number to cheat in some way that allows you to solve the problem efficiently. However, because of the cheat it's not necessarily correct. What you do then is write a formula for the probability of being wrong, run the program a bunch of times, and get a small error term by aggregating all of the results.

For example, checking if two numbers are equal is hard. It requires looking at every single digit, and some numbers have a huge number of digits. Instead, I can generate a random prime with 1000 digits and check if a = b mod p. Calculating a mod p and b mod p can be done really easily, and then to check the comparison I only have to compare 1000 digit numbers, which is a constant independent of a and b. If a and b are actually equal, then they're equal mod p too, but sometimes I'll get the answer wrong by accident because of the algebraic relationship between a and b.

However, I can do this many times with different randomly generated prime numbers. I say that a != b if I found a prime such that a != b mod p. This is guaranteed to be correct, when I output it. When I don't find a disproof, I guess that a = b. This will sometimes be wrong. However, it turns out that the rate at which the error goes to 0 as the number of primes I guess increases is exceptionally fast.

That's all really interesting. I once heard a story of people guessing the number of objects in a glass jar, and the average of all the guesses was very close to the actual number. This might be another example of what you're talking about.


text:
How do I write a good evaluation function for a board game?
I'm currently writing the Alpha-Beta pruning algorithm for a board game. Now I need to come up with a good evaluation function. The game is a bit like snakes and ladders (you have to finish the race first), so for a possible feature list I came up with the following:

field index should be high
in the lower fields my fuel should be high, when coming to the end it should be low (maximum of '10' required to enter the goal)
all 'power-ups' must be spent to enter the goal, so prioritize them
if it is possible to enter the goal (a legit move), do it!

There could be some more for some special cases.
I've read somewhere that it is the best (and easiest) to combine them in a linear function, for example:
$$0.75 * i - 5 * p - 0.25 * |(f - \text{MAX_FIELD_INDEX}/i)|,$$
where

$i$ = field index
$p$ = power-ups
$f$ = fuel

Since I can't ask an expert and I'm not an expert by myself, I have nobody to ask if those parameters are good, if I've forgotten something or if I've combined the factors correctly.
The parameters aren't that big of a deal because I could use a genetic algorithm or something else to optimize them.
My problem and question is: What do I have to do to find out how to put together my features optimally (how can I optimize the function/parameter arrangement itself)?

Based on your description, I'd maximize the following terms:

i
-max(f - 10 - (MAX_FIELD_INDEX - i), 0) - assuming consumption of one fuel per field; this becomes negative when you have too much fuel
a similar function of p, as spending them gets more important when approaching the goal

As having fuel is probably a good thing in the beginning, you could use a term like f. Similarly for the "power packs" (or are they rather "weakness packs"?).
I'd combine the terms using a linear function like you did and let it optimize. You may need more such terms. Maybe it's simpler to get rid of the power packs when you have enough fuel? Then something like -max(p-f, 0) may help.
You may generate some ad-hoc expressions or add some products of your terms as new terms. You may want to do this after the coefficients of the simpler terms have already been optimized (so you help the more complex optimization with a good staring point).

The fuel gets calculated as following: 1 + 2 + 3 + 4 = 10 for 4 fields and so on.. (the little gauss) so going it costs more to go more fields at once. Would this change anything in your function? I think if I have time I'll add the whole game instructions to the question later

I meant for advancing 4 fields.. e.g. advancing 6 fields whereever you are costs 1+2+3+4+5+6 = 21 fuel

@po0l Then you may need -max(f - 10 - (MAX_FIELD_INDEX - i) * (MAX_FIELD_INDEX - i + 1) / 2, 0) instead or in addition to the above expression.


text:
Neural network for data visualization
At my work, we're currently doing some research into data visualisation for highly interconnected data, basically graphs.
We've been implementing all sorts of different layouts and trying to see which fits best, but, due to the nature of the problem --it's a visual thing - we needed to come up with some automated way to analyse the result so welcome up with a bunch of metrics to analyse our layouts.
So far, the most important metrics have been information density, edge crossings, node overlap and edge length. This gives us some good results and has allowed us to fine-tune our layout algorithms.
However, when a new graph is loaded, we noticed that humans still tend to fiddle a lot with the structure of the layout. Moreover, it seems that our metrics do a good job of predicting where a user is likely to mess around. Graph layout is a tough problem, so after some discussion, the idea of just throwing data at a neural network and let it figure it out came up.
None of us are experts, or even experienced in AI.  I'm the one with the most contact with AI methods.  All I've ever done were simple NN models, no convolution, feedback or feedforward or anything of the sorts, but it seems to me this should be doable.
Maybe it's my lack of expertise here, but I haven't been able to find any good information on this sort of application for NNs, so I was hoping someone here could point me in the right direction.

What sort of model is best for such a situation? and why? Is this actually possible or would it be super complicated? Has anyone ever tried something like this before?

If it helps, our input data (for v1, I guess) would be two arrays of variable length, one for the nodes and another for the relationships between them and the output data would be an array with the node XY coordinates.

I doubt that this problem is a good application of NN. Take a look at this https://en.wikipedia.org/wiki/Force-directed_graph_drawing. And try Graphviz if you haven't already. Drawing graphs in a meaningful way is notoriously difficult.
An alternative approach would be to embed all nodes into a 2d or 3d metric space. I.e. for each node you will have coordinates of a point in such space. If you have certain rich descriptors of the nodes (i.e. some kind of information associated with each node, pictures, text, vectors of numbers) that can be done even with a NN. To have a good embedding you need more info than just node degrees or/and edge weights.

We are aware of Force layouts and other such algorithms. In fact, we've been improving on them, but we wanted to try a different approach, as we will never get rid of some of their limitations. Our own rendering software is, imo, already way better than Graphviz so that doesn't help either.
Thank you for your second suggestions though!

Neural networks are used to visualize high dimensional data through the use of autoencoding. It's similar to Principal Component Analysis and is regarded to perform better then PCA. Autoencoding will take your data and convert it to a 2 or 3-dimensional representation. Since you have an array of data you might want to use LSTM. You will have to make sure you reset the LSTM states when you perform the reverse pass. You will probably have trouble interpreting the autoencoded form of the data though, so that will be a task of its own.

Thanks for the information, particularly about autoencoding. We'll investigate further although we're now more focused on other tasks. I'll get back to you eventually whenever we investigate this and give you some feedback on our successes and failure :) Cheers


text:
Why isn't ethics more integrated into current AI systems?
I am a PhD student in computer science, and currently creating a state of the art overview in applications done in Machine Ethics (a multidisciplinary field combining philosophy and AI, that looks at creating explicit ethical programs or agents). It seems that the field mostly contains theoretical arguments and there are relatively little implementations, even though there are many people with a technical background in the field.
I understand that because ethics are involved, there is no ground truth and since it's part of philosophy one can get lost in arguing over which type of ethics should be implemented and how this can be done best. However, in computer science, it is usual to even try a simple implementation to show the possibilities or limitations of your approach.
What are the possible reasons there is so little done in explicitly implementing ethics in AI and experimenting with it?

Examples of approaches can be found in this and this overview paper.

This is necessarily a high-level answer, and highly speculative, but I've been thinking on this question, and here are my thoughts:

Implementing ethical algorithms requires a mathematical basis for philosophy because computers are difference engines

After Russell & Whitehead's famous failure, and Gödel's incompleteness theorem, this would seem to be problematic.

AI is a highly applied field, especially today per continuing validation of deep learning, and no company wants to go near the issue of ethics unless they are forced to

Thus, you see it in self-driving cars because the engineers have no choice but to grapple with the problem.  By contrast, I don't think you'll see many algorithmic stock trading firms, where the business is Pareto efficiency, worrying about the ethics or social impacts of financial speculation. (The solution to "flash crashes" seems to have been rules for temporary suspension of trading, instead of addressing the social value of high-frequency algorithmic trading.) A more obvious example is social media companies ignoring the extreme amounts of information abuse (disinformation and misinformation) being posted on their sites, pleading ignorance, which is highly suspect in that the activity generated by information abuse positively affects their bottom-lines.

Applied fields tend to be predominantly driven by profit

The primary directive of corporations is to return a profit to investors.  It's not uncommon for corporations to break the law when the fines and penalties are expected to be less than the profit made by illegal activity.  (There is the concept of ethics in business, but the culture in general seems to judge people and companies based on how much money they make, regardless of the means.)

Implementation of machine ethics is being explored in areas where they are necessary to sell the product, but elsewhere, it's still largely hypothetical

If superintelligences evolve and wipe out humanity (as some very smart people with superior mathematics skills are warning us about,) my feeling is that it will be a function of nature, where unrestricted evolution of these algorithms is due to economic drivers which focus on hyper-partisan automata in industries like financial speculation and autonomous warfare.  Essentially, chasing profits at all costs, regardless of the impacts.

Interesting take, the profit aspect! Hadn't thought of that since I was so focused on the research reasons, but it makes sense that there is no pressure from industry either for research to pursue it further.

@SuzanneTolmeijer Thanks.  I've been delving into how game theory connects to ethics for a few years now.  (Mostly early philosophy where, for instance, "balance" was said to be the "good"--balance is an equilibrium function and can be precisely, mathematically defined. It relates to economics in that stability is generally understood to be optimal, where instability is potentially catastrophic.) But I'm not the only AI contributor who has touched on economic drivers in this field, and the potential dangers of what I'm terming "hyperpartisan AI".

Intuitively speaking, it seems to be the case that there is little research into the implementation of AI ethics because:

Society as a whole seems to comfortably agree that the current state of machine intelligence is not strong enough for it to be considered as conscious or sentient. Thus we don't need to give it ethical rights (yet).
Implementation of ethical behaviour into a program requires a method for computers to be capable of interpreting "meaning", which we do not know how to do yet.

I feel part of the problem as to why there is very little in the way of ethical implementations of AI/ML technologies, is simply because there is no need or proper application of the theoretical frameworks.
By this I mean, there are no substantial ways we can apply this understanding to algorithms and models that cannot interact in a meaningful way. We have such a large theoretical framework on AI safety/ethics because it is extremely important. We need to come up with safe guidelines for implementing strong AI before it is created.
Some very focused papers have started to narrow down the issues in creating ethical/safe AI systems. See Concrete Problems in AI Safety

There seems to be a need for explicit ethics, at least many researchers call for advancement of explicit ethics (e.g. Moor 2006).
In terms of proper application of theoretical frameworks, you might be right: implementation implies some interpretation in some cases, which can prove difficult. However, you agree that safe guidelines for AI are important, which means practical guidelines on what we consider to be 'right' or 'ethical' for our machines to do. These guidelines will imply how to implement ethics in machines in some way.

With the imitation method, the most appropriate behavior can be integrated into artificial intelligence. Artificial intelligence can be reshaped when the ethical position changes. It is used for ideological purpose or to gather information. It's not clear what the robot is.

We can take the error model into accountant. Recognising bias and variance among the performance under neural networks can be a first step. And then we can discuss whether such performance is allowed. As far as we know, practicing ethnics requires empirical and field study. we cannot simply take rationales and paper essays to determine the doings of learnt machines is wrong or not. It can be further divided into accidents, errors , or even bugs created from  the developers.

An interesting point you raise here is basically, which measures can be used to test the system, and how do you determine if the system is performing accurately? This is something that is missing in the field, and I feel is part of the problem indeed: how to test your system without ground truth? Which (dimensions in) measures make sense in the context of explicit ethical algorithms? Without that, you have no way of comparing systems, which limits progress in the field.


text:
How do I use a genetic algorithm to generate the scores of an evaluation function for alpha-beta pruning?
I have created a Gomoku (5 in a row) AI using Alpha-Beta Pruning. It makes moves on a not-so-stupid level. First, let me vaguely describe the evaluation function of the Alpha-Beta algorithm.
When it receives a board as an input, it first finds all repetitions of stones and gives it a score out of 4 possible values depending on its usefulness as a threat, which is decided by length. And it will return the summation of all the repetition scores.
But, the problem is that I explicitly decided the scores (4 in total), and they don't seem like the best choices.
So, I've decided to implement a genetic algorithm to generate these scores. Each of the genes will be one of 4 scores. So, for example, the chromosome of the hard-coded scores would be $[5, 40000, 10000000, 50000]$.
However, because I'm using the genetic algorithm to create the scores of the evaluation function, I'm not sure how I should implement the genetic fitness function. So, instead, I have thought of the following:
Instead of using a fitness function, I'll just merge the selection process together: If I have 2 chromosomes, A and B, and need to select one, I'll simulate a game using both A and B chromosomes in each AI, and select the chromosome which wins.

Is this a viable replacement to the fitness function?

Because of the characteristics of the Alpha-Beta algorithm, I need to give the max score to the win condition, which in most cases is set to infinity. However, because I can't use infinity, I just used an absurdly large number. Do I also need to add this score to the chromosome? Or because it's insignificant and doesn't change the values of the grading function, leave it as a constant?

When initially creating chromosomes, random generation, following standard distribution is said to be the most optimal. However, genes in my case have a large deviation. Would it still be okay to generate chromosomes randomly?

Why exactly four values for the score? What do these values exactly represent?

Is this a viable replacement to the Fitness function?

Sure, the fitness is 1 for the winner and 0 for the loser. You're using some kind of the Tournament selection.
It might be better to use more chromosomes and let A play against B, C, D... and define the fitness as the number of wins. Or not, as such an evaluation is more precise but also more time-consuming.

... Infinity ... Do I also need to add this score to the chromosome?

Why should you? The exact value doesn't matter (as it only needs to be big enough), so there's nothing to evolve there. You don't represent the number  of players either, right? Just use common sense.

... Would it still be okay to generate chromosomes randomly?

I guess so, but a distribution providing values closer to the expected outcome should be better. This depends on how you mutate them (adding a small random value won't bring you far, multiplying by 1 + small_random_value would).
Alternatively, you can generate values from some fixed interval and scale them up.
Your values are IMHO far too big. Whatever your 5 and 40000 mean, I guess, that 5 and 400 would work the same.


text:
What other kind of AIs exist apart from goal-driven?
Goal-driven AIs is the only kind of AI I am aware of. However, Marcus Hutter claims the following

Most, if not all known facets of intelligence can be formulated as goal driven or, more generally, as maximizing some utility function. It is, therefore, sufficient to study goal driven AI.

which doesn't necessarily imply that there are other types of AIs (apart from goal-driven), but (at least, in the way it is phrased) suggests that there are other types of AIs. If there exist other types of AIs, which are they?

Comments are not for extended discussion; this conversation has been moved to chat.

I found the paper Learning, Goals, and Learning Goals: A Perspective on Goal-Driven Learning (1995) by Leake and Ram, which I have not yet the occasion to read, that may contain useful info to answer this question.

Whether all AI are goal-driven depends on how far you're willing to extend the definition of 'goal'.
AIs are often defined through their primary tasks. Face recognition is a process; a face recognition AI is a limited, special-purpose AI. Paperclip collection is a process, the Paperclip Maximizer is a general, goal-driven AI.
There will always be some sort of goal, but it may in certain cases become so nebulous it's really a stretch to continue calling it a goal. For example,  unsupervised learning (as suggested by Aiden Grossman) - the goal you set is "learn", with very little qualifiers. The underlying AI may develop in surprising and unexpected ways; it may even modify own goals.
Similarly, another "aimless" AI type is a simulator of biological intelligence - say, replicating neural connections of an insect in software, making it run and observing the behaviors. The AI itself is not given a specific goal - it's a tool/subject of study, not something directly producing results. Still, the goals are exactly the same as goals of the original insect: feed, multiply, stay safe; unattainable due to hardware limitations but still pursued.
Therefore, whether all AIs are goal-driven is arguable. The only AI that is not actually driven by any goal whatsoever is one that's switched off. But whether you consider all the possible nebulous, abstract or accidental goals that drive the AI to call it actually goal-driven, that's up to debate.

Comments are not for extended discussion; this conversation has been moved to chat.

I suppose, the context was as follows: suppose you have AI for which goals cannot be described by a function. It doesn't mean it has no goals, but that formulation of the goals cannot be functionally dependent on known reality model parameters.
In this interpretation text which you cited has clear meaning: author doesn't want to discuss such kinds of object, which is reasonable.
Such interpretation is interesting as well, because opens interesting cases to think about :)

Following the standard AI textbook, Artificial Intelligence:  A modern approach, other from goals based agents there are Reflex Agents, Model based Reflex agents, Model based Goal based agents, Utility agents and Learning agents.
Simple Reflex agents are based on condition action rules, kinda like If-then rule based agents.

Model based agents are whose which have a model of the world in it, I mean how the world evolves and what my action will do
They are in two variants, Model based reflex agent and model based goal directed agent.

Former one percepts the environment, infer from that how the world evolves and figure out what their action will do then that have some conditon base rules which then lead to action,
In latter type the module of conditon based rules are replaced by the module of goals, which can be kinda further broken into condition based rules,which they are literally.

Utility agents are reinforcement based agent they act in environment to maximize their utility, and have some element to check what the happiness measure or "Performance measure" is, the defintion of rational Agent is based on this, which is a rational agent will always act to maximize their average utility.

Learning agent are which contain some kinda critics, some learning element and then problem generator, an isomorphism might be drawn from to Neural networks, which has their critics as Cost function, learning element as Wieght tuning.
I have been saying "kinda" because they just seems to be metaphor to me.
I dont know why Marcus hutler described goals agents as Utility based agents, probably he had some kinda hybrid of two.

Interesting question.
Case One — Goals not Goal
When systems are developed, they are based on requirements.  The goals of the system is closely coupled to the goals of the development.  This is the first example.  Few systems and few development efforts to construct them are goal driven.  They are rather systems and processes that tend toward a set of goals.  Rarely is there a single loss function.
Consider an animal.  It's metabolic stasis system may generate what seems like a goal, a set of signals to other systems to find and eat food.  At another time its temperature stasis system may generate what seems like a goal, a set of signals to other systems to build a shelter to enter.  The notion that there is some goal to survive is inconsistent with the way neurological and metabolic systems actually interact.  Survival, some would say, was the goal of the animal's evolution, but that is not technically true.  It survived, not because survival was a goal but because each of its ancestors survived as a result of transient and recurrent command signals for eating and sheltering.  There is no aggregate loss function.  There is no single neural pathway that all these signals are funneled through.  The animal's behavioral subsystems may share some components, but the mechanisms of intent are largely independent.  It only appears to a casual observer that the separate systems are collaborating toward survival.
This is true of some vehicle automation systems.  Collision avoidance is a system with a set of goals and routing is a system with a separate set of goals.  The goals are decoupled even though they may share a CPU, vision systems, and a large number of functional components and computing resources.
These systems are goals driven, using the plural, not goal driven.
Case Two — Process of Elimination
Another type of AI approach that is not technically goal driven are genetic algorithms.  They replicate and some of the replicants are eliminated, but the entities themselves have no goal.  The elimination criteria is not technically a goal.  It is the opposite of one.  The appearance of replicants that are not eliminated after many rounds of replication and concurrent narrowing of criteria may achieve some goal of the person running the algorithm, but the algorithm itself has no representation of a goal but rather its opposite, the final criteria of elimination in the last round.
Case Three — Experimental Apparatus
Many AI experiments to discover the nature of intelligence and ways to simulate aspects of intelligence have the external goal defined by the researcher, but the system used in the experiment, the AI apparatus, is not driven by a goal.  It runs goal-less.
Case Four — Toys
Whether toys for adults or kids, toys may arise with some goal in mind, to amuse in some way, but the AI within them has no goal.  There is no loss function to minimize or reinforcement or wellness signal to maximize.  It's behavior may be less amusing if it were goal driven.  What makes it amusing is its ability to produce patterns associated with intelligence, but to no particular purpose.  In some ways, much human conversation and creative art is like this.  Too much purpose, and the level of amusement decreases.
Case Five — Robots That Mimic Postmodern Humans
Humans in postmodern society, the ones that are truly relativists and have rejected modernism, are not particularly goal driven.  They tend to wander around and do things that are intelligent, however they may act goal directed and then destroy the product of the transient goal.  Building sand castles on the beach is an example.  Although this may not be the focus of current AI research, since there is so much hope placed in AI to do things for us, such does not exclude the possibility of future robots that build something and then destroy it to build something else, without some practical objective.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
How to design a neural network that gets the author name of a piece of art as input?
I'm building a neural net to predict the value of a piece of art with a wide range of inputs (size, art medium, etc.) and I would like to include the author as an input as well (it is often a huge factor in the value of a single piece of art).
My current concern is that the name of the author isn't an ideal numerical input for a NN (i.e. If I just code each author with an increasing integer value I will be indirectly assigning more value to authors further down the list -_-). My thoughts were to create separate inputs for all the authors in my dataset and then just use one-hot encoding to better represent the input to the NN.
This approach, however, runs into a problem when an author that is not included in my training data is used as an input to the NN (i.e. a new author). I can get around this with an "other author" input field, but I am worried that this won't be accurate as I would not have trained the the NN for this input (all pieces of art with a valuation have an author).
I haven't fully thought this through but I thought of perhaps training 2 NN's, one for a valuation without an author and one for valuation with an author to ensure I have enough training data for an "authorless valuation" to still be reasonably accurate.
I am still trying to conceptualize the best NN architecture before I get stuck into the implementation so if anyone has any suggestions/comments I would be very grateful!
P.S. I am doing this as a small competition with a friend to test a NN vs the traditional commercial valuation techniques.

Not related to your question, but you might consider including features based on the previous offers / other works of the artist / and events in the art market world. Those usually have a big impact in the pricing of a piece.

The most straightforward approach I would recommend would be the one-hot encoding solution without a feature for ''other author''. If you use drop-out during training, the network should learn how to deal with input vectors that don't have any author-features set to 1. Then, whenever you have an unknown author, you simply have no 1s at all for any of the author-features, and it should still have learned how to deal with that.
Another possible approach would be a one-hot encoding with a feature for ''other author''. In order to still get training data for that weight, you could simply use data augmentation. The most straightforward data augmentation approach for this would be to create copies of the instances in your training data, but for those copies set the ''other author'' feature to 1 instead of the actual author.
The most complex solution I can think of would be taking the string representation of the author, trying to have your program find information about that author online (for example try to look up a wikipedia page), push all that text through an LSTM. The ''output layer'' of the LSTM can then be merged with your other features (those two together will be your ''input vector''), stack a few more layers on top of that, and train the entire thing end-to-end. Unless the competition with your friend involves a serious amount of money, this is probably going to be too complicated to be worth it though.

I would try to find some proxy features about the author, as opposed to encode the identity of the author. Likely good features of an author include averages of other features about the work (such as size, media types etc), and critically for your prediction, some stats on previous sale prices of any works. Remember for historical training data to include the proxy data as it would of appeared at the time of sale.
Having these proxies is likely to work much better than one-hot-encoding an author identity and getting the ML algorithm to assign weights based on the identity alone, especially for authors with only small amounts of data to represent them.
There is little reason as far as I know, to expect the character string of an author's name to correlate with value of the artwork. There may be some good vs bad brand name effect, but I think that will be very hard to predict even in isolation from other factors, and including historic sales data should take account of this and similar effects, except for artists with no known history.


text:
When do you back-propagate errors through a neural network when using TD($\lambda$)?
I have a neural network that I'm want to use to self-play Connect Four. The neural network receives the board state and is to provide an estimate of the state's value.
I would then, for each move, use the highest estimate, occasionally I will use one of the other moves for exploration.
I intend to use TD($\lambda$) to calculate the errors for each state to backpropagate through the network.
But I'm confused about when this should actually occur. Do I store the estimate of a state that is used and calculate the error based on the next state chosen?
Or do I store a history of all states and backpropagate only when the game is a win/lose/draw?
I guess overall I'm not sure I understand when the update occurs, partially because I don't quite understand how to implement the lambda? Like if I was to apply to back prop after every move, how would I even know the value of lambda at this time-step before I know how long the game will last?
When self-playing, is the error the difference between that "sides" last move? I.e. I compare move 1 against move 3, and move 2 against move 4, etc?

Are you using forward-looking or backward-looking TD lambda? Forward looking you calculate at the end of an episode, backward-looking you calculate on each step, but needs to add eligibility traces. If not sure, can you point to the reference that you are using?


text:
What is the "question" when using Dynamic Memory Networks to do part-of-speech tagging and sentiment analysis?
In the paper Ask Me Anything: Dynamic Memory Networks for Natural Language Processing the authors described a Dynamic Memory Network in the context of question answering. Then, they also tested the network on sentiment classification and part-of-speech tagging. In those applications, what is the 'question' being inputted to the network?


text:
Would it take 1700 years to run AlphaGo Zero in commodity hardware?
From this link, AlphaGo would take millennia to run in regular hardware.

They generated 29 million games for the final result, which means it's going to take me about 1700 years to replicate this.

Are these calculations correct?

It's not unreasonable to think they trained AlphaGo on (the equivalent of) thousands of parallel instances of commodity hardware-- the months of training time probably represent a few million dollars of cost to Google.  Note that a trained AlphaGo can run reasonably on a single commodity machine.

Although the above statement holds important analogies to communicate the technical advances made by deep mind in the development of Alpha Go. It is inaccurate and should be taken skeptically.
Firstly, although Alpha go was trained on specialized hardware such as high end NVidia GPU's and custom google TPU's. It should be noted that it can run on a regular desktop although it won't be as powerful as the distributed version of Alpha Go. Additionally anyone with a laptop can access a similar amount of computing resources in the cloud with the touch of a button.
Although a version of Alpha Go indeed ran on 1202 CPUs and 176 high end GPU's as reported by Nature magazine. Which roughly translates to a system 1000 times as powerful as a commodity laptop. We need to consider other important factors such as moores law which postulates that we could have a computer as fast as 1202 CPU's in our pockets within 20 years and not several millennia.
Furthermore, the latest version of Alpha Go, Alpha Go Zero, trained on a single server with only 4 TPU's. Considering that the latest generation of smartphones such as iPhone X and Huawei's Kirin come packed with specialised AI chips. I will not be surprised if a similar reduction of form factor is achieved in commodity desktop computers once new models packed with AI chips are introduced.
I respect and acknowledge the technical achievements of Demis Hassabis and the deep mind team in developing a system as powerful as Alpha Go. However I believe analogies such as the one used above are inaccurate and mis-represent facts.

You confuse runtime and trainings hardware. Alpha Go Zero ran on 4 TPUs, but it was trained on 5000 TPUs.

@Seth Simba so do you think alpha go zero would be faster and easier to train?


text:
How is the "right to explanation" reasonable?
There has been recent uptick in interest in eXplainable Artificial Intelligence (XAI). Here is XAI's mission as stated on its DARPA page:

The Explainable AI (XAI) program aims to create a suite of machine learning techniques that:

Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and
Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.

The New York Times article Can A.I. Be Taught to Explain Itself? does a good job at explaining the need for XAI from a human interest perspective as well as providing a glancing outlook on the techniques being developed for the same. The force behind XAI movement seems to center around the (up and coming?) concept of right to explanation, that is, the requirement that AI applications that significantly impact human lives via their decisions be able to explain to stakeholders the factors/reasons leading up to said decision.
How is the right to explanation reasonable, given the current standards at which we hold each other accountable?

The article Artificial intelligence (AI) Computer says no: why making AIs fair, accountable and transparent is crucial states that Europe is ahead of other parts of the world in drawing up regulations to protect people from badly made or used AI. However, these regulations are a little weak: for example, they do not apply to the cases where there is the involvement of humans or their decisions (apart from the AI ones).

This is question of morality... Not a question of AI. But in general, understanding is valuable; that's why people want it...

How is the right to explanation reasonable, given the current standards at which we hold each other accountable?

In short, it is quite reasonable.
More specifically, making AI accountable and responsible for explaining the decision seems reasonable because

Humans (DARPA in this case) has chosen to create, raise and evolve AI with the tax-payers money. In our society as well, whenever humans have come together for some purpose (some formal arrangement like government or otherwise), accountability and responsibility is assumed and assigned. Thus, expecting AI (who will take decisions on our behalf) to explain their decisions seems only natural extension of how humans currently operate.
Humans (generally at least) don't have super-power or resources to control and manipulate the rest of the population. But when they do (like our political leaders), we want them to be accountable.
In rare cases, powerful humans become powerful due to things which are accessible to everyone, while AI's super-power won't be. So, our civilization would feel safer and thus less insecure with an AI which doesn't shut the door on face when questions are asked.

Other benefits of AI that offers an explanation on how a decision is reached

Far easier to debug and improve in its early stage.
Can be customized further (and along the way) to amplify certain aspects like social-welfare over profitability, etc.

In the paper Slave to the Algorithm? Why a 'Right to an Explanation' Is Probably Not the Remedy You Are Looking For, the authors claim that the "right to explanation" is unlikely to provide a complete remedy to algorithmic harms for at least two reasons

It is unclear when any explanation-related right can be triggered
The explanations required by the law, "meaningful information about the
logic of processing", might not be provided by current (explainable) AI methods

The authors then conclude that the "right to explanation" is distracting, but that other laws of the European Union's General Data Protection Regulation, such as

right to be forgotten
right to data portability
privacy by design

might compensate the defects of the "right to explanation".
To conclude, the "right to explanation" is an attempt to protect citizens against the possible undesirable consequences of the use of AI methods. However, it is not flawless and clearer rights might need to be promulgated.


text:
How can I use neural networks for detecting TV channel logos in video frames?
I am trying to detect a TV channel logo inside a video file. So, simply, given an input .mp4 video, detect if it has that logo present in a specific frame, say the first frame, or not.
Here's the first example of a frame with a logo.

Here's the second example.

We have that logo in advance (although might not be %100 of the same size) and the location is always fixed.
I already have a pattern matching-based approach. But that requires the pattern to be %100 the same size.
I would like to use Deep Learning and Neural networks to achieve that. How can I do that? I believe CNNs can have a higher efficiency.

To perform image recognition you have to find a way to represent an image with certain features.
One of the defining characteristics of a good image recognition algorithm are it's ability to detect salient regions, that is, regions which contain the most information
There is a lot of attention on deep learning for content-based image classification at the moment. You can achieve decent results by implementing deep learning having three or more layers of CNN's where each layer is responsible for extracting one or more feature of the image.

Thanks. I'm not a CNN guy myself. But is there any pointer to a source code that given a logo image, it can detect if it exists or not?

Hey, please check out DeepLogo on Github by Satoj Kovic. It is written on Python and uses CNN's to recognise brand logo's. I have posted the link below. Cheers. https://github.com/satojkovic/DeepLogo

Because it is video input and the logos are usually stationary because they are layered over the live or recorded frames by either hardware or software, the task is not difficult.  Logos also usually have limited color palettes and crisp edges.  The features of their fonts, when they spell words or acronyms are usually consistent too.  These are generalities that can be exploited in deep learning.
As with the other similar question posted by this author, a combination of LSTM and CNN layers can be trained to find and isolate the logo.  With some image tricks, the image behind the logo can also be reconstructed with a reasonable accuracy and reliability from the pixels around the logo through a similar set of learning techniques.
These are a few starting points for the development.

Deep Learning for Logo Recognition — Imaging and Vision Laboratory
Logo detection using YOLOv2 – Akarsh Zingade
GitHub — satojkovic/DeepLogo: A brand logo recognition system using deep convolutional neural networks
Logo detection in Images using SSD – Towards Data Science
1701.02620 — Deep Learning for Logo Recognition


text:
How can I design the input layer of a feed-forward neural network to be trained with a medical dataset with three features?
I am building a feed-forward neural network with two hidden layers, which I will train with a medical dataset, which consists of both data, such as age and sex, and images of x-ray scans ($1024 \times 1024$). The labels are types of cancer.
I believe that ages and sex will affect the output of the network. But including the images will make the network biased towards the images, because images will occupy most of the input layer.
How can I design the input layer of the network?

@BrianO'Donnell A neural network with two hidden layer is a deep neural network

@kkr4k No expert in neural networks considers two hidden layers as a 'deep neural network.  nbro corrected the original erroneous reference to 'deep neural network'.

I would give some advice here:
You should probably use Convolutional Neural Network for image data or you'll end up with too many input features, and too much computation. What are your constraints on this point?
I would not mix image data with numerical data. Maybe you can implement two neural networks, one handling image data and another managing the numerical data, each one outputting their results. A second step would be to use their results to handcraft a global result or let another NN manage this.


text:
Is there a neural network in the literature that predicts the next game state based on the current state and the action?
I am trying to find literature on a network architecture that takes the following as in input:

Action (like 'Up', 'Down', etc)
Image of the current state

and outputs:

Image of next state

I already have a lot of training data for the inputs. However, I am trying to find relevant literature/architecture for this problem.

What kind of images are you looking at?

This is a whole sub-field of reinforcement learning known as model-based reinforcement learning. The idea in model based RL is to learn the mapping from current state/action to next state in order to facilitate learning good policies.
If you are dealing with images as inputs I would recommend checking out the Dreamer papers. The most recent being this one.

Note that, in model-based RL, we don't necessarily learn the model: the only requirement is that we use a model of the transition function.

I tried something similar before for 2048 game. I used the state of the board as x, and the move as y. I just trained the neural network with this dataset. The architecture is like a couple of layers with relu and the final layer as softmax. The major thing is that we should not feed the wrong moves in the dataset to the NN, or else the NN also tend to learn the bad moves, which in turn makes it less smarter.
I gathered my dataset by running the minimax on 2048 and assigning a reward for each move, and then eliminating the bad ones on it.
the above process also depends on the way you are taking the feature vector, if your feature vector is an image, then it makes sense to use CNN.
https://github.com/navjindervirdee/2048-deep-reinforcement-learning
DQN is also a good option. but do checkout the above link, it helped me too.
My repo: https://github.com/williamscott701/AI-vs-2048
my results are not actually that great.

You'll probably want to start out with "Action-Conditional Video Prediction using Deep Networks in Atari Games" (arXiv link: https://arxiv.org/abs/1507.08750). That's from 2015 though, I'm sure there have been lots of other interesting developments since then. This paper may still be a good starting point though, and provide you with the correct terminology to plug into google / google scholar to find more recent papers that build on top of this. Google scholar also provides functionality to automatically find papers that cite this one (interesting recent papers will probably cite this one).
As an additional point, you may want to reconsider your desired output. Given a current state-image and action, it may be easier to train a network to predict only the change in image (i.e., predicting NEW_IMAGE - OLD_IMAGE), rather than predicting the full image. You can then always still manually reconstruct the predicted new image simply by adding that output to the old image again. I'm quite sure I've seen this being done in a more recent paper too, but don't remember exactly the title / authors.


text:
Is Q-learning a type of model-based RL?
Model-based RL creates a model of the transition function.
Tabular Q-Learning does this iteratively (without directly optimizing for the transition function). So, does this make tabular Q-learning a type of model-based RL?

I guess, you are talking about the Q table? That is not the same as the transition model.

Tabular Q-Learning does not explicitly create a model of the transition function. It does not generate any output that you can afterwards use as a function to predict what the next state s' will be given a current state s and an action a (that's what a transition function would allow you to do). So no, Q-learning is still model-free.
By the way, model-based RL does not necessarily have to involve creating a model of the transition function. Model-based RL can also mean that you assume that such a function is already given. It just means that you use such a function in some way.

In model-based learning, the learning agent utilizes a model that was previously learned to accomplish a task, whereas model-free RL doesn't use the environment to learn but simply relies on trial and error experience for action selection. The similarity is that in both methods the learning agent is trying to maximize reward from its actions.
Q-Learning is a model-free RL method. It can be used to identify an optimal action-selection policy for any given finite Markov Decision Process. How it works is that it learns an action value function, which essentially gives the expected utility of an action in a given state, then follows an optimal policy afterwards.


text:
How does adjusted fitness penalize large species in NEAT?
I'm learning about NEAT from the paper Evolving Neural Networks through Augmenting Topologies.
I'm having trouble understanding how adjusted fitness penalizes large species and prevents them from dominating the population, I'll demonstrate my current understanding through an example and, hopefully, someone will correct my understanding.
Let's say we have two species, $A$ and $B$, species $A$ did really well the last generation and were given more children, this generation they have $4$ children and their fitnesses are $[8,10,10,12]$, while $B$ has $2$ children and their fitnesses are $[9,9]$. The adjusted fitnesses for $A$ will be $[2, 2.5, 2.5, 3]$ and for B will be $[4.5, 4.5]$.
Now, onto distributing children, the paper states:

Every species is assigned a potentially different number of offspring in proportion to the sum of adjusted fitnesses $f'_i$ of its member organisms.

So, the sum of adjusted fitnesses is $10$ for $A$ and $9$ for $B$, thus $A$ gets more children and keeps growing.
How does this process penalize large species and prevent them from dominating the population?

Consider what would the outcome be if offspring was assigned on the basis of fitness (unadjusted). Sum of A fitnesses would be 40 and B=18. Fitness ratio for both species would be 2.(2):1. In case of adjusted fitness the numbers are A=15 and B=9, which gives ratio of 1.(6):1, thus A is assigned less offspring based on adjusted fitness then unadjusted.
Also note that every new genome assigned to a species decreases all it's members adjusted fitness. In your case members of species A are more successful than members of B, so it should grow. The mechanism is designed to hinder growth of successful species, not to block it entirely. This allows more diversity, which is important when a successful species reaches a dead end and a previously less successful one can take over.

The reason the adjusted fitness prevents species from growing to big is do to the fact that the summation that determines the divisor in the adjusted fitness function reduces to the number of genomes in the species fi belongs to, so as species grow the adjusted fitness for every genome belonging to it is divided by a larger number and thus receives a lower adjusted fitness value which in turn will also reduce the summed adjusted fitness of the species. This smaller adjusted fitness then affects the number of offspring the species will get to create after elitism reduces the population.


text:
How to keep track of the same person detected in different frames of a camera?
At this moment, I am able to use NN to identify an object, such as a human, when given a frame from the camera. Once locate the object, then I can feed the human object image to either NN that's designed to classify male or female.
Let's say I get 1 frame per second from the camera and perform detection, the objective is to track the number of male and female walk pass the camera within the given hours.
My question is, the same person in multiple frames will be overcounted. I couldn't wrap my head around how can I train a NN to understand that this is the same person without dive into facial recognition? I'm sure there is some tracking technique that I just don't know.
One little constraint, if the person left the camera frame and come back into it later, it is fine to treat it as two people.

One way to solve the issue may be a tracker (like Kalman). It would be faster and easier approach than neural nets.
If you insist about solving this issue through neural nets, then some magic and creativity is needed. Based on the nature of tracking, you need to feed multiple frames to predict next location of the object and check if there is any object close by. So, you may consider combining RNNs with CNNs to predict next bounding box to track and replace Kalman filter's prediction with RNNs. (check)

Thanks for the quick reply. So, is there no real tracking? I believe Kalman filter is using a predictive motion model. So, if I only have 2 frames, I cannot track the object. Also, what if the object is human? I cannot really predict the motion as human may stop and turn around at random time. Any insight to my situations?

It would be better for you to check how tracking algorithms work. Tracking algorithms are designed to solve those kinds of problems.