text:
How can you do AI research by your own?
Let's say you want to do AI research and publish some papers just by your own. Would you send them to an AI journal using just your name? Which AI journals are recommended?

research and publish are not the same. In the best case, publish follows decades of research. Any particular can offer a paper to near than all science magazine. Recall publish is not mandatory, and nowadays it would be better if 99% of the current papers had not been published, these papers are more related to how universities promote their staff than to its science value. About recommended journals, google for "AI journals ranking" or "AI journals impact factor".

Let me preface this by acknowledging that this question is prone to opinion. This answer is, in so far as it is possible, primarily observation based.

My understanding is that, when it comes to publishing in a journal, doing so as an individual (without backing from an institution) is, in general, going to be frowned upon.
As you may know: scientific research, including AI research, is generally subject to peer review. When it comes to journals, this is mandatory. The review process attempts to enhance and preserve the integrety of the information published. As an additional safeguard, submissions will often expected to be backed by endorsement from someone within the academic community (typically an institution).
However, and as Pasaba correctly points out (in his comment on the OP), research and publishing are not neccessarily the same thing. Furthermore, not having an endorsement does not not stop you from making a contribution to the field.
For example, you can publish code and/or articles on websites such as Github, and engage with communities of professional and hobbiest researchers around the web, (e.g. this Stack Exchange).
Note that there is also some scope for endorsement without being a direct member of a research institution. For example: arXiv, whilst not strictly a journal, is an open archive that supports endorsement by request.
Without knowing your circumstances, it's hard to know exactly what to do. However, my general advice is to find and engage with communities, and build a network of collaborative peers, rather than trying to succeed in an isolated fashion.

To complement Cosmo's response and maybe address the "using just your name" part of your question, I would like to acknowledge that biased towards submissions do exist. Reviewers may be biased due to several reasons, such us authors' age, publications record, gender, nationality (Lotfi and Mahian, 2014).
If you are concerned about these aspects, rest assured that there are mechanisms to ensure that the authors' reputation does not influence reviewers' judgments. A good example is the "Double-Blind Review" process, which means that identities from the author(s) and reviewer(s) are concealed throughout the review process.


text:
Speeding up CNN training
So I built a CNN without any scientific libraries like TensorFlow or Keras (only NumPy). It is taking a huge amount of time to train. What are some of the tricks and tips followed by people to speed up training of a CNN? (I am not talking about division of jobs into different processors but subtle redundant codes i.e. giving pre-calculated results which is not visible to common programmers).

This is rather hard to answer without knowing more details... Have you tried profiling it to see where the bottlenecks are? Is it floating-point arithmetic? Data I/O?

@OliverMason I agree details are not there but there must be some standard checks performed by programmers...this is more of a general open ended question for future references rather than code specific

@OliverMason what do you mean by profiling? (I am not conversant with programming terminology)

Measure where the time is being spent within the program. Then you can focus on those parts, rather than spend lots of time speeding up something that is fast anyway.

Why are you even caring about the speed of a hand-rolled, non-accelerated CNN implementation? You only build those to understand what's happening. The goal in insight, not performance.

@MSalters in this specific scenario it is...but i have to tackle a problem in the future which demands self made code...also your view is correct, but if i want to gain intuition then i cannot let the network train for such a long time..i need quick action->cause

@DuttaA: A simple CNN for MNIST digits will do for insight. As for the "a problem that demands self-made code"... - problems don't demand that. You may have non-technical issues here, but there are other sites for that sort of issues (depending on where that mistaken demand is coming from)

@MSalters i do not think you are correct..not all devices can support tensorflow or keras now is it? Even older models with older GPU's do not support porcessor sharing properties of tensorflow...so it is bad to jump to conclusions...I could have posted this first on DS.SE (I eventually did) but this site has a problem of user growth and good questions so I present more general problems here

@DuttaA: CUDA 9.2 requires Kepler or newer. That's for instance a GT 710, available for € 33,17 including VAT. But Tensorflow doesn't require a GPU at all. As for multi-processor? Are we still talking about MNIST or similar-sized problems? I mean, my company has spent a very significant amount of money collecting our dataset, it's far larger than MNIST, and yet we still don't need multiple GPU's.

[Ref-some standard checks performed by programmers]
Speeding up Convolutional Neural Networks with Low Rank Expansions
From the abstract:

The focus of this paper is speeding up the evaluation of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability.
Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain.
Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition, showing a possible 2.5x speedup with no loss in accuracy, and 4.5x speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks.

Recommendations:

Try deleting some variables that will no longer be used during run-time
Use more efficient data structure
Get your hands on some optimized library for your hardware, e.g. if you are using Intel processors use the Intel distribution of python
Pay careful attention to your data types and try to trim them as much as possible

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Are Neural Net architectures accidental discoveries?
Recently, I have been learning about new neural networks, which are used for specialized purposes, like speech recognition, image recognition, etc. The more I discover the more I get amazed by the cleverness behind models such as RNN's and CNN's. Questions about working, intuition, mathematics have been asked a lot in this community, all with vague answers and apparent understandings.
So, my question is: did the researchers come up with these specialized models accidentally, or did they follow particular steps to get to the model (like in a mathematical framework)? And how did they look at a particular class of problem and think "Yeah, a better solution might exist"?
Since the understanding of NN's is so vague, these are 'high risk, high reward' scenarios, since you might be chasing only the mirage (illusion) of a solution.

Although there is a strong element of "try and see" that has driven successful architectures, the drivers for what to try are often inspired by underlying theory or knowledge from other disciplines.
Specifically for basic CNN, which led to AlexNet and many of the best image processing, the concept of using local receptive fields in layers was inspired by study of neurons in the cat visual system.
Modern RNNs also did not appear out of nowhere, there has long been an appreciation of the difference between a feed-forward network and a recurrently-connected one, and the different applications possible. The step change to LSTM was deliberate response to analysis of problems training simplest forms of RNN.
Like much of science, these things are also driven by success in the real world following the research. Many promising ideas have been tried and rejected. Some have been used for a while then superseded, e.g. using RBMs or stacked auto-encoders to pre-train deep networks before ReLUs and Xavier initialisation were discovered - although both RBMs and auto-encoders still have their niches.
Tweaks to architectures, such as variants of LSTM/GRU, may even be deliberately searched and assessed as part of research. That is done with the explicit knowledge that this part of finding a good design is best done as a search across possibilities.
Despite the evolution-like progress, presenting all such advances as completely random or pure GA-like search is ignoring the conscious effort and research that leads to the designs. If you search literature on any major successful design (such as the existence of RNNs or CNNs in the first place), and read the papers, you will often find that modern neural network architectures have deep roots in older research, plus have mathematical and/or scientific justifications for the choices made.

Researchers may follow specific mathematical frameworks, techniques to come-up with amazing works just like in any field, but I believe in Darwinian natural selection as a base theory for human's discoveries as well as for the Evolutionary Neural Net Architectures.

"Principle by which each slight variation [of a trait], if useful, is preserved".

A Novel Mathematical Framework for the Analysis of Neural Networks +PDF

How is Darwin relevant here? Are you saying older architectures are getting phased out by newer better architectures?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
CNN's vs Densely Connected NN's
In image classification we are generally told the main reason of using CNN's is that densely connected NN's cannot handle so many parameters (10 ^ 6 for a 1000 * 1000 image). My question is, is there any other reason why CNN's are used over DNN's (densely connected NN)?
Basically if we have infinite resources will DNN trump CNN's or are CNN's inherently well suited for image classification as RNN's are for speech. Answers based either on mathematics or experience on the field is appreciated.

The keyword here is Parameter Sharing or Weight Sharing across various image portions.
If we take a simple example of grayscale binary image of an alphabet 'F', it is a combination of multiple patterns. The patterns here are vertical lines and horizontal lines. These patterns are based on relation between intensities of contiguous cells. This relation between contiguous cells is established using a weight matrix.
Also, for identifying multiple horizontal lines, we dont need multiple node-sets in hidden dense layer trying to identify different horizontal lines in the image. The pattern is same but present in different locations. Hence the sharing of weights comes into picture.
In the 1st hidden layer, encode the pattern horizontal line in a weight matrix(learnt during training and used in testing). Place it over small grid and check for presence. As this matrix is slided and tested across the image, the presence of horizontal lines is marked in various locations. This weight matrix is called a kernel.
Combining the above points, kernel provides a way of handling parameter / weight sharing between contiguous cells to identify patterns. Dense layer instead of kernels would solve it eventually but start in a random manner. Since a efficient way was identified, it is being used.
Next to identify vertical lines, another kernel needed and slide across.
Suppose next we have dense layer as 2nd hidden layer. This layer looks for combination of patterns ('p' horizontal lines and 'q' vertical lines in this case for 'F') present and learns combinations to identify output.
Just to compare with traditional programming, kernels are like regular expressions. dense layers are like loops. Just sharing my thoughts. Any better explanation is welcome.

Convolution Neural Networks can detect more of the spatial features compared to Densely Connected Network. Consider this in any given real world image the pixel values of neighboring cells to do not vary  highly, But when this image are passed to a Densely Connected Neural network for training the spatial relations between neighboring pixels is lost as all other cells can heavily influence the training whereas in Convolutional networks due to operation of convolution by local information is preserved, it is called local connectivity.

That is not the actual reason , "convolution" layers are inspired by cells in visual-system. This is derived from the work of hubel-wiesel.
for more information check hubel-wiesel experiment.

It was 'inspired' but many systems exist which do not follow hubel weisel yet perform better, also hubel wesiel is hypothesised and I think is correct unless proved wrong

"It was 'inspired' but many systems exist which do not follow hubel wiesel yet perform better" which systems on what tasks? for almost all image classification related tasks convolutional networks perform better , and the main takeaway from hubel and wiesel is that visual system responds in a "local" manner .

the use of filters is solely for reducing computation...a dNN is also a filter...A dnn with many outputs for a single rgb channel can be easily achieved (basically a cnn but the filters at a particular layer is different for the entire image, one giant filter spanning the entire image, i can use as many giant filters and build a dnn), whereas main takeaway of cnn is edge detection, there is no reason for this densely connected system to not perform better than a common filter for all

"i can use as many giant filters and build a dnn" you would endup overfitting

That's not the point there are lot of methods to prevent that

http://yann.lecun.com/exdb/publis/pdf/lecun-bengio-95a.pdf      check page:5


text:
Can AI solve jumbled words?
Is there any general idea on how humans solve jumbled words? I know many people will say we match it against a commonly used words checklist mentally, but it is kind of vague. Is there any theory on this? And how might an AI learn to do the same?

I remember a problem similar to this  https://vladris.com/puzzles/facebook/puzzle_master/snack/breathalyzer.html
this is given as a problem in facebook engineering puzzles.
But i doubt that it is not much of a machine learning/AI problem. you could implement an algorithm that converts each word into a set of its characters , then pick the word in your master list with minimum distance based on its character list .
Even when humans solve jumble we do it in a systematic/algorithmic way , if the jumbled word is not present in our memory we can't do anything otherwise we can solve the jumble.
But human brain can simply recognize scrabled words if some of the structure is retained and not fully scrambled like an anagram .
We generally index words based on the first syllable/character in our brain , like  or even this
In the second picture we recognised the words even if they are not made of english characters because our brain doesn't scan the text character by character like "7-H " but treats it like an image.
So the model should not immediately classify the segmented characters but should find the "nearest" characters in every class that optimises the combined probability of characters of the word being one of the classes of words we have in our dictionary.

Great answer..But in the first example if you notice, all the complex words are kept with their starting letter untouched..Even for machines finding nearest distance can be a tough job speaking from purely combinatorics viewpoint..As far as I solve I just try to speak different combinations of the letter mentally and suddenly it matches with pre heard sound wave and we have a match

for making the model more human-like which by the way suits the first example , you could include an "attention" mechanism that focuses more on the parts of the words which humans emphasize more when reading

Like vowels I guess

or it could be framed as a reinforcement learning problem , where the reward is +1 when the solved word is in the dictionary  like in https://arxiv.org/pdf/1805.07470.pdf


text:
Can ML/AI understand incomplete constructs like humans?
We have AI's predicting images, predicting objects in an image. Understanding audio, meaning of the audio if it is a spoken sentence.
In humans when we start seeing a movie halfway through, we still understand the entire movie (although this might be attributed to the fact that future events in movies have a link to past events). But even if we see a movie by skipping lots of bits in-between we still understand the movie.
So can a Machine Learning AI do this? Or do humans have some inherent experiences in life which makes AI incapable of performing such a feat?

Central Questions
Can ML/AI understand incomplete constructs like humans?
Do humans have some inherent experiences in life which makes AI incapable of performing [some capacities of human intelligence]?
Comprehension of Literature and Film
Whether software exists today that is able to understand like humans is not something that the general public can know.  No such system has been released to the general public by any military or commercial organization thus far.  Yet such an achievement would not necessarily be something the government or commercial entity would want to disclose outside of the lab and its management.
Deeper comprehension of full or partial speech may have been achieved by anti-terrorist units, since funding has been available for that work for well over a decade, but it also unlikely that some software somewhere can read or scan parts of a book and do a book report that would produce a passing grade.
Determining whether a movie will return its investment at the box office may have been accomplished by the researchers for the big studios, but that doesn't require understanding like someone understands their favorite movie.  The answer to the title question is, "Not yet."
Will it occur?
Most in the fields of computer science, robotics, and artificial intelligence say, "Yes."  For religious reasons, some say, "No."  I'm in neither camp, and have not seen indisputable mathematical rigor that proves either the inevitability of artificial brains or the impossibility of them.  It is scientifically irresponsible to make a positive statement based on either recent technology trends or superstitious fears.
What are Humans Like?
The phrase, "Like humans," places a significant demand on researchers and engineers.  Human beings can do much more than work with data sets (audiovisual in this case) with missing information, what statisticians call sparse matrices.
Whether software can realize higher achievements of human brains is unknown, and the predictability of such capabilities presents gross difficulties.  Consider these human capabilities.

Write a screenplay, cast it with artificial characters, direct it, and produce it.
Initiate and partly develop a new branch of science, as did Isaac Newton or Lavoisier.
Love beyond a superficial expression of love

Conversely, what humans generally do poorly is distinguish between reliable projection and baseless conjecture.  Most humans are prone to musings of technical visionaries, propaganda, marketing, rumor, innuendo, and gossip.  It would not be surprising for software to someday soon be wiser in this respect, but only because the bar has been set so low by human culture.
There are other capabilities of the human brain that are exceptional and present enormous difficulties in even considering an approach to realizing in software.  These may be a result of a hundred thousand years of DNA refinement yielding significant complexity and precision.  It is also not outside the realm of possibility that a form of causality exists in the human brain that defies scientific study.
The Possibility of Impossibility
No one has every proven that all things that exist can be measured.  Heisenberg has actually proven the opposite to most theoretical physicist's satisfaction.
A phenomenon that cannot be measured at all cannot be studied scientifically.  Whether the phenomenon of choice is a unique condition has yet to be understood even in question form, prohibiting the emergence of a proper answer thus far.
Imminent Change as Significant as Industrialization
Nonetheless, important capabilities of human intelligence have been simulated and others are emerging.  These are now part of the world economy and will not likely disappear.  The possibility of Asimov like scenarios of robots and humans coexisting and conversing in more human-like is very likely.
It is when that occurs that autonomous vehicles and walking robots will begin to have experiences that are like human experiences and we will be able to directly observe just how much like humans they can behave in terms of intelligence and also emotions.
About the Specific Capabilities Mentioned
These are the capabilities mentioned in the question.

Predicting images
Predicting objects in an image
Understanding audio
Understanding the meaning of spoken sentences
Understanding a movie from the first half
Understanding a movie from parts of it

These are the more canonical ways of stating the first four capabilities for which research has produced usable system approaches.

Learning to distinguish image categories
Learning to distinguish object categories from within images
Parsing audio into notes, vocal tones, and transient sounds
Extracting semantics from a vocalization sufficiently to respond intelligently some of the time

These are a more accurate cognitive science description of the last two.

Guessing much better than random guessing would the story arch to the end of a movie from the sound and frame set of the its first half
Filling in character and story arch details from portions of a movie's sound and frame set

There is no obvious reason why such couldn't be done and done well by software, given sufficient research time to develop such a system and sufficient data to train with.  Also, significant computing resources would be needed, of course, and possibly some nontrivial period of time.

Humans are poor at distinguishing highly reliable and baseless conjencture..What does it mean?

I believe humans are prone to error only due to previous experiences, a scientific person will not view the news same as  a normal person, humans probably tend to romanticise things....also I did not understnd Heisenberg's role here, as far as I know you cannot measure position + momentum, only due to the effect of the thing which is measuring the quantity (i maybe wrong though)

I think it is a kind of combination of denial, giving their life a meaning and evolutionary advantage/prospects

I believe that an AI can understand much more of a movie than we do. As you said, sometimes a detail at the beginning of the movie is the key to understanding the final outcome. The problem is that the storyline distracts us. In the case of an AI, he would just be waiting for the movie to complete because he has all the details at all times.
Imagine that an AI is by your side watching the movie. He will not feel emotion like you. He is only analyzing the images, colors, soundtrack, main characters and is extracting from the lines, whatever he can, to understand the movie. What would it be to understand, in your eyes? To be able to generate a synopsis or to compare the film with some reality or criticizes the life of the human?
An artificial superintelligence would have the task of overcoming us. We could pass a movie and let her explain why we cried watching the movie, because we were surprised, and everything. Since an artificial superintelligence would have the role of not being distracted by the film but analyzing all possible content, try to analyze human reactions and spit out an analysis that could leave you reflecting on life for a few days.
But this is an artificial superintelligence that I imagine may one day exist, but I believe it is far away.
Something real and what we can build is an AI that can extract the genre from the movie and maybe even generate a synopsis with a spoiler like "X character dies."
Another factor that also makes it easier for us to understand movies is our huge film base. We watch so many American cliches movies, that most are already very predictable: "The main character does not usually die," "When the villain is close to killing the main character, something surprising happens and he is prevented," ...

But what about building the movie by watching it in bits?

Here is a glimpse for you.
Basing on what the second paragraph says;
In humans when we start seeing a movie halfway through, we still understand the entire movie (although this might be attributed to the fact that future events in movies have a link to past events). But even if we see a movie by skipping lots of bits in-between we still understand the movie.
We refer this to what is called "memory recall/retrieval and awareness of the current situation",hence attributes to remembering which can be thought of as an act of creative re-imagination, simply because of the way human memories are encoded and stored effectively.
note :memories are being triggered by biological neural networks(neurons).
So can a Machine Learning AI do this?

Recall and Recognition in an Attractor  Neural Network (ANN) model of memory retrieval and storage.
This paper was presented by E'Ruppin on 23.Oct.1996,at Tel Aviv University.

It gives an insight on how machine learning algorithms(ANN),pointed above can be applied in re-accessing of events or information bits from the past which have been previously encoded and stored,just like how the human brain does it.
However,if you also want to implement such conceptual idea here is another paper which explains the implementation of the above algorithm
"Neural Memory Networks" gives a little bit of insight on a simple implementation of memory in Neural networks.
Or do humans have some inherent experiences in life which makes AI incapable of performing such a feat?(This question is kinda like two sided).
To some extent,humans don't inherent experiences but rather,past information is triggered by some situations they do face,for instance; I myself experienced some music rhythms way back, and when I try to replay my old jams again,i recall back that situation nor memories.
Hope this can give you a little bit of insight for you to go indepth,concerning "Recall and Recognition in an Attractor  Neural Network" (ANN).


text:
Using a neural network for learning a Motion Graph?
In a recent paper about progress in computer animation a so called motion graph is used to describe the transition between keyframes of facial animation. Easy Generation of Facial Animation Using Motion Graphs, 2018 As far as i understand from the paper, they used a motion capture device to record faces of real people and extract keyframes. Then a transition matrix was created to ensure that a walk from keyframe #10 to #24 is possible but a transition from keyframe #22 to #99 is forbidden.
The idea itself sounds reasonable good, because now a solver can search in the motion graph to bring the system from a laughing face to a bored face without interruption or unnatural in-between-keyframes. But wouldn't it be great if the transition matrix can be stored inside a neural network? As far as i understand the backpropagation algorithm , the neural network can learn input-output relations. So the neural network has to learn the transition probability between two keyframes. And a second neural network can then produce the motion plan which is also be trained by a large corpus. Is that idea possible or is it the wrong direction?

... the neural network has to learn the transition probability between two keyframes. And a second neural network can then produce the motion plan which is also be trained by a large corpus. Is that idea possible or is it the wrong direction?

That would be trivial.
More difficult, but not hard, would be to take a 14 hour video of someone and create a new video with them saying something they have never said previously. Here is a demonstration, from the BBC YouTube channel, titled: "Fake Obama created using AI video tool - BBC News".
The University of Washington demonstrates, using a variety of clips and only audio from an impersonator, that they can create realistic videos of the former president saying anything they want.
It's possible to sketch out an idea and let AI create a finished product of excellent quality.
The YouTube channel ACMSIGGRAPH has a video titled: "Technical Papers Preview: SIGGRAPH 2018", which is discussed on the webpage: "Could These Be The Next High-Tech Tools That Animators Use Daily?".
What you have asked about was possible long ago and not particularly difficult to do.


text:
Are AI algorithms capable of self-repair?
Do AI algorithms exist which are capable of healing themselves or regenerating a hurt area when they detect so?
For example: In humans if a certain part of brain gets hurt or removed, neighbouring parts take up the job. This happens probably because we are biologically unable to grow nerve cells. Whereas some other body parts (liver, skin) will regenerate most kinds of damage.
Now my question is does AI algorithms exist which take care of this i.e. regenerating a damaged area? From my understanding this can be achieved in a NN using dropout (probably). Is it correct? Do additional algorithms (for both AI/NN) or measures exist to make sure healing happens if there is some damage to the algorithm itself?
This can be particularly useful in cases where say there is a burnout in a processor cell  processing some information about the environment. The other processing nodes have to take care to compensate or fully take-over the functions of the damaged cell.
(Intuitionally this can mean 2 things:

We were not using the system of processors to its full capability.
The performance of the system will take a hit due to other nodes taking over functionality of the damaged node)

Does this  happen in the case of brain damage also? Or is my inferences wrong? (Kindly throw some light).
NOTE : I am not looking for hardware compensations like re- routing, I am asking for non-tangible healing. Adjusting the behavior or some parameters of the algorithm.

Good question.  It is related to the genetic algorithm concept, automated bug detection, and continuous integration.
Early Genetically Inspired Algorithms
Some of the Cambridge LISP code in the 1990s worked deliberately toward self-improvement, which is not the same as self-repair, but the two are conceptual siblings.
Some of those early LISP algorithms were genetically inspired but not pure simulations of DNA mutation with natural selection through sexual reproduction.  A few of these evolution-like algorithms evaluated their own effectiveness based on a fixed effectiveness model.  The effectiveness model would accept reported objective metrics at run time and analyze them.  When the analysis returned an assessment of effectiveness below a minimum threshold, the LISP code would perform this procedure.

Copy itself (which is easy in LISP)
Mutate the algorithm in the copy according to some meta-rules
Run the mutation in parallel as a production simulation for a while
Check of the effectiveness of mutation out performed its own

If the mutation was gauged as more effective, it would perform four more selfless steps.

Make a record of itself
Attach its own performance for later meta-rule use
Load the mutation it created in its own place
Perform apoptosis

Unlike biological apoptosis, apoptosis in these algorithms simply pass computational resources and run time control to the mutation that was loaded.
This procedure was and probably still is easier in LISP than in other languages, although lovers of other languages would argue endlessly that point.
Extensions of Continuous Integration
This is also the closed loop continuous improvement strategy intended when bug reporting is integrated with continuous integration development platforms and tools.  We see extensions of continuous integration in the feeding of bug lists from automated detection, especially for crashes, in many applications, frameworks, libraries, drivers, and operating system today.  Many of the elements of closed loop self-repair are already in general practice among the most progressive development teams.
The bug fixes themselves are not yet automated in the way researchers were attempting in the LISP code above.  Developers and team leaders are following a process similar to this.

Developer or team lead associates (assigns) bug to developer
Developer attempts to replicate the bug with the corresponding version of the code
If replicated, the root cause is found
A design for a fix occurs at some level
The fix is implemented

If continuous integration and proper configuration management is in place, at the point when a commit of the change to the team repository occurs, it is applied to the correct branches and the test suite of unit, integration, and functional tests is run to detect any breakage that the fix may have caused inadvertently.
Several Pieces of Full Automation are Already in Use
As one can see, many of the pieces are in place for automatic algorithm, configuration, and deployment package self-repair.  There are even projects underway in several corporations to automatically create functional tests by recording user behavior and user answers to questions like, "Was this helpful?"
What is Missing
What needs further development to more completely see full life cycle self-improving and self-repairing software?

Automatic bug replication
Automatic unit test creation
Automatic repair design
Automatic creation of code from design

Next Steps
I suggest that the next steps to be done are these.

Assess work already done on the four missing automations above
Review the LISP procedure that was perhaps shelved in the 1990s, or perhaps not, since we cannot see (and should not see) what was classified or made company confidential)
Consider how the machine learning building blocks that have emerged within the last two decades may help
Find stakeholders to provide project resources
Get working

A Note on Demand, Ethics, and Technological Displacement
Truth be told, the quality of software was a problem in the 1980s, 1990s, 2000s, and 2010s.  Just today, I found over a dozen bugs in software that is considered a stable release, when performing some of the most basic functions the software was designed to do.
Given bug list sizes, just as accidents make the question of whether humans should be driving cars questionable, whether humans should maintain software quality is questionable.
Humanity has survived replacement in a number of things already.

Arithmetic with a pencil and eraser is gone
Professional farming with garage tools is gone
Creating advertising mechanicals with Exacto knives is gone
Sorting mail by hand is gone
Communicating by horse-back courier is gone

Few software engineers are happy just fixing bugs.  They seem to be happiest creating new software filled with bugs that someone else is supposed to fix.  Why not let that someone else be artificial?

Yes, this was an active area of research in a number of different AI fields.
Probably the most directly related work is Bongard, Zykov & Lipson's self-repairing robots from the early 2000's.
There's some more recent work from Mark Yim that you can see here too.
There are lots of different ways to do this, but Bongard et al's approach was probably the most elegant. The basic idea was to frame it as a learning problem: the robot is able to learn the shape of its body by performing controlled experiments. When the body is damaged, the robot can detect that it's body has changed shape (sensors don't report the expected values when it tries to move), perform new experiments to determine the extent of the damage, and then generate new movements that work around the damaged area. Lipson covers the basics of this system very briefly in this video.
The more modern system uses a similar approach, but tries to repair its body, rather than working around the damage. It's got an internal model of what it's body should look like, and then a set of cameras that help it locate the various pieces and move them to reassemble.
Dropout is sort of a similar idea, but dropout is usually done to encourage redundancy during training, which can help a model avoid overfitting. It's usually not done explicitly to heal a damaged system, although it would make a system more resistant to damage in the first place.

The question and the example are a few contradictory.
The example is about a physical brain damage. Computer systems with the ability to self-repair exists from 1970's. They can repair a damaged disk (RAID), replace a CPU by an idle one (active/passive), mark faulty memory blocks, redirect network traffic from broken links to available ones, ... nowadays near than all hardware failures are covered.
However, the question is about "algorithms capable of healing themselves", that has a parallelism in "persons capable of healing from a psychological problem".
Like in the case of persons, it depends of the problem, and the amount of recovery expected.
Some easier cases are:

Lots of non-AI systems has the ability to re-synchronize, auto-calibration, ...
Any minimal intelligent system can "stop" if it detects it is producing continuous wrong results.

Going a step forward, thinking in ML (Neural Nets, ..) we can remark that all unsupervised learning machines can recover from a misalignment of their parameters, just re-executing the learning process (or continuously executing it).
Finally, we could ask "can a machine recover from an error in his reward function" ? And, at this point, my answer is "I do not known any system able of that, because they have no common sense".


text:
How do I keep track of already visited states in breadth-first search?
I was trying to implement the breadth-first search (BFS) algorithm for the sliding blocks puzzle (number type). Now, the main thing I noticed is that, if you have a $4 \times 4$ board, the number of states can be as large as $16!$, so I cannot enumerate all states beforehand.
How do I keep track of already visited states? I am using a class board each class instance contains a unique board pattern and is created by enumerating all possible steps from the current step.
I searched on the net and, apparently, they do not go back to the just-completed previous step, BUT we can go back to the previous step by another route too and then again re-enumerate all steps which have been previously visited.
So, how to keep track of visited states when all the states have not been enumerated already? Comparing already present states to the present step will be costly.

You can use a set (in the mathematical sense of the word, i.e. a collection that cannot contain duplicates) to store states that you have already seen. The operations you'll need to be able to perform on this are:

inserting elements
testing if elements are already in there

Pretty much every programming language should already have support for a data structure that can perform both of these operations in constant ($O(1)$) time. For example:

set in Python
HashSet in Java

At first glance, it may seem like adding all the states you ever see to a set like this will be expensive memory-wise, but it is not too bad in comparison to the memory you already need for your frontier; if your branching factor is $b$, your frontier will grow by $b - 1$ elements per node that you visit (remove $1$ node from frontier to "visit" it, add $b$ new successors/children), whereas your set will only grow by $1$ extra node per visited node.
In pseudocode, such a set (let's name it closed_set, to be consistent with the pseudocode on wikipedia could be used in a Breadth-First Search as follows:
frontier = First-In-First-Out Queue
frontier.add(initial_state)

closed_set = set()

while frontier not empty:
    current = frontier.remove_next()

    if current == goal_state:
        return something

    for each child in current.generate_children()
        if child not in closed_set:    // This operation should be supported in O(1) time regardless of closed_set's current size
            frontier.add(child)

    closed_set.add(current)    // this should also run in O(1) time

(some variations of this pseudocode might work too, and be more or less efficient depending on the situation; for example, you could also take the closed_set to contain all nodes of which you have already added children to the frontier, and then entirely avoid the generate_children() call if current is already in the closed_set.)

What I described above would be the standard way to handle this problem. Intuitively, I suspect a different "solution" could be to always randomize the order of a new list of successor states before adding them to the frontier. This way, you do not avoid the problem of occasionally adding states that you've already previousl expanded to the frontier, but I do think it should significantly reduce the risk of getting stuck in infinite cycles.
Be careful: I do not know of any formal analysis of this solution that proves that it always avoids infinite cycles though. If I try to "run" this through my head, intuitively, I suspect it should kind of work, and it does not require any extra memory. There may be edge cases that I'm not thinking of right now though, so it also simply might not work, the standard solution described above will be a safer bet (at the cost of more memory).

Comments are not for extended discussion; this conversation has been moved to chat.

Dennis Soemers' answer is correct: you should use a HashSet or a similar structure to keep track of visited states in BFS Graph Search.
However, it doesn't quite answer your question. You're right, that in the worst case, BFS will then require you to store 16! nodes. Even though the insertion and check times in the set will be O(1), you'll still need an absurd amount of memory.
To fix this, don't use BFS. It's intractable for all but the simplest of problems, because it requires both time and memory that are exponential in the distance to the nearest goal state.
A much more memory-efficient algorithm is iterative deepening. It has all the desirable properties of BFS, but uses only O(n) memory, where n is the number of moves to reach the nearest solution. It might still take a while, but you'll hit memory limits long before CPU-related limits.
Better still, develop a domain specific heuristic, and use A* search. This should require you to examine only a very small number of nodes, and allow the search to complete in something much closer to linear time.

Comments are not for extended discussion; this conversation has been moved to chat.

While the answers given are generally true, a BFS in the 15-puzzle is not only quite feasible, it was done in 2005! The paper that describes the approach can be found here:
http://www.aaai.org/Papers/AAAI/2005/AAAI05-219.pdf
A few key points:

In order to do this, external memory was required - that is the BFS used the hard drive for storage instead of RAM.
There are actually only 15!/2 states, since the state space has two mutually unreachable components.
This works in the sliding tile puzzle because the state spaces grows really slowly from level to level. This means that the total memory required for any level is far smaller than the full size of the state space. (This contrasts with a state space like Rubik's Cube, where the state space grows much more quickly.)
Because the sliding-tile puzzle is undirected, you only have to worry about duplicates in the current or previous layer. In a directed space you may generate duplicates in any previous layer of the search which makes things much more complicated.
In the original work by Korf (linked above) they didn't actually store the result of the search - the search just computed how many states were at each level. If you want to store the first results you need something like WMBFS (http://www.cs.du.edu/~sturtevant/papers/bfs_min_write.pdf)
There are three primary approaches to comparing states from the previous layers when states are stored on disk.

The first is sorting-based. If you sort two files of successors, you can scan them in linear order to find duplicates.
The second is hash-based. If you use a hash function to group successors into files, you can load files which are smaller than the full state space to check for duplicates. (Note that there are two hash functions here -- one to send a state to a file, and one to differentiate states within that file.)
The third is structured duplicate detection. This is a form of hash-based detection, but it is done in a way that duplicates can be checked immediately when they are generated instead of after they have all been generated.

There is a lot more to be said here, but the paper(s) above give a lot more details.

How would undirected help you avoid duplicates in other layers? Surely you'd be able to get back to a node in another layer by moving 3 tiles in a circle. If anything, directed would help you avoid duplicates, because it's more restrictive. The linked paper talks about duplicate detection, but doesn't mention undirected or directed at all, nor seems to mention avoiding duplicates at different levels (but I could've missed that in my very brief scan of it).

@NotThatGuy In an undirected graph a parent and a child are at most distance 1 apart in the depth they are found in the BFS. This is because once one is found, the undirected edge guarantees that the other will be found immediately afterwards. But, in a directed graph, a state at depth 10 can generate children at depth 2, because the child at depth 2 doesn't have to have an edge back to the other state (this would make it depth 3 instead of depth 10).

@NotThatGuy If you move 3 tiles in a circle you create a cycle, but a BFS will explore that in both directions simultaneously, so it won't actually take you back to the much shallower depth. The full 3x2 sliding tile is shown in this demo, and you can track the cycles to see how they occur: https://www.movingai.com/SAS/IDA/

Ironically the answer is "use whatever system you want."  A hashSet is a good idea.  However, it turns out that your concerns over memory usage are unfounded.  BFS is so bad at these sorts of problems, that it resolves this issue for you.
Consider that your BFS requires you to keep a stack of unprocessed states.  As you progress into the puzzle, the states you deal with become more and more different, so you're likely to see that each ply of your BFS multiplies the number of states to look at by roughly 3.
This means that, when you're processing the last ply of your BFS, you have to have at least 16!/3 states in memory.  Whatever approach you used to make sure that fit in memory will be sufficient to ensure your previously-visited list fits in memory as well.
As others have pointed out, this is not the best algorithm to use.  Use an algorithm which is a better fit for the problem.

Approaches to the Game
It is true that the board has $16!$ possible states.  It is also true that using a hash set is what students learn in a first year algorithms courses to avoid redundancy and endless looping when searching a graph that may contain graph cycles.
However, those trivial facts are not pertinent if the goal is to complete the puzzle in the fewest computing cycles. Breadth first search isn't a practical way to complete an orthogonal move puzzle.  The very high cost of a breadth first search would only be necessary if number of moves is of paramount importance for some reason.
Sub-sequence Descent
Most of the vertices representing states will never be visited, and each state that is visited can have between two and four outgoing edges.  Each block has an initial position and a final position and the board is symmetric.  The greatest freedom of choice exists when the open space is one of the four middle positions.  The least is when the open space is one of the four corner positions.
A reasonable disparity (error) function is simply the sum of all x disparities plus the sum of all y disparities and a number heuristically representing which of the three levels of freedom of movement exists because of the resulting placement of the open space (middle, edge, corner).
Although blocks may temporarily move away from their destinations to support a strategy toward completion requiring a sequence of moves, there is rarely a case where such a strategy exceeds eight moves, generating, on the average, 5,184 permutations for which the final states can be compared using the disparity function above.
If the empty space and positions of block 1 through 15 are encoded as an array of nibbles, only addition, subtraction, and bit-wise operations are needed, making the algorithm fast.  Repeating the eight move brute force strategies can be repeated until disparity falls to zero.
Summary
This algorithm cannot cycle because there is always at least one of the permutations of eight moves that decreases disparity, regardless of the initial state, with the exception of a starting state that is already complete.


text:
How to tinker with CNN architectures?
I was thinking of creating a CNN. Now it is known CNN takes long times to train so it is advisable to stick to known architectures and hyper-parameters.
My question is: I want to tinker with the CNN architecture (since it is a specialised task). One approach would be to create a CNN and check on small data-sets, but then I would have no way of knowing whether the Fully Connected layer at the end is over-fitting the data while the convolutional layers do nothing (since large FC layers can easily over-fit data). Cross Validation is a good way to check it, but it might not be satisfactory (since my opinion is that a CNN can be replaced with a Fully Connected NN if the data-set is small enough and there is little variation in the future data-sets).
So what are some ways to tinker with CNN and get a good estimate for future data-sets in a reasonable training time? Am I wrong in my previous assumptions? A detailed answer would be nice!

There is no easy way to play around with the hyper-parameters (number of layers, layer configuration, number of outputs per layer) of a CNN and get an accurate view of how these will affect the resulting performance of your model. However there are a few things that you can do to avoid wasting too much time training and re-training.
Why?
When training a CNN we aim to minimize a loss function, thus a better CNN model is defined as one which converges to a set of model parameters with a lower loss function. Identifying the minimum of the loss function for a given CNN is already very difficult, and there is no guarantee that the true minimum will every be reached by fault of gradient descent.
Each variation of the CNN resulting from the different hyper-parameters will still result in a very large number of model parameters. There is no way to know how the loss function will look in this hyper-dimensional space and even harder to estimate its minimum.
What to do?
First, you should try and understand how each layer in the network affects its inputs. You should know what kind of layers to use for what kind of data. You should also know what kinds of activation functions to use for different data distributions. You should also know how many model parameters per layer to try in order to sufficiently compress your data whilst not losing significant information.
You can get a lot of this intuition by reading papers which have found successful models for specific tasks.
In addition, you can train with smaller amounts of data and estimate the potential minimum loss function by seeing how fast the loss function moves towards its minimal value (momentum). Usually a lower minimum is achieved when the loss function decreases faster. However, this is in no way always true. A loss function can converge slowly at first and then speed up later. This is entirely possible. But, you can get some sense of the potential of your model in this way.

Loss function is actually not a good measure of performance since over fitting can also cause very low losses

@DuttaA, always make sure to look at the loss function of your validation or test set, not that of your training set.


text:
What is the physics engine used by DeepMimic?
I found a video for the paper DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills
 on YouTube.
I looked in the related paper, but could not find details of how to the environment was created, such as the physics engine it used. I would like to use it, or something similar.

Its better you analyse your question inline with community guidelines.

@neilslater Any idea what software stack was used besides physics engine?

@gfdsal No, but I didn't answer this question. Your comment could be directed to user12075 on the answer, or you could start a new question given that this one was asked and answered over 18 months ago (the two posters might not be available here today).

@NeilSlater yup, thats why I tagged you directly. I shall start a new question.

Bullet physics engine
Their paper says
Physics simulation is performed at 1.2kHz using the Bullet physics engine [Bullet 2015].


text:
Does a solution for Wumpus World with neural networks exist?
The Wumpus World proposed in book of Stuart Russel and Peter Norvig, is a game which happens on a 4x4 board and the objective is to grab the gold and avoiding the threats that can kill you. The rules of game are:

You move just one box for round
Start in position (1,1), bottom left
You have a vector of sensors for perceiving the world around you.
When you are next to another position (including the gold), the vector is 'activated'.
There is one wumpus (a monster), 2-3 pits (feel free to put more or less) and just one gold pot
You only have one arrow that flies in a straight line and can kill the wumpus
Entering the room with a pit, the wumpus or the gold finishes the game

Scoring is as follows: +1000 for grabbing the gold, -1000 for dying to the wumpus, -1 for each step, -10 for shooting an arrow. Fore more details about the rules, chapter 7 of the book explains them.
Well now that game has been explained, the question is: in the book, the solution is demonstrated by logic and searching, does there exist another form to solve that problem with neural networks? If yes, how to do that? What topology to use? What paradigm of learning and algorithms to use?
1*: My English is horrible, if you can send grammar corrections, I'm grateful.
2*: I think this is a bit confusing and a bit complex. if you can help me to clarify better, please do commentary or edit!

Yes! If you read ahead to the chapters in reinforcement learning in the same book, you'll see that the wompus world appears again there. Techniques like Q-learning can be used to solve it, and since Q-learning involves learning the shape of a function, a neural network can be employed as a function approximator.
The basic idea is to treat this problem as an input/output mapping (states -> actions), and to learn which actions produce the greatest rewards.
Note however, that these approaches rely on trial and error. The logic based approach reasons about the rules of the game, and can play reasonably well right away. The learning approach will need to try and fail many times before playing well.


text:
What is the difference between the study of evolutionary algorithms and optimization?
I have a course named "Evolutionary Algorithms", but our teacher is always mentioning the word "optimization" in his lectures.
I am confused. Is he actually teaching optimization? If yes, why is the name of the course not "Optimization"?
What is the difference between the study of evolutionary algorithms and optimization?

Optimization problems are formally defined by two things:

Optimization objective: $$\min_{x \in \mathbb{R}} f(x)$$
List of constraints: $\text{s.t. } x > 0$; $\text{s.t. } x^2 < 100$, ...

Optimization theory as a field deals with variations of such problems. The algorithms that are part of optimization include gradient descent and variants, the simplex algorithm, simulated annealing and many more. I would also include evolutionary algorithms as one of the algorithms in this field.

I am confused. Is he actually teaching optimization? If yes, why is the name of the course not "Optimization"?

Evolutionary algorithms are a subfield of optimization. The name of the course is evolutionary algorithms as you likely don't deal with the other aspects of optimization theory (e.g. the algorithms mentioned above).


text:
Are 1D CNNs really the appropriate model for human activity recognition?
This article on human activity recognition states that 1D convolutional neural networks work the best on the classification of human activities using data from the accelerometer. But I think that human activities, like swinging the arm, are sequential actions and they require LSTMs.
So, which one should be more effective for this task: CNNs or LSTMs? In other words, is spatial learning required or sequence learning?

Human activity recognition, for example as done with the Sports-1M Dataset, is a classification task. Given a video, say which activity / action was executed.
So it is a special case about video classification.

human activities like swinging the arm are sequential actions and they require LSTMs

This is wrong. As a simple example, think of handwriting recognition as done on write-math.com: I receive a sequence of points (x, y) -- the pen-tip --  and classify which symbol is written. So it is a sequence, but still there a plenty of ways to use a simple multi-layer perceptron (MLP) for it. Two big groups of approaches:

Feature-Engineering: Instead of giving an arbitrary number of coordinates, one can normalize it to a fixed amount. This is what is currently done on write-math.com and works pretty well (see my Bachelors thesis)
Space Transformation: Similar to the feature engineering, but more extreme. Instead of looking at (variants of) coordinates, one can look at the image which is there in the end. This way one can apply CNNs.

You can do similar things for movies:

Multiple Classifications: Classify based on single frames and average the results
Feature-Engineering: Normalize the length to a fixed number; interpret the fixed-length video as an image with more channels.
Use CNNs as feature extractors / for dimensionality reduction and LSTM for the sequence classification.

You might be interested in Beyond Short Snippets: Deep Networks for Video Classification
Efficiency vs Effectiveness
Model A is more effective than model B, if its optimizarion criterion is better.
Model A is more efficient than model B, if it uses less resources (e.g. memory) to achieve the same optimization value (e.g. same accuracy / MSE)


text:
Using unsupervised learning for classification problems
Let's say there are two types of cancer(Type 1 and Type 2). Say we want to see if one of pour friends has cancer Type 1 or 2. We can treat this as a classification problem. But what if we use unsupervised learning (clustering) to separate the data into to 2 different groups and see each whether each item in group 1 belongs to a person with cancer Type 1 or 2. We will then  see whether our friend belongs to group 1 or 2. I know it is stupid to do this and we have to do extra work but can we even do this?
Let's say that the features are only the age and the height (I know it's really dumb but just bear with me). The data associated with people with cancer Type 1 is [10, 150], [12, 153], [9, 143], [13, 160] and for people with Type 2 cancer : [20, 175], [23, 180], [19, 174]. Let's say we plot the data on a graph (without labelling the data) and the unsupervised program (Clustering) just separates the two groups (Say group 1 for Type 1). We then can see that to whom each data in group 1 belongs. We see those people have cancer Type 1. So given new data, we see what group our friend  belongs to. If she/he belonged to group 1, he's got cancer Type 1 and if not, she/he has cancer Type 2.

I think you're quite confused regarding the nature of the data. Let me ask you a question. What data would you use to classify if a person has a certain cancer type? How does the data look like? What are the features, if any?

@nbro Let's say that the features are only the age and the height(I know it's really dumb but just bear with me). The data associated with people with cancer Type 1 is [10, 150], [12, 153], [9, 143], [13, 160] and for people with Type 2 cancer : [20, 175], [23, 180], [19, 174]. Let's say we plot the data on a graph(Without labeling the data) and the unsupervised program(Clustering) just seperates the two groups(Say group 1 for Type 1).We then can see that to whom each data in group 1 belongs.We see those people have cancer Type 1.So given new data, we see what group they belong to.

You cannot use clustering as an "unsupervised classifier", with the goal of targeting specific known classes that are already defined. For that you need supervised learning and a labelled dataset.
A clustering algorithm would only separate into clusters as you hope if the feature variables you use are already:

All strongly associated with the trait you want the clusters to find
Carefully scaled to match the expected relationship

Now this might happen by chance with an arbitrary dataset. However in practice in a complex dataset with many features (as you might use in cancer classification) you would need to select and scale just relevant features. Thus you would need to already know the relationship between features and class from some other model.
The main task of clustering is to discover naturally occurring groups within your data, that might give you insight into the distributions within a dataset. This might let you compress or summarise information about examples. The summary feature might even be predictive, but it would very rarely overlap with pre-defined classes, even when the full feature vector could be used in an accurate supervised learning predictive model.
With your given example of [age, height] you might discover that the population splits naturally into people who are still growing, fully-grown youths and mature people. If this happens to overlap with predictions of type 1 and type 2 cancer that might be interesting - perhaps a person's cluster is even a good predictive variable for having type 1 or type 2 cancer. However, whatever the features are, it is highly unlikely that it would be so strong a predictive variable that you would use it on its own to make a classification. The only variables that would be that predictive would be the clinical measurements of the cancer growth that clinicians already use to decide the cancer type.
Potentially you could build a Bayesian model around the clusters, where you can ask questions such as:

What is the probability that a random person is in cluster A $p(x \in A)$?
What is the probability that a random person has cancer type 1 $p(x \in C_1)$?
What is the conditional probability that a person with type 1 cancer is found in cluster A $p(x \in A|x \in C_1)$?

Bayes' Theorem would then allow you to calculate the probability of having type 1 cancer based on their clustering:
$$p(x \in C_1 | x \in A) = \frac{p(x \in A|x \in C_1)}{(x \in A)}p(x \in C_1)$$
To do this fully with Bayesian statistics would involve setting some prior beliefs about the probabilities and using the data from your analysis to adjust them, which is beyond the scope of this answer. However, you could as a crude shortcut just take the measured frequencies, and that might be fine as a guide and give you some intuitions about what your clustering is doing towards helping with predictions.

There are several ways to use clustering for classification.

To use features associated with classes, then do clustering and find the relationships between clusters and classes.
To use classes in the training set as clusters and to classify each new data vector by its closeness to each cluster.
To use class label as one of features for clustering. Then, find association between clusters and classes.


text:
Can a LFSR be approximated by a Neural Network?
I was wondering whether a LFSR could be approximated by a NN (output or current state). We know that a LFSR is called linear in some sort of mathematical sense, but is that true? Considering it follows Galois field mathematics. So can a Neural Network approximate a LFSR?
Answers with mathematical proof or actual experience is preferred.

According to the abstract of the following book, yes it's possible:
"Prediction of Sequences Generated by LFSR Using Back Propagation MLP"
Abstract:
Prediction of the next bit in pseudorandom sequences is one of the main issues in cryptology in order to prove the robustness of the systems. Linear complexity has served as a reference measurement to evaluate the randomness of the sequences, comparing them with the shortest LFSR that can generate those sequences. Several tools based on artificial intelligence have also been used for the next bit prediction, such as the C4.5 classifier. In this paper, we apply a different approach, the back propagation neural networks, to predict the sequences generated by LFSR. The results confirm that these networks can predict the entire sequence knowing less input patterns than techniques based on classifiers.


text:
Is it possible to generate "Karel the robot" programs with genetic programming?
Karel the robot is an education software comparable to turtle graphics to teach programming for beginners. It's a virtual stack-based interpreter to run a domain-specific language for moving a robot in a maze. In its vanilla version, the user authors the script manually. That means, he writes down a computer program like

move forward
if reached obstacle == true then stop
move left.

This program is then executed in the virtual machine.
In contrast, genetic programming has the aim to produce computer code without human intervention. So-called permutations are tested if they are fulfilling the constraints and, after a while, the source code is generated. In most publications, the concept is explained on a machine level. That means assembly instructions are generated with the aim to replace normal computer code.
In "Karel the robot" a high-level language for controlling a robot is presented, which has a stack, but has a higher abstraction. The advantage is, that the state space is smaller.
My question is: is it possible to generate "Karel the robot" programs with genetic programming?

Yes. The question is, whether they produce any useful output. Your 'fitness function' could be a parser which checks whether your programs are syntactically well-formed, but the output would probably be fairly random. Unless you incorporate the desired output into the fitness function in some way.

A genetic algorithm is used to find an optimal solution to a problem. The parameters of the problem are encoded in the population, where each individual contains a solution to the problem in itself. Each individual is evaluated using a fitness function, and then the highest scoring individuals are used to create a new set of individuals by mixing up their parameters, and this new set replaces the worst scoring individuals from the previous generation. Over time, the average fitness of the population will (hopefully) increase, and a good solution be found. Essentially, the search space is explored in parallel through the many individuals in the population, which means that local maxima can probably be avoided.
The question now is: What is the problem you want to optimise?
The Karel language is fairly simple, and has few tokens. A program should be easily encodable in the right format, though genetic algorithms usually require individuals to have the same size (though shorter programs could be padded with no-ops). Recombining the best programs could mean swapping tokens. Maybe syntactic constraints could be factored into this to avoid crass syntactic errors. That should not be an insurmountable problem.
However, that still leaves the question of the fitness function. A basic criterion would be syntactic: if the program doesn't parse, it's not 'fit'. This could potentially wipe out a lot of offspring, and doesn't really help you much: it would produce more or less random programs which are syntactically well-formed.
What you really want is to evaluate the outcome of the programs. Like, which program can best navigate a maze. So you need to execute them and see what they produce. Then you run straight away into the halting problem: your little program might contain an endless loop and will never terminate. So you need to add a kind of time-out, or a limit on the steps the program can execute. This should be easy to do in a virtual machine.
So, to answer your question: Yes, it is possible to turn creating Karel programs into a problem that a genetic algorithm can be applied to. However, you need to be clear about what problem you want to solve. I don't quite understand what you mean by "learning from demonstration" or how you can achieve the aim of replicating human actions, but if you can somehow encode that in a fitness function, GAs are a tool you can use.

To generate high level programming language code in the context of genetic programming the Grammatical Evolution technique could be a good start. It allows to generate syntactically correct samples according to a grammar, so there will be no (syntactic) garbage in a population.
In the original implementation it has (very simple and) quite destructive mutation and crossover operators. This could be changed by making the operators more sophisticated, so they respect the actual tree-like structure of the samples and the grammar constraints, but effectively it will result in implementing the classical tree-based Genetic Programming system (which isn't bad).
The evaluation of such samples should be done by executing them in an appropriate environment (the actual VM and the desired map or whatever).

Is it possible to generate "Karel the robot" programs with genetic programming?

Yes, it is possible.
Robocode is a programming game where players program a tank-like robot to battle other robots in an arena. This has some similarities to "Karel The Robot" (i.e. the player programs a robot that has to perform a task in an arena). There is a paper describing the use of genetic programming in creating Robocode programs: GP-Robocode: Using Genetic Programming to Evolve Robocode Players by Yehonatan Shichel, Eran Ziserman, and Moshe Sipper.
In Genetic Programming: On the Programming of Computers by Means of Natural Selection by John R. Koza, Appendix B, there is an example solution to a "discrete non-hamstrung squad car game". Description of the game:

The discrete non-hamstrung squad car is a game of perfect information involving two players moving on a checkerboard grid of indefinite size. The pursuing player is a squad car with a speed advantage over the pedestrian evader. The object is to discover a strategy for the pursuer that results in capture of the evader in optimal (i.e., minimal) time.

The Common Lisp source code of a genetic programming solution is also available from the author's website: http://www.genetic-programming.org/gplittlelisp.html. The genetic programming representation used to solve this problem could conceivably be adapted to "Karel The Robot" problems.


text:
How can I prevent the CNN from classifying a new input into one of the existing labels (it was trained with) when the input has a new different label?
I'm trying to perform image classification with a CNN. In my case, the inputs are the covers of 9 books, so there are 9 labels. I am using TensorFlow's Keras.
If I pass a new input (that has a label different than one of the 9 labels the CNN was trained with), it will be classified as one of the 9 books, even though it's not a book (but it's e.g. a wall, sofa, house, etc.). I want to avoid this. I want the model to first classify whether there is a book in the image and then classify the book in 9 classes. How could I achieve this?

You can introduce another class to your network - "not a book". After that, you will need to add new data to your dataset, random images that do not contain books to classify and train your network on that data. So when your network won't see a book it will output high probability for "not a book" class, if an image with a book will be shown to the network probability of the "not a book" class should be low.

I'm wondering which would work better: adding a new class of "not a book" or first of all classifying between "book" and "not a book" and then only passing the image to the book classifier if it first of all classifies as "book"?

Well, with this approach, you already classify an image as "book" or "not a book". I think having one ML model to do this is better since if you will introduce another model the number of correctly predicted classes will depend on two models and you will need to fine tune them both. For example, your first model may fail to categorize an image as a book when there is a book and it will not be passed to the 2nd model. So with this approach, you will need to worry only about one model, not two.


text:
Python Network for Simple Image Classification
I'm wondering if there exists a network for simple image classification. What I mean by this is if I have two image datasets, one of horses and one of zebras, I want to train off the horses and classify an image as either a horse or not a horse, so if I test it on an image of a horse, it says it is a horse, but if I use a zebra, it says it is not a horse. Does any library/project for this exist?

Using neural networks
If it is as simple as that, probably a neural network consisting of just a simple convolutional network with a few filters, a fully connected layer with 2 neurons, and a softmax layer, might do it.
To develop that, you can use Keras wrapper over tensorflow or theano, but I think scikit-learn also provides some basic neural network functionalities.
Depending on the simplicity of the images on your dataset, you might do it even with no convolutional layer if there are not many non-linearities, or you might need more complexity (eg. adding convolutional and max pooling layers).
I made personally for teaching a workshop a very simple example of image classification, with 3 groups (trees, people, vehicles), after building a dataset collecting and selectioning images from the CIFAR-100 dataset.  You can see this ultra simple example here. However, it is in spanish. Let me translate the most important piece of code from it:
model = Sequential()

# CONVOLUTION. Final size: 29x29x32
model.add(Conv2D(   32,
                    (4, 4),
                    padding='same',
                    input_shape=data_x_train.shape[1:]
        ))

model.add(Activation('relu'))

# CONVOLUTION. Final size: 26x26x32
model.add(Conv2D(   32,
                    (4, 4),
                    padding='same',
                    input_shape=data_x_train.shape[1:]
        ))

model.add(Activation('relu'))

# MAX POOLING. Final size: 13x13x32
model.add(MaxPooling2D(pool_size=(2,2)))

# Dropout to prevent overfitting
model.add(Dropout(0.25))

# CONVOLUTION. Final size: 10x10x64
model.add(Conv2D(   64,
                    (4, 4),
                    padding='same',
                    input_shape=data_x_train.shape[1:]
        ))

model.add(Activation('relu'))

# MAX POOLING. Final size: 5x5x64
model.add(MaxPooling2D(pool_size=(2,2)))

# Dropout to prevent overfitting
model.add(Dropout(0.25))

# Fully connected layer. Neuron number: 1600
model.add(Flatten())

# Output layer: 3 neurons
model.add(Dense(3))
model.add(Activation('softmax'))

So the input for this model is an array of shape [num_rows, 32, 32]. 32x32 is the image size, and I assume 1 color channel. As I say, this is ultra simple. The output, some kind of probability for each of the 3 classes.
Using other aproaches
However, there are many other supervised learning models that you can use for that, most of them supported by scikit-learn, such as kNN, random forests decision trees, RBF SVM, or even just a kernelized multiclass logistic regression. See here.


text:
How to design an AI that discovers more complex concepts on its own?
How would I go about designing a (relatively) simple AI that discovers and invents random more complex concepts on its own?
For example, say I had a robot car. It doesn't know it's a car. It has several inputs and outputs, such as a light sensor and the drive motors. If it stays in the dark, it's score drops (bad), and if it moves into the light, it's score rises (good). It'd have to discover that it's motor outputs cause the light input to change (because it's moving closer or farther away from a light source), and that brighter light means higher score.
Of course, it'd be easier to design an AI that does specifically that, but I want its behaviour discovery system to be more generic, if that makes any sense. Like later on, it could discover a way to fight or cooperate with other robots to increase its score (maybe other robots destroy light sources when they drive over them, and they can be disabled by driving into them), but it'd have to discover this without initially knowing that another robot could possibly exist, how to identify one, what they do, and how to interact with one.
Also, I want it to be creative instead of following a 'do whatever is best to increase your score' rule. Like maybe one day it could decide that cooperating with other robots is another way to increase its score (it finds out what love is), but if it's unable to do that, it becomes depressed and stops trying to increase it's score and just sits there and dies. Or it could invent any other completely random and possibly useless behavior.
How hard would it be to make something like this, that essentially builds itself up from a very basic system, provided I give it lots of different kinds of inputs and outputs that it can discover how to use and apply to its own evolving behavior?

OpenAI just created something similar with reinforcement learning. https://openai.com/blog/emergent-tool-use/

This is not an answer. I couldn't comment, so here are some remarks about your question: This is a very broad question, and considered The Holy Grail for building artificially intelligent systems - meaning that some scientists have been dreaming about this since time immemorial.
Some homework is warranted from your side; you could have offered some of your solutions or maybe identified the multiple layers to your query, as they invoke concepts from many fields within the study of AI (or AGI to be general).
For example, the following layers are taken up around the big-words in the question, though rhetorically -

On evolution - what registers as evolution (is only genetic mutation evolution or should it involve some form of natural selection or seeking a niche for increasing the chances of survival or etc). How would evolution of software look like? Should it be able to modify its own code in the process of evolution?
On concept - what constitutes a concept, identifying from the environment a concept by calculating its relevance (w.r.t other concepts), coming with a process of selecting the concept for use in a given environment (natural or artificial).Referring to the example in the question, is the road a relevant concept or the trees and the sky and the bees pollinating some flowers along the roadside? What is a more fundamental concept - the trees or the bees, and how does one measure that?
On rewards - for us humans, rewards are maximization of survival of genetic machinery they carry (translating to reproductive success). what reward system should we come up that machines could use to increase their chances of survival in the physical world? what value should be put on a reward for moving the car on a straight line, or moving from darkness towards light? Shouldn't moving from darkness to light be more fundamental action (concept) and therefore rewarded higher than moving along the straight line? However, given that the car has learned to move from darkness to light, shouldn't the value of the reward be lowered so that it can learn other actions/concepts?

As can be seen from this brief detailing, there are layers within layers. Therefore, it is proper to establish now that there are no simple answer to this essay-requiring soul-searching time-eating heavy-research-wanting question. However appropriate directions can be given about work that is being done by some of the most prominent scientists of our times. The thing that one is looking for are known as Universal Problems Solvers, such as Gödel Machine (by Jürgen Schmidhuber) and AIXI - Artificial Intelligence (AI) based on Solomonoff's distribution ξ (by Markus Hutter).
Here is a quote lifted from Wikipedia page on AIXI that is pretty self-explanatory on how it maximizes the rewards over time.

AIXI is a reinforcement learning agent. It maximizes the expected
  total rewards received from the environment. Intuitively, it
  simultaneously considers every computable hypothesis (or environment).
  In each time step, it looks at every possible program and evaluates
  how many rewards that program generates depending on the next action
  taken. The promised rewards are then weighted by the subjective belief
  that this program constitutes the true environment. This belief is
  computed from the length of the program: longer programs are
  considered less likely, in line with Occam's razor. AIXI then selects
  the action that has the highest expected total reward in the weighted
  sum of all these programs.

Gödel Machine goes further - it allows the agent to modify its own code that allows it to maximise the rewards of its actions - which is to modify its own code that allows it to maximize the rewards of its actions - as so on. This is kind of a recursive definition, that simulates evolution (a rapid one) by choosing to evolve that code/state of the agent that converges towards a code that is superior to the code that the agent is currently running.
Here is a quote lifted from page on summary of Gödel machine. See that Hutter is also referenced (discovered of AIXI above).

Our Gödel machine will never get worse than its initial problem
  solving strategy, and has a chance of getting much better, provided
  the nature of the given problem allows for a provably useful rewrite
  of the initial strategy, or of the proof searcher. The Gödel machine
  may be viewed as a self-referential universal problem solver that can
  formally talk about itself, in particular about its performance. It
  may "step outside of itself" (Hofstadter, 1979) by rewriting its
  axioms and utility function or augmenting its hardware, provided this
  is provably useful. Its conceptual simplicity notwithstanding, the
  Gödel machine explicitly addresses the `Grand Problem of Artificial
  Intelligence' by optimally dealing with limited resources in general
  environments, and with the possibly huge (but constant) slowdowns
  buried by previous approaches (Hutter, 2001, 2002) in the widely used
  but sometimes misleading O()-notation of theoretical computer science.
The main limitation of the Gödel machine is that it cannot profit from
  self-improvements whose usefulness it cannot prove in time.

Reading the papers of the stated research would certainly be useful in finding answers to some of the layers in the question. There is also good work going on in the AGI community that somewhat aligns with the direction. Hope this helps.

This starts with "this is not an answer", then goes on to be an excellent answer that turns around the premises of the question. That is called a "frame challenge" and IMO a perfectly acceptable way to answer when the OP presents a top-level idea that needs a step back and context added.


text:
How does one characterize a neural network with threshold-based activation functions?
In an attempt at designing a neural network more closely modeled by the human brain, I wrote code before doing the reading. The neuron I have modeled operates on the following method.

Parameters: potential, threshold, activation.
[activation] = 0.0
Receive inputs, added to [potential].
If ([potential] >= [threshold])

[activation] = [potential]
[potential] = 0.0

Else

[potential] *= 0.5

In short, the neuron receives inputs, and decides if it "fires" if the threshold is met. If not, the input sum, or potential, decreases. Inputs are applied by adding their values to the input potentials of the input neurons, and connections multiply neuron activation values by weights before applying them to their destination potentials. The only difference between this an a spiking network is the activation model.
I am, however, beginning to learn that Spiking Neural Networks (SNNs), the actual biologically-inspired model, operate quite differently. Forgive me if my understanding is terribly flawed. I seem to have the understanding that signals in these networks are sharp sinusoidal wave-forms with between 100 and 300 "spikes" in a subdivision of "time," given for 1 "second." These signals are sampled for the "1 second" by the neuron, and processed by a differential equation that determines the activation state of the neuron. Synapses seem to function in the same manner -> multiplying the signal by a weight, but increasing or decreasing the period of the graph.
However, I wish to know what form of neuron activation model I created. I have been unable to find papers that describe a method like this.
EDIT. The "learnable" parameters of this model are [threshold] of the neuron and [weight] of the connections/synapses.

I just want to know what are the learnable parameters in your model and how they are learnt. Please modify your question with these details. One thing that I thought of is that you could make the multiplier to [potential] as a learnable parameter which can be learnt from the labelled data (assuming you are attempting a supervised learning problem with this model).

The model you describe is a kind of a leaky integrate-and-fire (LIF) neuron (see p. 7). It is leaky because the membrane potential decreases steadily in the absence of input. In contrast, in the simple integrate-and-fire (IF) model the membrane potential is retained indefinitely until the neuron spikes, at which point it is reset to 0. However, LIF neurons are usually modelled with exponential decay of the membrane potential, where you have a time constant $\tau$ and you compute the potential $P_{t}$ at time $t$ based on the potential $P_{t_{last}}$ at time when the last input arrived as
$P_{t} = P_{t_{last}} exp(- \frac{t - t_{last}}{\tau})$
This is the same formula as radioactive decay (see here for more details). The idea is that this model is inherently 'aware' of time, whereas the IF model (and your design above) do not factor in the timing of the spikes, so they act like a classical neural network activation. In any case, whether or not a neuron would fire does depend on the firing threshold, so I think that treating the threshold as a learnable parameter is justified - you just have to decide what rules to use for updating it.
Based on what you describe as your understanding of spiking neural networks, it seems that you have been reading about the Hodgkin-Huxley (HH) model (also in that paper I linked to). (Please correct me if I'm wrong.) You are correct in thinking that spikes in the brain are not infinitely narrow like a delta function but more like a very sharp sinusoidal signal, and the HH model faithfully reproduces that. However, the reason why the HH model is not actually used for simulations is that it is computationally very taxing. In practice, in most cases we do not actually care about the state of the neuron between inputs, as long as your model accurately describes the neuron state and what happens to it when an input arrives.
There are other models that approximate the HH model very closely but are much faster to simulate (like the Izhikevich model). However, the LIF model is very fast and sufficient in most cases.
Hope this helps!


text:
Robot Arm Deep Q Learning Actions
Hello I am new to reinforcement learning and robotics. So far I have an understanding of the concept on 2D world. You can make agent move one step in one direction. However, how do you define movement action of a robot arm? I am a bit lost over here. Any useful links or keywords would be very appreciated! :)

What do you mean by "define movement action of a robot arm"?

in 2D grid world agent can move up, left, right, down. Hence, has option of 4 moves. But the robot arm can move in any direction in any amount. So my question was how to make agent choose an option of move and how to define them.

It depends a lot on the hardware of your robot arm.
Assuming that your servos have encoder information, if you have access to servos that have limited control like "rotate left/rotate right" functionality, you can phrase the your action space to be ["move left", "stop", "move right"]. In this way you can implement a discrete action space with 3 actions per servo and have an agent learn to move the servos around the space.
If your servos are connected to each other in an elbow/shoulder configuration, you can have a 9 discrete action setup essentially making a box of cardinal directions:
Up+Left----------Up-------Up+Right
Left--------------Stop---------Right
Down+Left-----Down-----Down+Right
If you have 3 or more servos, you can still use the same idea of discrete actions but the number of discrete actions grows by a factor of 3 with each servo as your action space is now the cross product of all of the other servos.
Alternatively you can use a "multi-headed" agent where each head chooses actions for a certain servo but there are pros and cons for both depending on your usecase.

If you have more advanced servos like Dynamixels which have high quality encoders, you'll have access to more advanced controls schemes. For instance, Dynamixels allow you to give actions in encoder space, angle space, and even velocity space. For example, you could give the action of "go to encoder value of 500" or "go to 90 degrees" or "move .5 radians/second". All of these approaches are useful for certain tasks. For humans controlling the arm using a joystick, the velocity based is the most intuitive and, in my experience, the same is true for RL agents using continuous control.
If you are using continuous control, you should normalize all of your action spaces within your agent then "unnormalize (?)" them before giving the actions to your servos.
For instance if your servo velocity ranges from -3.5 rad/sec to +3.5 rad/sec, have your agent select actions in the range of [-1,1] then multiply by 3.5 to get the velocity.

In either case, one thing that should be noted is that you give your robotic arm enough time to actually perform the action that the agent selected. If not you will see your robot "jitter" back and forth quickly as your agent selects actions randomly. This is bad for a few reasons but most importantly because it might break your servos. To overcome this issue, give each action a little more time to be "actualized" by your robot. This can either come in the form of adding a delay in your code while your servos do "the thing" or by using an "iterations since last action update" counter to get a new action from the agent after a certain number of iterations. Not only is this better for your hardware, this also leads to better exploration of your state space as your agent can move through the state space encountering similar states more frequently.
A last thing to be aware of is to set hard coded limits on the servos so that your agent doesn't kill your servos by banging the arm against a table or something. Finding these safe bounds isn't always easy as multiple jointed configurations can have multiple ways of hitting the bounds and it might require a forward dynamics model to find these limits.
If you have only 2 servos, it should be pretty easy to find though :).

Thank you very much for such detailed answer! We have a 6 axis robot at uni. I can tell robot to go to point where coordinates are x, y, z. Hence, I can define boundary box easily. Should I define movements of one unit in x, y, z for each possible direction?

That is definitely a possibility. It depends on the application. A 6 axis arm with coordinate controls implies that there is a lowerlevel system of forward dynamics and when given a command of "go to xyz", there are infinitely many rotations of the joints that can perform that movement and the forward dynamics model finds 1 based on some metric (distance). If the agent doesnt need to know about objects that are in its reach then I would say this is fine. If, however, the agent would need to know how to avoid hitting certain objects, then only using xyz isnt enough and youd need servo control.

Jaden, thank you very much. Now I understand it much better. To start, collision control is not a problem but aim is to have it in action. I have an idea for that as well. Ačiū labai!!!