text:
Checkers AI game engines
I have coded an AI checkers game but would like to see how good it is. Some people have informed me to use the Chinook AI opensource code. But I am having trouble trying to integrate that software into my AI code. How do I integrate another game engine in checkers with the AI I have coded?

Please explain what is preventing you integrating Chinook - e.g. do you get errors, is there a clash of libraries or language support on your machine? Also, it is worth mentioning your project's coding language. Your own code and whatever you integrate with will need to share a game board and be able to inform each other of when they have moved/whether they resign etc.

I'm having trouble wrapping my head around how to integrate it into my code

Is it just a matter of copying and pasting it?

I suspect that is unlikely. Are the two projects written in the same language? Do they use the same board representation? Do you have a reference to the Chinook AI code and documentation for its API?

To put it simply. I have to say no to all of those questions. Can you help me?

Is there a site whereby it offers a variety of languages for me to choose from so I can just download it?

No I think it is very unlikely that you will find a conversion to your language if the original does not match. I don't think you have an AI problem here, but are limited by your knowledge/experience of working with third party code or systems. What language is your project in?

I coded it In C

OK, so C doesn't come with much help, but it is fast and can connect low-level. Your ideal integration will be with another checkers library written in C, which has at least some documentation. A quick search turned up this: http://www.fierz.ch/checkerboard.php . . . which is a shared checkers "environment" that runs in Windows and that multiple different player engines can share. It may be your best bet - it is unlikely that anyone on AI site can answer questions about integrating it though.

Final thing that may help - CheckerBoard is written so that you can implement your AI as a Windows DLL and copy it into an Engines folder - i.e. all the integration work done for you, including a UI for human-vs-AI play, provided you follow the API that it expects of you in your DLL: http://www.fierz.ch/cbdeveloper.php

@NeilSlater  That is mighty helpful of you Neil, thank you


text:
Variable sized input-Multi Label Classification with Neural Network
I have a data input vector ( No Image classification) which size varys from 2 to 7 entrys. Every one of them belongs to a class Out of 7. So I have a variable Input size and a variable Output size. How can I deal with the variable Input sizes? I know Zero padding is a option but maybe there are better ways?
Seconds: Is multi Label classification possible in one Network? What I mean: The first entry has to bei classified in one of the seven classes, the second entry... and so on.
I am also open to other classification techniques, If there is a better one that suits the problem.
Best regards,
Gesetzt

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
To what level of abstraction must fully automated vehicles build their driving model before safety can be maximized?
There are several levels of abstraction involved in piloting and driving.

Signals representing the state of the vehicle and its environment originating from multiple transducers1
Latched sample vectors/matrices
Boundary events (locations, spectral features, movement, appearance and disappearance of edges, lines, and sounds)
Objects
Object movements
Object types (runways, roads, aircraft, birds, cars, people, pets, screeches, horns, bells, blinking lights, gates, signals, clouds, bridges, trains, buses, towers, antennas, buildings, curbs)
Trajectory probabilities based on object movements and types
Behaviors based on all the above hints
Intentions based on behavior sequences and specific object recognition
Collision risk detection

Moving from interpretation to control execution ...

Preemptive collision avoidance reaction
Horn sounding
Plan adjustment
Alignment of plan to state
Trajectory control
Skid avoidance
Skid avoidance reaction
Steering, breaking, and signalling
Notifications to passengers

What, if any, levels of higher abstraction can be sacrificed?  Humans, if they are excellent pilots or drivers, can use all of these levels to improve pedestrian and passenger safety and minimize expense in time and money.
Footnotes
[1] Optical detectors, microphones, strain gauge bridges, temperature and pressure gauges, triangulation reply signals, voltmeters, position encoders, key depression switches, flow detectors, altimeters, radar transducers, tachometers, accelerometers

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Does overfitting imply an upper bound on model size/complexity?
Suppose that I have a model M that overfits a large dataset S such that the test error is 30%. Does that mean that there will always exist a model that is smaller and less complex than M that will have a test error less than 30% on S (and does not overfit S).

If you want to discuss theoretical bounds, you should probably define a few more terms. Such as relative size of test and train sets, what counts as over-fitting in numerical terms (100% accuracy on training data - or something else?) If overfitting is 100%, the training and test data are drawn from the same distribution and test set is smaller, then you should be able to construct a smaller model which 100% fits the test data - such a model probably exists, but it would be very unlikely to be reachable by training on the training data. That seems trivial - is it what you are looking for?

@NeilSlater for now I only wanted an intuitive explanation. But it would be interesting to have some sort of theoretical bound. Could you point me to some resources that discuss these theoretical bounds?

@DuttaA for now I am using the term complexity quite loosely. Here I mean the size of the model/number of parameters it has.


text:
Can we make Object Detection as human eyes+brain do?
I am so much curious about how do we see(with eyes ofc) and detect things and their location so quick. Is the reason that we have huge gigantic network in our brain and we are trained since birth to till now and still training.
basically I am saying , are we trained on more data and huge network? is that the reason?
or what if there's a pattern for about how do we see and detect object.
please help me out, maybe my thinking is in wrong direction.
what I wanna achieve is an AI to detect object in picture in human ways.thanks.

Thanks for posting.Add some references also for effective feedback.

Object detection can conceivably imitate what the human visual system does.  Research along these lines began in the 1980s in multiple laboratories and was termed Computer Vision.

How do we see ... and detect things and their location so quick[?]  Is the reason that we have huge gigantic network in our brain and we are trained since birth to till now and still training[?]

Neurological visual systems continue to train as long as the eyes and neurons continue to function.  A person may, long after doing well in sports, learn how to catch glass objects that fall off a counter using peripheral vision, using the same neural pathways as were used for visually coordinated movement in sports but.  They are newly trained to the new cognitive intention of saving glassware and clean-up time.
The mapping of signals from the rods and cones to the visual cortex has been studied and some generalities have influenced artificial vision research and development.  The convolution kernels used in CNNs are partly derived from biological research.  In summary, the way mammals and other organisms with the equivalents of retinas see is via a set of transformations and pattern matching circuits.  The sequence of information types through biological visual networks can be summarized.

Light as a function of compound angle and time.
Edges in both space and time
Elements of objects and actions — These are real world features, not in the machine learning sense but in the natural language use of the term features.
Objects and actions

The development of object and action recognition in computers follows this same basic sequence.  The goal stated in the question is this.

Achieve an AI to detect object in picture in human ways

Something like this has been the goal of many government, academic, and corporate laboratories, however the ability to recognize objects in still pictures is not as valuable as detecting the trajectories and other expressions of objects (such as changes in facial expressions) in real time.
Automated aiming, piloting, walking, and driving have been computer vision objectives since the first control systems were developed for anti-aircraft defense systems.  Considerable effort is being invested into how steering, breaking, and signaling can be automated in road vehicles.  A large array of household and industrial devices are being developed that rely on mapping the environment and navigating a robotic device through the map.
Stationary object recognition work is largely surrounding either categorizing images or drawing vector graphics from raster images, but these functions have limited application.  As mobile devices develop further and images on the web give way to videos on the web, a very strong and uninterrupted trend, the temporal dimension will continue to gain importance in the field computer vision.
None of these objectives are novel.  They are all based on animal capabilities, only some of which are uniquely human, and have been the subject of research since before the advent of digital computers.  However the feasibility of many designs has improved, largely due to a significant reduction in costs for fixed computing resources.  The vision system hardware of today costs roughly one thousandth of its 1990 cost.

If you have studied about Convolution Neural Network, you probably know how present day object detection algorithms works. Basically, an Artificial Neural Network tries to mimic the way our brain might work. So, CNN is probably the way our brain works to perform object detection, trying to recognize each small parts at each stage, that slowly grows to compile into the larger object to be recognised.  However since, we are not restricted to data from just 2D plane, so the processing might be more complicated. We all learn things after being trained to do so. You could never have recognized a particular fruit if you have never seen it. The way we store the information, we gain, is different, that helps us to remember/recognise objects after only a few look. Until, we have more research that provides more details into the working of our brain, we are all good with Artificial Neural Network. Honestly, we are still far away from achieving general artificial intelligence. Now, if you want to implement an object detection algorithm, CNN is your only way.

Thanks for answering,can you also add some references to make it great.


text:
Data / model preparation for spatio-temporal deep-learning analysis for traffic congestion events detection
I am preparing the Bus movement dataset for deep learning (ANN/CNN/RNN) analysis for congestion events detection. This is an extension to my original question, which can be located at 'Deep learning model training and processing requirement for Traffic data' for the general approach on this topic, and this question is for preparing the dataset and need your kind advice on it. In simple words, I would like to know the state of congestion for a bus route at a specific point in time (year).
Here are my entities:

Routes
Bus_Scheduled_Routes
Bus_Route_Stops
Bus_Trips (operational_date, Vehicle_id, Trip_id, Vehicle_Position_Update, Trip_stop_id, passenger_loaded, velocity, direction, scheduled_arrival_time, actual_arrival_time)
Events (human and non-human induced)
Points of Interests (POIs)

If I have these entities based data and I create a view that gives me a time reference based view comprising of week(52), day(7), Vehicle_id, Trip_id,  Stop/Position_update_interval, speed, acceleration, velocity, scheduled_arrival_time, actual_arrival_time. Will this view be recommended to start training the model?
Secondly, how can I integrate the human / non-human induced events and Points of Interests (POIs) data into this view so my model can predict better results? To generalize the model data will be 'time segment / trips time (seasons), location component (Bus Routes and Stops), Arrival time / trip completion time'. I am thinking to add an attribute for human/non-human induced events as type tying with the 'time segment' and adding the POIs as type and vicinity to the stop points. What are your recommendation about it? Thanks in advance for your help.


text:
Why did fuzzy logic fall out of fashion?
Fuzzy logic seemed like an active area of research in machine learning and data mining back when I was in grad school (early 2000s). Fuzzy inference systems, fuzzy c-means, fuzzy versions of the various neural network and support vector machine architectures were all being taught in grad courses and discussed in conferences.
Since I've started paying attention to ML again (~2013), Fuzzy Logic seems to have dropped off the map completely and its absence from the current ML landscape is conspicuous given all the AI hype.
Was this a case of a topic simply falling out of fashion, or was there a specific limitation of fuzzy logic and fuzzy inference that led to the topic being abandoned by researchers?


text:
What were the criticisms of the BACON algorithm?
The BACON algorithm, introduced by Pat Langley and Herbert Simon etc. were meant to automate scientific discovery -- producing causal explanation to variations in given data.
It was found, in particular, to have been able to "discover" the laws of planetary motion, but there were criticism concerning whether or not its achievement could be consider true scientific discovery.
The one I remember was that--the data given to the algorithm are pre-processed by human operators so as to include mostly relevant factors of the world. For example, the distances of planets from the sun was given, showing that the human operators implicitly understood the importance of this variable for the laws of planetary motion.
I was wondering if there were other significant criticism of the algorithm--either in general as a automaton of discovery or concerning its particular feat with planetary motion.

Please either state the BACON algorithm or give a literature reference

reference   AI program "BACON" 1987 was written by Herbert A Simon PhD Student Gary L Bradshaw.
https://www.ijcai.org/Proceedings/81-1/Papers/025.pdf
criticism: heard in 1980's Isaac Newton did not invent the inverse square law of gravity, he invented mass.  Others in 17nth century may have mentioned inverse square law as well.  History of science shows understanding mass does not depend on non-zero gravity shows deeper view of newton's physics.


text:
neural network deconvolution filters
I understand the concept of convolution.
Let's say that my input dimension is 3 x 10 x 10
And if I say that I will have 20 activation maps and a filter size of 5, I will end up with 20 different filters for my layer, each with the dimension of (3 x 5 x 5)
My output will therefor be (20 x ? x ?). I Put a "?" there, because it obviously depends on the filter stride etc.

Now I wanted to implement deconvolution but I am stuck at the following point:
For the following questions, let's assume that the input size for the deconvolution is (5 x 8 x 8),

If we think about a filter in 3 dimensions. Can I choose any depth for the filter?
How would the effect of the amount of filters (amount of activation maps) work with deconvolution? Do I only have one filter?
How does the input depth (5) come into play. Would the output depth be equal to         (filter depth) * (input depth) ?

I am trying to find the symmetry to forward convolution but I do not understand how to use the amount of activation maps in deconvolution.
I am very thankful for any help.

Does https://datascience.stackexchange.com/q/6107/8820 help you?


text:
3D environment for RL research in Academia
I'm doing my thesis on Reinforcement Learning. My focus on Partially Observable Environments like 3D Games. I want to choose a 3D platform for testing and doing research.
I know some of them. DeepMind Lab and OpenAi Universe. But my question is that which of these environments is good for me? Is there any environment for this purpose that is benchmark and reliable?
I want a platform that accepted in Academia and reliable. For example DeepMind is not a standard or Open Source friendly, Is it rational to use their platform for research in academia?
What i have to do?

For example DeepMind [Lab] is not a standard or Open Source friendly

I'm not sure where you got that info from... as far as I'm aware, DeepMind Lab is definitely used in various publications (maybe primarily publications from DeepMind, but still). Considering the github repo has the GNU GPL 2 license, it also seems Open Source-friendly to me.
Another framework of which I'm sure that it would widely be considered suitable within academia would be the Unity ML-Agents Toolkit, which uses the Unity game engine.
I suppose you could also consider using ViZDoom, which is also used in various publications, but (as far as I'm aware) it only supports one specific game (Doom).
I do not have enough experience with using any of the above personally to be able to recommend one of them over the others... but they would all seem suitable to me.


text:
Are there any pretrained models for human recognition from all angles?
I need to be able to detect and track humans from all angles, especially above.
There are, obviously, quite a few well-studied models for human detection and tracking, usually as part of general-purpose object detection, but I haven't been able to find any information that explicitly works for tracking humans from above.

When you say "above," do you mean humans approaching from the top down? In this case you can use existing human detection in libraries and analyze the size of the object. The closer, the smaller the distance and the larger the object.

@GuilhermeIA Yes, top down, as in only top of head and shoulders discernable. Do existing human detection libraries account for this?

"from the top" like satellite images or like surveillance cameras?

@MartinThoma From a ceiling-mounted camera anywhere from 10-35 feet above the floor. But ideally it could be mounted anywhere at any angle, so ideally detecting humans from all possible angles

There were no pre-trained models for human that were taken from all angles (which is a somewhat extreme requirement) or specifically from above.  This condition may change due to the high value of such to the surveillance industry.
There are already several projects for which there is human specific pattern recognition and there have emerged some data sets to support this more specific field of objectives, some of which are based on moving rather than stationary images.  This is for two reasons.

Multiple frames has the potential (should computing resource utilization be overcome) to produce higher accuracy and reliability in human recognition partly because of the identification of joints in movement and partly because there is simply more information available on each object that could be a human.
Trajectory of pedestrians is important for both law enforcement and security and for avoiding pedestrians in vehicle automation.

These are some of the projects and download sites that are related to the goal of the question, but unfortunately do not seem to provide data sets of humans from above or pre-trained networks from that visual perspective.

https://software.intel.com/en-us/openvino-toolkit/documentation/pretrained-models
https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/pretrained-cascade-image-classification
https://github.com/matterport/Mask_RCNN
https://drive.google.com/drive/folders/1zvl89AgFAApbH0At-gMuZSeQB_LpNP-M
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md
https://modelzoo.co/model/human-pose-estimation-with-tensorflow

Look at all the Haar boosted classifiers that are available with OpenCV and the dedicated class CascadeClassifier to use it. Here are a list of what the classifiers have locally...

You can try frontalface. The shoulders I searched and did not find much. You may have to create a shoulder dataset and train to detect it.
https://stackoverflow.com/a/5727231/5336017

Wouldn't "frontalface" be face-on, or am I missing something? I need literally the tops of heads and shoulders as distinct from other objects.

As I said, these are the ready-made libraries that exist. In your case, as something is perhaps specific and you have not yet on the internet, you will need to create your own head and shoulder top dataset. From a researched on tensorflow-image-classifier. You will need a huge base of images and train them.

Sure, I get that. I would just be surprised that there wouldn't already be a dataset for this purpose used in surveillance, drones, etc..


text:
Can AutoKeras be used for neural networks of PyTorch
I use PyTorch, bauces AllenNLP is built on it and good libraries are for it. But can AutoKeras be used for PyTorch based ML pipelines, or am I required to switch to Keras? Google is quite silent when asked for tutorials for this combination. Maybe PyTorch is lacking automated ML framework?

Welcome to Ai! We discourage posts that simply state a problem out of context, and expect the community to solve it. Assuming you tried to solve it yourself and got stuck, it may be helpful if you wrote your thoughts and what you could not figure out. It will definitely draw more answers to your post. Until then, the question will be voted to be closed / downvoted. You may also want to check out our community guidelines/hints, or use the search engine of this site to find similar questions that were already answered. So lets do something timely scientific.

"Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano."
But I found a useful tool which collects different deep learning model convertors.
Hope this will be useful for you :D


text:
How to use Genetic Algorithm for varying lengths of solutions
Until now, I always thought that Genetic Algorithm can be used for problems of which the solution space can be encoded (modeled) as a chromosome of a specific length. However, some people claim that they used GA for this game and this game. They are basically games in which we control an agent on a 2-dimensional area.
Obviously, the length of the genome sequence depends on how fast the game is finished. So, how is GA used for such games?
If you think GA is not the most suitable method for this kind of problems can you explain why and give better alternatives?

When it comes to alternative solutions, how about Evolutionary Algorithms (EA)?
Seems to me EAs are more flexible then GAs and can find their use in more applications than GAs.

Evolutionary Algorithm is a class of machine learning comprising of at least 3 sub-classes namely: Genetic Algorithms (GA) [Holland  et al](Including GPs), Evolutionary Programs (EP) [Fogel  et al], Evolutionary Strategies (ES)(Rechenberg et al). Back was the first to categorise these algorithms in 1997. See his book https://dl.acm.org/citation.cfm?id=548530

Before I answer your question, it is important to frame what a GA is, so please allow me to cover some history.
Friedman and Fraser performed some of the earliest evolutionary computation experiments. Fraser presented the case of diploid organisms represented by binary strings of a given length. Each bit in the string represented a gene. Fraser proposed the process of single-point crossover for binary string reproduction.
American computer scientist John Henry Holland and his colleagues at the University of Michigan formalised the evolutionary algorithms originally implemented by Bremermann, Fraser, Friedman and Friedberg.
Holland's formal algorithm is known as a Genetic Algorithm (GA). A GA has a fixed length chromosome structure. Traditionally it was binary based. However due to hamming distances of various bit strings a GA using a binary encoded chromosome string can get stuck at local minima. Holland experimented with integer based GAs too.
Getting back to your question, theoretically and in its strictest sense, a Genetic Algorithm has a fixed length chromosome vector. The primary reason for this is that each gene represents something. If it was variable then the meaning of the gene must be encoded into the chromosome, which makes evolution more difficult. However, there are other chromosome structures that are variable length. The chromosome structure itself implies meaning.
Koza experimented with an alternative version of a GA called a genetic programming (GP). A GP replaces the fixed length vector structure used in a genetic algorithm with a variable length, non-linear, hierarchical abstract data structure, known as a tree.
The tree consists of branch nodes and leaf nodes. Leaf nodes have no children and implement a number or constant from a terminal set. A leaf node is also known as a terminal node. A branch node is connected to other branch nodes or leaf nodes and implement an operator from a functional set. A branch node is also known as a non-terminal node.
The tree structure naturally accommodates a game rule. A tree is not restricted to a fixed length. A tree is a natural structure to represent hierarchical decisions rules, and a tree can store both binary, logical, and numeric functions within its nodes. Logical functions allow the GP to evolve conditional execution of game play functions not easily implemented using a fixed length vector. Tree does need a traversal path and grammar (syntax and semantics).
In short to answer your questions:

A GA (fixed length) is generally not used for games, but rather a GP (variable with decisions).
A GA is not the most suitable method for this kind of problem as it is fixed
length, and each gene corresponds to a rule or value.

I would implement a GP, where the operators of mutation and crossover are adapted for a GP. I would also use the game itself as a fitness function, where a pool of individuals either using a single population, or co-evolution evolve players that gradually increase their score.
The paragraphs above were taken from Nicholls' MSc thesis. His thesis was on stock market trading rule generation which is similar to game rule generation. His thesis is available here (Only available after April 2019).


text:
Using features extracted from a CNN as convolutional filter
I'm a bit confused about this. Assume I have a CNN network with two branches:

Top
Bottom

The top branch outputs a feature vector of shape 1x1x1x10 (batch, h, w, c)
The bottom branch outputs a feature vector of shape (1, 10, 10, 10).
I want to use the top feature vector as a convolutional filter, and convolve it with the bottom feature vector. I can do this in pytorch with the "functional.Conv2D" function, the problem is, I don't know how back-prop works in this case (will it be unstable?) since the output feature is a now a parameter as well, do I need to stop gradients or do something else in this case to backprop correctly?


text:
How to add external training in chatterbot?
I created a very simple bot to learn how to use chatterbot. This library already comes with a training, but I wanted extra training with an import of a corpus in Portuguese that I found in github.
from chatterbot import ChatBot

bot = Futaba(
"Terminal",
storage_adapter="chatterbot.storage.SQLStorageAdapter",
logic_adapters=[
"chatterbot.logic.MathematicalEvaluation",
"chatterbot.logic.TimeLogicAdapter",
"chatterbot.logic.BestMatch"
],

input_adapter="chatterbot.input.TerminalAdapter",
output_adapter="chatterbot.output.TerminalAdapter",
database_uri="../database.db"
)

print("Type something to begin...")

while True:
    try:
        bot_input = bot.get_response(None)
    except (KeyboardInterrupt, EOFError, SystemExit):
        break

That's all I have.
How can I import this corpus into my chatbot?

I'm voting to close this question as off-topic because it's not really about AI, but about an implementation detail with a particular software package, which happens to be on an AI-related topic.

Wow, but then I do not understand. If I ask questions about developing a chatbot in a programming forum the question is closed because it refers to AI. If I ask in an AI forum close for not being on IA.

This seems to me to be about a specific piece of software. To answer that question you need no expertise in AI, so that's my reasoning. I have worked with/in AI for about 20+ years, and I have no clue how to answer your question.  I don't even know what library you are talking about, so I cannot look up the documentation for it.

Chatterbot for python

Programming issues are off-topic here. Please, read https://ai.stackexchange.com/help/on-topic for more details and to understand which questions are on-topic here. If you have a programming issue, the best place to ask your question is Stack Overflow.

I'm not sure if this is the right one, but according to the documentation you need to create a trainer for it:
from chatterbot.trainers import ChatterBotCorpusTrainer

# Create a new trainer for the chatbot
trainer = ChatterBotCorpusTrainer(chatbot)

# Train based on the english corpus
trainer.train("chatterbot.corpus.english")

Instead of "chatterbot.corpus.english" you obviously choose your own training data.

Basically, I just need to put my corpus, put in the chatbot folder and import on that page?

That's how I understand it. Not sure what format the corpus needs to be in -- you might have to look at the other example data sets.


text:
Is there any deep learning object detection algorithms that can work without bounding boxes annotated data?
For example Haar Cascade can be trained using only positive and negative examples, you don't need any bounding box annotations. But it not a deep learning approach.
Another example can be the most straight forward Image recognition model + sliding window. But it is very slow


text:
How Hopfield neural networks are connected with "industrial" neural networks used in machine learning?
I am trying to read https://arxiv.org/abs/1701.01727 about generalisation of Hopfield neural networks and I like the clear ideas that physics and Hamiltoanian framework can be used for modeling such networks and for deducing lot of properties. My question is - how Hopfield networks are connected to the standard networks?
I see at least 3 differences:

Hopfield neurons has binary values on/off (+1, -1) but machine learning neurons have real (are at least approximately (within machine limits) real) values.
threshold function is the only activation function used in Hopfield neurons, machine learning has lot of  more complex, real-valued functions
Connectivity patterns in machine learning networks (LSTM, GRU cells) are far more richer.

So - maybe Hopfield neural networks can be generalized up to the level of machine learning networks and at the same time preserving the use of Hamiltonian framework. Is it possible, is there any work in this direction?


text:
RNN and LSTM for discovering time lag
Is there a good reference / tutorial for using RNN/LSTM to determine lag interval for 2 time series? E.g. I have {x_n}, {y_n} and I want to figure out by how much does {x_n} typically lags behind {y_n}?


text:
Use of backpropagation for weight updates in a combination of 2 neural networks
Every neural network updates its weights through back-propagation.
How is back-propagation used for updating weights in a combination of 2 or more neural networks (e.g.:CNN-LSTM, GAN-CNN, etc.).
For instance a CNN-LSTM model is a CNN model stacked on top of an LSTM model. When CNN model is stacked on top of an LSTM model, do we consider hidden layer of both model or hidden layer of outer model(LSTM)?

CNN-LSTM is just one continous model so backprop works like usual. GANs-CNN has the same backprop schema as normal GANs.


text:
Are artificial intelligence learnings or trainings transferable from one agent to the other?
One disadvantage or weakness of Artificial Intelligence today the slow nature of learning or training success. For instance, an AI agent might require a 100,000 samples or more to reach an appreciable level of performance with a specific task. But this is unlike humans who are able to learn very quickly with a minimum number of samples. Humans are also able to teach one another, or in other words, transfer knowledge acquired.
My question is this: are Artificial Intelligence learnings or trainings transferable from one agent to the other? If yes, how? If no, why?

The simplest case is copying the software. That is instant duplication of the learnings. In a similar, but less trivial way you can adjust pretrained neural network classifiers to new datasets by simply re-initializing the last layer.
It gets more interesting when you have multiple agents and you want to combine their knowledge. This can be done with many techniques; the simplest is averaging the weights of identical neural network architectures. One example of this combined learning happened is Google Has a Room Full of Robot Arms Learning Hand-Eye Coordination. The relevant paper is Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection.
The most complicated case, where you want to transfer just the gist and prevent forgetting is -- up to my knowledge, and I'm relatively certain it's outdated in that area -- unsolved. The problem is post-hoc explanations. Humans decide to do something, observe the outcome and then explain why they did something / why it was good to do it. So we automatically generate hypotheses about the world. And sometimes we are able to formulate them in a way that others can understand them. That is not possible with current learning machines. We don't know yet how to abstract automatically from arbitrary problems.

I am confident transfer of learnings from one agent to the other will be a surer route to achieving artificial general intelligence.


text:
Overfitted model performs better in test set
There are two models for the same task:
model_1: 98% accuracy on training set, 54% accuracy on test set.
model_2: 48% accuracy on training set, 47% accuracy on test set.
From the statistics above we can say that model_1 overfits training set.
Q1: Can we say that model_2 underfits?
Q2: Why model_1 is bad choice if it performs better than model_2 on test set?


text:
Image prediction model when data-set classes have visual similarity
Lets say we have a data-set of all cats and we have to identify the cat breed based on given test image. As, the two different cat breeds have visual similarity can we use existing networks (VGG, ImageNet, GoogleNet) to solve this problem?

Should faceNet be applied here? As, the problem is similar to face detection where face characteristics of two different people are same yet it can correctly recognize a person.
What if with visual similarity in data-set we have only few example of each class? Like for a problem (random) we have good amount of data but for each class we have only few examples.

Is there any model that can be applied here?

You could use transfer learning. Find some state of the art neural net (something newer than VGG) already trained for image classification. Then remove the last layer and add a new one, training it while keeping the remaining old weights fixed. A simple softmax at the end would work.This is the simplest approach.
Because you're mostly only training the last layer, you won't need as much data.
I think FaceNet will perform poorly because it uses different features and you would need to retrain all its layers.
If you have too little data for some classes, maybe you should use an autoencoder as a features extractor. Then you could classify an image by comparing its feature vector to those for each breed, for example with cosine similarity.


text:
Why are all the actions converging to the same index?
I am using PPO with an LSTM agent. My agent is performing 10 actions for each episode, one action is corresponding to one LSTM timestep and the action space is discrete. I have only one reward per episode which I can compute after the last action of the episode.
For each timestep (~ action) my agent has 20 choices. The following plot shows the reward (y-axis) versus the current episode (x-axis).
The plot shows a decreasing reward because I want to mimize this reward so I use: minus of the true reward.
At the beginning of the process, the agent seems to learn very well and the reward is decreasing but then it's converging to a value which is not the best. When I look at the results of my experiment it appears that the index of all actions are same (for example the agent is always choosing the second value of my discrete action space).
Does anyone have an idea about what is happening here ?

This is a common problem, it would be nice if someone could weigh in with some information on this.


text:
Training by one batch of examples, what does it mean
Say I have a batch of examples, each examples represent a state:
[0.1, 0.2, 0.5] #1st example
[0.4, 0.0, 0.3] #2nd example
..........
[0.1, 0.1, 0.1] #16th example

I feed through the NN, and then the NN predict the following class:
[move up]   #1st example
[move down] #2nd example
........
[move left] #16th example

And then I take the square loss (which calculated to be 0.1 after taking average over 16 examples), and do backward propagation.
So, can I assume that each of these examples will assign (or contribute) to a 0.1 loss?


text:
How is AlphaZero different from Stockfish or Rybka?
I don't know much about AI or chess engines, but what is the fundamental difference between AlphaZero and Stockfish or Rybka?

Stockfish is free software (source while AlphaGo can only be used by employees at Deepmind. This is important, because it means all answers to this question can only rely on the AlphaGo paper. Some decisions during the matches (e.g. giving limited time per move and not per game; giving different computational power) lead to unclarity how good alpha go really is.
Stockfish  uses Alpha-beta search with a hardcoded metric to evaluate how good a position is.
In contrast, AlphaGo uses Monte Carlo tree search and convolutional neural networks. They are trained in an reinforcement learning setting.
AlphaGo is promoted as being almost directly applicable to different settings (Go, Chess), while stockfish is only applied to Chess. The techniques of stockfish can also be applied to Go, but the problem is the board evaluation. For Chess it's easy compared to Go.
References

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm, December 2017.
Mastering the Game of Go without Human Knowledge is about Alpha Go Zero

Your references to AlphaGo should be to AlphaZero, although AlphaZero is derived from work on AlphaGo, and AlphaGo Zero.


text:
What does it essentially mean if the neural network has convex error surface?
Suppose if I am building a Linear Regression model with one fully connected layer and a sigmoid with minimizing mean squared error as objective. Why would the error surface be convex?
Does finding the optimal parameters for this network mean we cannot do better than this? Under what assumptions, this solution would be optimal? If we relax the linearity assumption and add some non-linearity to the network can we do better than this? Why so?


text:
Does IBM Cloud Private for Data run on public clouds like AWS or Azure?
I just started using IBM Cloud Private for Data this week and I wasn't sure if I can use other public clouds to connect with my ICP for data account. So I spoke with an IBM representative and I wanted to share their responses...

Even though the community is partially sponsored by IBM, but try to go through the community guidelines for effective feedback.

Yes, ICP for Data can run on any public cloud infrastructure as long as it supports Docker containers and Kubernetes.
Already tested: Amazon AWS, IBM Cloud, Azure, and Red Hat OpenStack


text:
How a game playing agent could identify potential objects and proximity?
Most implementations I'm seeing for playing games like Atari (usually similar to DeepMind's work using DQN) have 4 graphical frames of input fed into 3 convolutional layers which are then fed into a single fully connected layer. The explanation of no pooling layer is due to positioning of features/objects being very critical to most games.
My concern with this is that it may be weighing visual features based on position without regard for feature->feature proximity. By this, I mean to question if learning to avoid bullets in the bottom left of the screen is knowledge also used in the bottom right of the screen in a game like Space Invaders.
So, question 1: Is my concern with only using 3 conv layers into a fc layer legitimate regarding spatially localized learning?
Question 2: If my concern is legitimate, how might the network be modified to still treat feature position as significant, but to also take note of feature to feature proximity?
(I'm still quite the novice if that isn't extremely obvious, so if my questions aren't completely ridiculous on their own, please try to keep responses relatively high level if you would.)


text:
Can I detect unique people in a video?
I am having a video feed with multiple faces in it. I need to detect each face and the gender as well and assign the gender against each person. I am not sure how to uniquely identify a face as Face1 and Face2 etc. I do not need to know their names, just need to track a person in the entire video. I thought of tracking methods but people are constantly moving and changing so a box location can be occupied by another person after some frames.
I am interested in a way where I can assign an id to a face but I am not sure how to do it. I can use Facial Recognition Based embedding on each face and track that. But that seems to be an overkill for the job. Is there any other method available or Facial Recognition/Embedding is the only method to uniquely identify people in a video?

Can I detect unique people in a video?

Yes. However, for some applications commercial systems are not good enough: German source
The concepts you are looking for are:

Optical Flow: for two frames, define for each point how it moved
Object tracking: given a single object in one frame, find it in the next. Optical flow helps with that.
Object detection: look at YOLO / SSD. Small introduction here.

Thanks Martin, is object tracking robust if the object in interest are coming in and out of the frames. Optical flow will just track it if it is continuously in the stream.

Correct. But Optical flow gives you a good idea where to look for new objects.

Makes sense, Let me try that. Thanks a lot Martin.

You can detect faces in video frames by employing the standard method of Viola-Jones detector (though there are sophisticated ones available now). You can get the paper from here. All the standard image processing libraries will have it implemented. You have to use the necessary API of that library.
This will give a bounding box around each of the detected face (it may miss out a few faces which are at different poses or illumination). You can now encode the detected face, inside the bounding box, into a vector using the features given by face2vec. In the face2vec model remove the classification part alone. This will give you a feature vector of size 905 dimension.
Now given two face detections, you can find the euclidean distance between their feature vectors. If it is below a threshold say $T$ then, you have detected same face, else you have detected two different faces.

This is helpful Varsh, I will try face2vec. I am already trying encoding and graph clustering using euclidian distances. I get 128 dimension vector today. But I want to avoid Deep Learning Approaches as they are computation heavy and I want to deploy the system in a small device.
Is there any other possible representation of the face which can help me cluster them.

You can use PCA to perform dimensionality reduction on raw pixels and use the reduced size vector as features. In this case you have the control over the number of dimensions thereby reducing computations for your small device.


text:
How can I evaluate the performance of a system that generates text?
I am preparing to perform research comparing the performance of two different systems that probabilistically generate the next word of an input sentence.
For example, given the word 'the', a system might output 'car', or any other word. Given the input 'the round yellow', a system might output 'sun', or it might output something that doesn't make sense.
My question is, how can I quantitatively evaluate the performance of the two different systems performing this task? Of course if I tested each system manually I could qualitatively determine how often each system responded in a way that makes sense, and compare how often each system responds correctly, but I'd really like a meaningful quantitative method of evaluation that I could preferably automate.
Precision and recall don't seem like they would work here, seeing as for each given input there are many potentially acceptable outputs. Any suggestions?

This is a tricky issue. I assume you are using transition probabilities to pick the next suitable word, so you could use some other corpus data, derive probabilities from it, and compare  those to your system. Not very satisfactory, though, as you might end up evaluating the system in a circular way, deriving your test data in the same way as you generate your output.
A better, if somewhat more laborious evaluation would be to present the trigger sequence to human beings (on a fairly large scale), and then measure the overlap between your system and the human choices. Of course, with the you will get a very large set of possible replies, but the round yellow would be a lot more restricted (and it would make more sense to include that kind of sequence in an evaluation anyway).
The figure you will get might be quite small (depending on the range of possible options), but that is to be expected, so your system could score well even with a small overlap only. That's how I would approach this problem.

Human evaluation is the gold standard as stated in this podcast by Asli Celikyilmaz, even if you only test a very small part of the generated text.
You needed an automated method and this one: BLEURT by Google would be helpful. It's a flexible, semantic-level metric/model trained in a multi-stage way: 1) masked language model like BERT; 2) pre-training on synthetic sentence pairs; 3) fine-turning on public human ratings(WMT metrics shared task); 4) fine-tuning on application-specific human ratings. And it applies very well even in the presence of domain drift.


text:
Are Modular Neural Networks more effective than large, monolithic networks at any tasks?
Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network.
In principle, the hierarchical organization could allow us to make sense of more complex problem spaces and reach a higher functionality, but it seems difficult to find examples of concrete research done in the past regarding this. I've found a few sources:
https://en.wikipedia.org/wiki/Modular_neural_network
https://www.teco.edu/~albrecht/neuro/html/node32.html
https://vtechworks.lib.vt.edu/bitstream/handle/10919/27998/etd.pdf?sequence=1&isAllowed=y
A few concrete questions I have:

Has there been any recent research into the use of MNNs?
Are there any tasks where MNNs have shown better performance than large single nets?
Could MNNs be used for multimodal classification, i.e. train each net on a fundamentally different type of data, (text vs image) and feed forward to a higher level intermediary that operates on all the outputs?
From a software engineering perspective, aren't these more fault tolerant and easily isolatable on a distributed system?
Has there been any work into dynamically adapting the topologies of subnetworks using a process like Neural Architecture Search?
Generally, are MNNs practical in any way?

Apologies if these questions seem naive, I've just come into ML and more broadly CS from a biology/neuroscience background and am captivated by the potential interplay.
I really appreciate you taking the time and lending your insight!

I'd thought about implementation of such a modular system to improve performance, and filter out - definitely - unnecessary dependencies on inputs. Thought it was a eureka moment, but didn't know it was already an established structure.

There is indeed an investigation in progress, regarding this topic. A first publication from last march noted that modularity has been done, although not explicitly, since some time ago, but somehow training keeps being monolithic. This paper assess some primary questions about the matter and compares training times and performances on modular and heavily recurrent neural networks. See:

Castillo-Bolado et al. Modularity as a Means for Complexity Management in Neural Networks Learning

Some others are very focused on modularity, but staying with the monolithic training (see Jacob Andrea's research, specially Learning to reason is very related to your third question). Somewhere between late 2019 and march next year, there should be more results (edit: mentioned results here).
In relation to your two last questions, we're starting to see now that modularity is a major key towards generalisation. Let me recommend you some papers (you can find them all in arxiv or google scholar):

Stochastic Adaptive Neural Architecture Search for Keyword Spotting (variations of an architecture to balance performance and resource usage).

Making Neural Programming Architectures Generalize via Recursion (they do task submodularity and I believe it's the first time that generalisation is guaranteed within field of neural networks).

Mastering the game of Go with deep neural networks and tree search (network topology is actually the search tree itself, you can see more of this if you look for graph neural networks).

A benchmark comparison of systems comprised of separately trained networks relative to single deeper networks would not likely reveal a universally applicable best choice.1 We can see in the literature the increase in the number of larger systems where several artificial networks are combined, along with other types of components. It is to be expected. Modularization as systems grow in complexity and demands on performance and capability grow is as old as industrialization.
Our laboratory works with robotic control, thermodynamic instrumentation, and data analysis, artificial networks are components in these larger system contexts. we have no single MLPs or RNNs that by themselves perform any useful function.
Contrary to the conjecture about hierarchies decades ago, the topology approach that seems to work well in most cases follows the more common system module relationships that are seen in power plants, automated factories, aeronautics, enterprise information architectures, and other complex engineered creations. The connections are those of flow, and if those are designed well, oversight functions are minimal. Flow occurs between modules involving protocols for communications, and each module performs its function well, encapsulating the lower level of complexity and functional detail. It is not one network overseeing another that seems to emerge most effective in actual practice but balance and symbiosis. Identification of clear master-slave design in the human brain seems to be equally slippery. It may be naive to assume that some all knowing subsystem can be a single causal factor in the remarkable collaborative synchronization of a number of other subsystems.
The challenge is not finding the information paths that make up the system information topology. The information flow is often obvious upon problem analysis. The difficulty is in discovering the best of strategies to train these independent networks. Training dependencies are common and often critical, whereas in animals, the training occurs in situ or not at all. We're discovering conditions under which that kind of learning in our systems is practical and how to achieve it. Most of our research along these lines is intended to discover ways to achieve higher reliability and lower burden in terms of research hours to get it.
Higher functionality is not always of benefit. It often produces lower reliability and consumes additional development resources with little return. Find a way to marry higher level automation, resource thrift, and reliability into one development process, and you might win an award and honorable mention around the web.
Parallel systems that have the same objective is a good idea, but not a new one. In one aeronautics system, nine parallel systems have the same objective, in groups of three. Each group uses a different computing approach. If two of the systems using the same approach provide the same output and the third differs, the matching output is used and the difference in the third is reported as a system fault. If two of the different approaches provide similar results and the third differs substantially, a merge of the two similar results is used and the third is reported as a use case to further develop the dissenting approach.
The improved fault tolerance has a cost, eight more systems and associated computing resources and connectivity plus the comparators at the tail, but in systems that are a matter of life and death, the extra costs are paid and the reliability is maximized.
Dynamic topological adaptation is related to redundant systems and fault tolerance but in some ways is quite distinct. In that area of development, the technology to follow is neuromorphic computing, which is partly inspired by neuroplasticity.
One last distinction to consider is between process topology, data topology, and hardware topology. These three geometric frames can produce greater efficiency together if aligned in specific ways that produce more direct mappings between the relationships between flow, representation, and mechanics. They are, however, distinct topologies. The meaning of alignment may not be apparent without diving deeply into these concepts and the details that emerge for specific product or service objectives.
Footnotes
[1] Deep networks that are trained as a single unit and function without connectivity to other artificial networks are not necessarily monolithic. Most practical deep networks have heterogeneous sequence of layers in terms of their activation functions and often their cell types.


text:
How to encode Azul game state as NN input
Question to NN practicioners. I'd like to encode Azul board game state as an input to NN, let's focus on 2-player variant for a while.

There are 5 round "Factories" on the table (7 on picture, ignore it). Each one can keep 4 tiles of 5 colors. There is also center of the board which can keep up to 15 tiles. What are the advantages and disadvantages of different encodings? Here are my ideas:

Every Factory has five integer counters, one for each tile color. Sample encoding of single Factory: [3,1,0,0,0]
Every Factory has 20 binary flags, four for each color. Single flag encodes presence of tile of given color. Sample encoding of single factory: [1,1,1,0, 1,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0]
Every Factory has 20 binary flags, four for each color, but only one flag can be set for given color and position of raised flag encodes number of tiles of given color. Sample encoding of single factory: [0,0,1,0, 1,0,0,0, 0,0,0,0, 0,0,0,0, 0,0,0,0]
Every Factory has 4 enum fields, with 6 possible values each (5 colors + empty). Sample encoding of single factory: [red, red, red, blue] or [Empty,Empty,Empty,Empty]

(note: encoding schema would also cover center of the board, up to 15 tiles, as said earlier. Of course player's board would also be encoded, but I don't want to ask too broad question)
I'd like to train NN to play Azul, which means it needs to properly process number of tiles taken in given round (up to 15 in theory, 2-4 in practice) because it would also need to indicate where to put all those tiles to player board positions.
Based on your experience, which encoding is most promising? Or maybe there is some better method I didn't think of? Or it is not possible to tell or it doesn't matter?


text:
Maximum number of neurons in a layer given number of neurons in previous layer
Consider an extremely complicated feed-forward neural network training example but with no need of computational efficiency or limiting of processing time.
What is the maximum number of hidden neurons h that a hidden layer should possess to detect all unique features/ correlations between input data from the previous layer which has n nodes?
In other words if we wanted to create a neural network with a large number of neurons in a hidden layer, what is the maximum neuron count possible that helps the network train (give n neurons are in the previous layer)?


text:
Geometry shape identification and vertex/side label association
I want like to be able to draw a shape outline e.g. (pen and paper) triangle, square, circle.. then label the vertices and sides And have ML identify the shape and the symbol associates with each vertex / side.
For example a triangle and I label it with the adjacent, opposite and hypotenuse. Or draw parallel and perpendicular lines and label angles and such.
I would prefer it make one myself rather than use a pre built one. However if you know of a simple pre built one then I will ould love to pick it apart.
Can you please share some guidance on how to solve the above problem, namely shape identification / vertex and side labelling.
Any language


text:
How to use AI to depth map video?
To be honest, I had no idea where to put this question, but it's sure that it's related to AI. I want to build an application which uses camera, and by the movement it can calculate the
-camera's position compared to the objects
-the objects creator and edge points by the movement.
What it means that if the camera is in a static position, it's just a picture. A set of coloured pixels. If we move the camera, we calculate the time, the gyroscope's values, but most importantly, we can have a comparison of two images taken by the same objects. This way:
-we can detect the edges
-from the edges, we can detect which is closer than the others
Today's phone camera's are accurate enough to create ~60 crystal clear images per second, and it should be enough resource to accurately create high res models from just moving the camera according to some instructions (that's why I'm surprised why it isn't existing in just a phone app). Here comes the problem. I think the idea is worth for the try, but I'm just a JavaScript developer. The browser can have access to the camera, with TensorFlow I can use machine learning to detect edges, but if I want to be honest, I have no idea where to start, and how to continue step by step. Can you please provide me some guidelines how it would be ideal to create the idea?

I think that the tasks you are referring to are the Simultaneous localization and mapping (SLAM) and, in particular, the Structure from Motion (SfM).
These methods are usually based on geometrical constraints and do not employ neural networks, but there exist some recent methods that make use of CNNs (such as this one).
Structure from Motion algorithms are a fundamental component of the Google ARCore (former Project Tango) and the Apple ARKit.
Unfortunately, these kits usually provide an interface to put elements in a specific position, but they don't provide the access to the 3d reconstruction of the scene.
If you'd like to play with SfM algorithms, I suggest you to start from this repository, but you can find a lot of other valuable tutorials online.


text:
How to get a binary output from a Siamese Neural Network
I'm trying to train a Siamese network to check if two images are similar. My implementation is based on this. I find the Euclidian distance of the feature vectors(the final flattened layer of my CNN) of my two images and train the model using the contrastive loss function.
My question is, how do I get a binary output from the Siamese network for testing (1 if it two images are similar, 0 otherwise). Is it just by thresholding the Euclidian distance to check how similar the images are? If so, how do I go about selecting the threshold? If I wanted to measure the training and validation accuracies, the threshold would have to be increased as the network learns better. Is there a way to learn this threshold for a given dataset?
I would appreciate any leads, thank you.

I am working on something rather similar in the sense that I get a continuous value (between 0 and 1) as output that I want to be binary. I think you should be using the ROC curve (Receiver-Operating-Characteristic) instead of the accuracy. If you want a single value take the area-under-the-curve. The ROC curve is basically true-prositive-rate vs false-positive-rate as you test for different values of threshold. You could also (or instead) use precision vs recall.
Also, the optimal threshold does not depend as much on the dataset as it depends on your neural network. As you said this value will change as your network learns. Simply use the outputted value and test different thresholds.
If do want to keep the accuracy, simply pick the threshold for which is it best (at the given iteration). Keep in mind that you should not define the threshold from the validation set as you would be cheating.
ps: from rereading it seems like you were implying to learn the threshold from inside the network. I believe it should just be a post-process

I'll look into the ROC curve idea, thank you.
more than learning the threshold, I was wanted to know if there are other ways to let a Siamese Network give a 0/1 output or if thresholding the Euclidian distances is the only way

Unfortunately I don't know for your question regarding the siamese network. I would think there is no such method.
Also note I edited my answer as my description of roc was not accurate.

Don't use the Euclidian distance as the similar/dissimilar factor, you'll get better results if you put a couple Dense layers at the top of your Siamese network.  You don't mention how large your feature vectors are, but if they are 128D face encodings, this is what I used:
# dist_euclid_layer is a layers.Lambda that performs the euclid distance on the input
first_dense = layers.Dense(512, activation='relu')(dist_euclid_layer)
drop_one = layers.Dropout(0.5)(first_dense)
output_dense = layers.Dense(2, activation='sigmoid')(drop_one)
model = models.Model(inputs=[input_layer], outputs=output_dense)
model.compile(loss=losses.binary_crossentropy, optimizer=opt, metrics=['accuracy'])

In training, use [0,1] or [1,0] for Y to indicate different / similar.
Then in production, use the np.argmax to find the positive match:
predictions_raw = model.predict(vstack_batch_input, batch_size=len(batch_input))
predictions = np.argmax(predictions_raw,axis=1)
tot_matches = np.sum(predictions)
if tot_matches == 1:
    # ... good times, whatever face[N] has prediction[N] == 1 is the matching face


text:
Recognition of small objects
I'm currently implementing an Android app for street sign recognition. My solution works quite well for the GTSRB dataset, since it provides a labeled test set of centered images. However, it doesn't scale up to more realistic scenarios like for images in the GTSDB, where the signs only take up some pixels. Is it still recommended to downsample the image to 224x224?

There is a big difference between the 2 datasets. When you train on the GTSRB dataset, your model performs image recognition, while for the GTSDB dataset you need to perform object detection. These are 2 related but different tasks (with detection being considerably harder).
You can consider using a sliding window approach, or use some prior attention model that will learn to extract the area of the signs for later classification.
I don't think that it is recommended to down sample images of 1360x800 pixels to 224x224.
(224x224)/(1360x800)=0.046. You are loosing 96% of the information (including your street signs which might reduced to nothing), granted most of it is not interesting, but simple down-sampling doesn't get to choose what to forfeit.


text:
How TensorFlow will know if the prediction is true or false?
I'm completely new at ML, but really interested. To be honest, read many articles about it, but still don't understand the workings of it.
I just started to understand this example: https://storage.googleapis.com/tfjs-examples/mnist/dist/index.html
My thinking about it is that TF has some resources, some examples of how numbers look like, and try to match them with the ones in the test. I saw that sometimes the test changes a right prediction to a wrong, but makes better and better predictions. But how? I think that the program doesn't know the right predictions (and this way it won't know the wrong ones). In the training how it makes better predictions? Test by test, from what exceptions it will change it's predictions? What happens in a new test?

I'm not quite sure what you mean in your post however I will try and address some of the points you made. Firstly, I would suggest you to go over some supervised machine learning fundamentals (there are lots of resources online).
The general optimisation approach is to find the parameters of a model that minimise the error in your training set (using the ground truth labels). The model may not be complex enough to achieve absolutely zero error on all the training samples, and if it did it would likely generalise poorly to new data. We don't want a model to "memorise" the labels, for example, picking up contextual cues in a training image that are not relevant to the label (such as the background of the picture of a car).
An ML model aims to reduce the error over all the samples and this is why the correct classification of a specific input can change as the model learns.

During the training process, the machine learning program is shown example inputs (in this case pictures of digits) and also what the answer should be for that input. That is how it learns the correct answer.
However, it does not learn an exact mapping between the input data and required output, instead it learns a series of patterns that generally map the input to the required value. The advantage of this approach is that it can perform the same mapping on new inputs that were not included in the training data but are similar (but not necessarily identical) to examples that it has already seen.

But if it knows what answer should be in a specific input, then why it changes a right answer to a wrong sometimes?

Are you asking why it sometimes gives an incorrect answer? That is because it has not learned perfectly. In the case of the MNIST dataset a few of the input images are also a bit ambiguous even for humans. Note also that these programs are tested on inputs that were NOT included in the training data, to check that they have learned to generalise properly.

I don't understand the mechanism of it. If the program knows the right answer, how can we talk about learning? What do you mean by "not learned perfectly"?

OK, it sounds like a introduction to the basics of Machine Learning would be valuable for you. I recommend the following course: https://www.coursera.org/learn/machine-learning which should give you an idea how these programs work.


text:
Why do layered neural nets struggle with continous data?
In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled.
It was my understanding that this was the entire purpose of techniques that use neural nets, to make assumptions about a system with a non-continuous data set.
So why do we need to switch to a non-layered design to deal with these data sets better?

If time (or intervals between measurements) matter then neural nets don't work as well because Neural nets excel at finding patterns. Neural nets tend to find patterns that have the same sequence, regardless of interval between inputs. If a sequence of inputs is exactly the same taken at fixed intervals or sparse intervals, the neural net will tend to find the patterns as the same; which may not be desirable. If a person's health is evaluated at 30 then 50, you'd want different results than evaluations between 30 and 31, assuming the 31 year old's health now matches the 50 year old's health.

I only posted because you got no answers and I thought I could shed a little bit more light on what the article said but did a poor job of conveying in an easily understandable way. My comment isn't worthy of being posted as an answer to your question but thanks for the offer.

They struggle because if your network have an inductive bias towards modeling datasets which are described with ODEs well, you will learn faster, and with smaller dataset. I think, this what the authors of the original article meant.
In a similar way, CNNs recognize images much better, because their features are translation invariant whereas fully connected net needs to learn to recognize a cat in each different position from scratch.


text:
Is there bidirection sequence-to-sequence neural machine translation?
I have heard about bidirectional RNN LSTM units (endcoders-decoders), but my question is - is there bidirectional neural machine translation, that uses A->B weights for the translation in the opposite direction B->A? If not, then what are the obstacles to such system?

Bidirectional Mapping of Arbitrary Functions
Interchange between what properties are considered labels and what properties are considered features can support bidirectional map learning.
Unidirectional training represents the objective of functional approximation $f_p$ in terms of features $\mathcal{X}$ and labels $\mathcal{Y}$ through convergence of parameters $p$, ideally toward an accurate and reliable approximation.
$$ \mathcal{Y} \Leftarrow f_p(\mathcal{X}) $$
Because only a small proportion of arbitrary functions are monotonic (have only one value in the domain for any given value in the range), the existence of an inversion of $f_p$ is unlikely. This is certainly true of natural language processing functions. This fact will be further illuminated below.
Bidirectional training of a non-monotonic translation function $f_p$ and a reverse translation $g_p$ can be represented as two adjacent training objectives.
$$ \mathcal{Y} \Leftarrow f_p(\mathcal{X}) $$
$$ g_q(\mathcal{Y}) \Rightarrow \mathcal{X} $$
Function $g_q$ is not a guaranteed inversion of $f_p$ and may rarely be. Parameters resulting from convergence $q$ may not be derivable from parameters resulting from convergence $p$ or vice versa either.
Temporal Bidirectionality to Minimize Round Trip Inaccuracy
For reasons further illuminated below, temporal bidirectionality may be important in some use cases. For this reason, use two B-LSTM networks.
There may be some efficiency gains by training both B-LSTM networks in the same training phase, especially if more than one host, GPU, parallel signal processor, or AI accelerator is available at training time. The same is true of test and validation.
The ontology of network types is as follows.

RNNs (recurrent artificial networks) are a stateful extension of MLPs (multi-layer perceptrons).
LSTM (long short term memory) networks are an extension of RNNs, and use gating within each cell to adaptively regulate retention of state and its dissipation.
B-LSTMs provide bidirectionality in the temporal direction, which allows more variety in what causality and reactance can be modeled.

Importance of Contemporary Concepts in Linguistics
Since the natural-language-processing tag was included with the posting of the question and the term, "Machine translation," was mentioned, it appears that bidirectional natural language translation using AI is the project goal. If that is the case, a contemporary linguist's understanding of words sequences in text and the sequence of phonetic elements in speech will help with developing an effective solution using AI. An engineer should be aware of the interplay between expectations placed on project results and technical feasibility at the current state of NLP technology.

Human translation experts rely on cognition to accurately represent text that may contain cultural references, analogies, colloquialisms, and other literary or oratory devices.
Because of rhyme, alliteration, cadence and other phonetic devices, prose and poetry may not have an equivalent in the target language.
Linguistic elements are not strictly words. A single word may have multiple linguistic elements because of tenses, plurality, gender, prefixes, suffixes, and other modifiers. For example, "Multicolored," is an adjective made up of three linguistic elements: {"multi-", "color", "-ed"}. A single idea is sometimes represented as multiple words, the individual meanings and juxtaposition of which does not imply the meaning of the word group. For example, "I don't wish to hitch my apple-cart to that wobbly wagon," has nothing to do with apples, carts, mechanical stability, or wagons.
The translation $\mathcal{A} \Rightarrow \mathcal{B} \Rightarrow \mathcal{C}$ may not lead to $\mathcal{A} = \mathcal{C}$.
When a series of linguistic elements are communicated, the semantics represented are not often a series. For instance, "Flying birds can also jump," is equivalent to, "If a bird is able to fly, it can jump."  This works in natural language because the associations and modifying aspects between linguistic elements is a semantic network that correlates with a cognitive ontology. The serialization of this network and its reconstitution in another mind cannot be formalized in a way that eliminates all ambiguity. Communication $\mathcal{K}_o \Rightarrow \mathcal{L} \Rightarrow \mathcal{K}_r$, where $\mathcal{K}$ are cognitions at originating intelligence $o$ and target intelligence $t$ respectively and $\mathcal{L}$ is a language expression in between, may not lead to $\mathcal{K}_t = \mathcal{K}_o$.
For the above reasons, formal grammar has been discarded as a set of principles that can facilitate NLP. Humans neither write nor speak in conformance with any formal grammar on a regular basis.

Expected Dominance of Recursive Network Technology
For this reason, LSTM networks may not be the direction of NLP technology development. Recursive network designs (often using RNN as an acronym but referring to a distinct subset of recurrent networks) can train semantic network models based on serial streams of linguistic elements. That appears to bear closest resemblance to language centers in the human mind, and may emerge as a dominant NLP component in emerging AI systems.


text:
a question about Zeiler's paper “Deconvolutional Networks”
In "4.1 Learning multi-layer deconvolutional filters" section, the last paragraph says that "Since our model is generative, we can sample from it. In Fig. 3 we show samples from the two different models from each level projected down into pixel space. The samples were drawn using the relative firing frequencies of each feature from the training set."

I don't know how the pictures in Fig.3 are generative. Since that the filters has been learned and feature maps in every layer can be infered, for example, in terms of fruit samples, in "Layer 1", is the the first layer's feature map ? i feel that not true...seems like the sample are low-level...paper says "... from each level projected into pixel space", these words are short and confuse me.
somebody could explain that for me? thank you very much!


text:
Relationship between input range and channel means, standard deviations for CNNs
So, I'm using a pretrained PNASNet-5-Large model to do some image classification.
In the file, it says that the input range is in [0,1] (I'm assuming pixel values of input images). The images I have are already in this range.
The channel means and standard deviation for RGB channels are stated as [0.5, 0.5, 0.5], [0.5, 0.5, 0.5] respectively.
Now when I use the torchvision.transforms.Normalize to normalize the images using the stated means and standard deviations, the pixel values get to the range [-1,1].
The code I wrote for normalization:
transforms.Normalize([0.5, 0.5, 0.5],[0.5, 0.5, 0.5])

I believe I'm missing something fundamental. Should I normalize the images or should I not? Thanks!


text:
Translating a single word Neural Networks
I've been given an assignment to create a neural network that will suggest a Croatian word for a word given in any other European language (out of those found here). The words are limited to drinks you can find on a bar menu.
I've looked at many NN examples, both simple and complex, but I'm having trouble with understanding how to normalize the input.
For example, words "beer", "birra" and "cervexa" should all translate to "pivo". If I include those 3 in the training set, and after the network has finished training I input the word "bier", the output should be "pivo" again.
I'm not looking for a working solution to this problem, I just need a nudge in the right direction regarding normalization.

The training examples are not mentioned, but, assuming the assignment involves supervised learning, normalization is, in this case, mapping the inputs and outputs to binary values.
Translation, in this simplified context, is a map of strings to strings, which isn't really an AI device by contemporary standards. Since an artificial network is given as a requirement for the solution, the goal is to train the network to act like a hash map.
$$ \Big\{..., \;
\big(
[\text{"beer", "birra", "cervexa"}]:
\text{"pivo"}
\big)
, \; ...\Big\} $$
$$ \Downarrow $$
$$ \Big\{..., \; \big(\text{"beer" : "pivo"}), (\text{"birra" : "pivo"}), (\text{"cervexa" : "pivo"}\big), \; ...\Big\} $$
Assuming there is a list of pairs $(\mathcal{X}, \mathcal{Y})$, where $\mathcal{X}$ is the word in any of the source languages and $\mathcal{Y}$ is the word in the target language, and the vocabulary of both is known to have word counts of $X$ and $Y$ respectively, one can determine the input and output layer widths (cell counts) $w_1$ and $w_{\ell}$, where $\ell$ is the number of network layers.
$$w_1 = \text{ceiling} ( \log_2{X} )$$
$$w_{\ell} = \text{ceiling} ( \log_2{Y} )$$
The assignment of source words to binary values is arbitrary. You can sort them by language and then by whatever common character encoding used (such as UTF-8) just for debugging purposes. The same is true of target words.
A simple MLP (multi-layer perceptron) with mini-batch training using basic propagation map be appropriate.

So, the normalization as you call it is just encoding the word into some vector of fixed length. After you encode it, you should be able to decode it and match the words. You should google a little, for a seq2seq problems and Encoder/Decoder structure. There are encoder-decoder frameworks out there and lots of resources. You don't care about a language so just pretend that words like "beer" and "birra" have the same meaning in Croatian and in this case, it is "pivo".


text:
Choice of fuzzification function
I'm a relative newbie to fuzzie logic systems but I have some knowledge in mathematics. I have the following problem:
I want to fuzzify certain values. Some are in the range [-$\inf$,$\inf$] and some are in the range [$0$,$\inf$]. For the first range I have chosen the sigmoid function:
$f(x) = \frac{1}{1+e^{-x}}$
The question is, which fuzzification process I should use for the second range. Since the function $f(x) = \ln(x)$ transforms [$0$,$\inf$] to [-$\inf$,$\inf$] a natural choice could be:
$f(x) = \frac{1}{1+e^{-\ln(x)}} = \frac{x}{x+1}$
A different function could also be:
$f(x) = 1 - 2^{-x}$
Which one would be more suitable? Particularly when considering that I may want to compare values from both ranges.


text:
Image Segmentation Prediction with cropping 256x256 grids is very slow
I have only a limited dataset (<25) with large-sized images (>1500x2000) and their pixelwise labels. The aim is to find unusual patterns in this industry dataset and highlight them.
To generate training images I crop 256x256 grids out of every image and do some data augmentation and use these images to train my U-Net.
For my prediction I split my image with numpy again into 256x256px grids and predict every grid separately and put them together to an image. But this will take some time, like >10 Minutes. But it has a quite good accuracy. How can I optimize my prediction to be faster?
Is it faster to create this with a Tensorflow Pipeline? When I want to predict the full image with giving shape(None,None,3), I get "ConcatOp : Dimensions of inputs should match" after some time.


text:
I need to predict ball position from set of Images
I am a novice developer in AI. Any help appropriated.
I have a set of images and from that I want to predict position(x,y co-ordinates) of the Ball.
Thanks in advance.

There are so many challenges here for a novice or even a very seasoned AI developer!  There hasn't been a lot of development in this area.  I assume you are working with broadcast video.  Your biggest challenges will involve occlusions of the ball and changes in the view as broadcasters constantly switch cameras which have very different positions on the field or around the stadium.  You will need to track the ball and the players that take action on the ball.  The ball can be in motion from a player heading, kicking, or throwing the ball.  The ball can also be in motion from a bounce off a player or the ground.  To get started I recommend building off some work that at least can track the players and the ball and can identify certain actions such as a kick.  If you start there you can build some type of ball trajectory predictor.
The following paper Soccer Event Detection, by Abdullah Khan1, Beatrice Lazzerini, Gaetano Calabrese and Luciano Serafini, presents an event detector for "Ball possession" and "Kicking the ball".
A paper entitled The Newton Scheme for Deep Learning by Junqing Qiu, Guoren Zhong, Yihua Lu, Kun Xin, Huihuan Qian, and Xi Zhu, addresses predicting the trajectory of a ball in sports.  They address the complexities of a ball in motion.  They address situations where "the ball is flying with a self-spinning, resulting in a Magnus force on it, which makes it difficult to predict the position by traditional methods".

Some related articles and papers you might find of interest:

Identifying Player Possession in Spatio-Temporal Data
Deep Learning using CNNs for Ball-by-Ball Outcome Classification in Sports
Detecting events and key actors in multi-person videos
Learning to Track and Identify Players from Broadcast Sports Videos
Video Artificial Intelligence Powers Creation
AI predicts how athletes will react in certain situations
How AI-Based Sports Analytics Is Changing the Game

Requirements Analysis
The term prediction has a temporal element, but the other text in comments indicates that the position of the ball relative to the field of play and goals at the time the image is taken is desired. Predicting where the ball is likely to be in some number of seconds is a much more difficult problem.
We can consider the problem of determining ball position $(x, y, z)$ relative to a coordinate system superimposed onto the field where $(0, 0, 0)$ is the center point in team A's goal line and $(1, 0, 0)$ is the center of team B's goal line and  distances in $x$ are scaled similarly in $y$ and $z$.
Since some of the images may have the ball in the field of view, the beginnings of a solution may be available. The number of images would need to be large. If a sufficiently large set of images, some of which have the ball in view, were available, this process would likely produce a viable solution with satisfactory accuracy and reliability characteristics.

Develop an unsupervised system that recognizes round objects in the air based on a model of balls.
Locate the ball in all frames that have one and assign the ball position as a label for those frames.
Use face recognition AI to locate eyes and label them.
Use CNN feature extraction and a deep network to train functional correlation
between eye features and ball position
Use the functional correlation on frames without a ball in the frame to locate it

There are two dimensional challenges in using these techniques

Lack of depth perception
Orienting players in the field

It is possible to train the trigonometric relations as part of item 4, however orienting players in the field may be untenable, rendering the entire project infeasible. The solution of this problem is dependent upon the solution of this one: https://ai.stackexchange.com/search?q=user%3A4302+field+boundary.


text:
Convolutional Sequence to Sequence Learning: Training vs Generation
I am struggling to understand the use of the Convolutional Sequence to Sequence (Conv-Seq2Seq) model.  The image below is take directly from the paper and is the nearly canonical diagram of the parallel training procedure.  After puzzling over it for quite some time, it has come to seem straight forward to me:

An input sentence of N tokens can be encoded in one step because the input sentence exists prior to the start of training, and therefore the token-wise convolution can be trivially parallelized. (Compare to RNN encoders, which require N steps)
During training, an output sentence can similarly be parallelized in the decoder because during training, the entire output sentence is known.
Therefore, during training, the attention function can be fully parallelized in the two dimensional array of dot products shown below
Finally, during training, the attention is used to weight the input embeddings and encodings, combined with the output training encodings (as such) and the final output assembled.

This is clearly not the case after the network is trained and evaluation input sequences are translated without reference outputs.  I understand from various resources (including but not limited to Gehring's conference presentation) that post-training, output sequences are generated token by token in a fashion vaguely similar to earlier architectures, but I cannot find a clear description of that process.
(I speculate that this is because the parallel training routine was so revolutionary at the time, that the focus of the publications was rightly on the training routines.)
Can someone please help me understand the post-training generation algorithm, if possible in terms of the training diagram?
My current non-confident understanding is that the sentence below would be handled something like the following:

Prime the decoder with a default token string of <p> <p> <s>, this results (due to convolution) in a single decoder encoding (as such) input into the attention function, and would hopefully generate the single token <'Sie'> as output
Restart the decoder with the token string <p> <p> <s> <'Sie'> which would generate two inputs into the attention function and hopefully output <'Sie'> <'stimmen'>
Proceed with lengthening input sentences until the final token generated output ends with a </s> token signifying the end of the sentence.

If that is a correct understanding, can someone confirm it?  If close, can someone correct me?

Convolutional Sequence to Sequence Learning, Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin, 2017

I know it's too late to answer your query after 1.5 years
The inference algorithm is explained at https://charon.me/posts/pytorch/pytorch_seq2seq_5/#inference
Implementation of the algorithm is available at: https://github.com/pytorch/fairseq.
One could get a complete understanding from their source code.


text:
How do the relative number of cells between neighboring stacked LSTM layers affect the network's behavior?
It seems that stacking LSTM layers can be beneficial for some problem settings in order to learn higher levels of abstraction of temporal relationships in the data. There is already some discussion on selecting the number of hidden layers and number of cells per layer.
My question: Is there any guidance for the relative number of cells from one LSTM layer to a subsequent LSTM layer in the stack? I am specifically interested in problems involving timeseries forecasting (given a stretch of temporal data, predict the trend of that data over some time window into the future), but I'd also be curious to know for other problem settings.
For example, say I am stacking 3 LSTM layers on top of each other: LSTM1, LSTM2, LSTM3, where LSTM1 is closer to the input and LSTM3 is closer to the output. Are any of the following relationships expected to improve performance?

num_cells(LSTM1) > num_cells(LSTM2) > num_cells(LSTM3) [Sizes decrease input to output]
num_cells(LSTM1) < num_cells(LSTM2) < num_cells(LSTM3) [Sizes increase input to output]
num_cells(LSTM1) < num_cells(LSTM2) > num_cells(LSTM3) [Middle layer is largest]

Obviously there are other combinations, but those seem to me salient patterns. I know the answer is probably "it depends on your problem, there is no general guidance", but I'm looking for some indication of what kind of behavior I could expect from these different configurations.


text:
At which point we have to stop post pruning in decision tree?
Post pruning is start from downward discarding subtree and include leaf node performance. so what is the best point or condition of the tree where we have to stop further pruning.

Hi. You should really add more context to your questions. For example, it's not automatically clear what this "post pruning" procedure specifically does. You could describe it in the question or link to a website that does it for you. Otherwise, people probably won't know what exactly you're talking about.

Maybe this might help: https://www.saedsayad.com/decision_tree_overfitting.htm?

post pruning is one of  a  method to avoid overfitting. it is use to reduce the complexity of tree.

@hinamunir How "post-pruning" different from "pre-pruning"? I had only heard of "pruning" in general. Your question is actually a useful question. Please, edit your post to include the definition of your terms, like post-pruning.

There are a variety of conditions we can use when deciding whether to prune a sub-tree or not after generating a decision tree model. There are three common approaches.

We can prune branches with less support than a specific threshold. These are branches which were constructed using very few points from the training data.
We can prune branches where the information gain from a split (or any other splitting measure we are using, like GINI), is smaller than a threshold.
You can do what is done in Quinlan's 4.5 & C5.0 learners (which are the standard approaches; J48 is another implementation of the same algorithm). Quinlan performs a Chi-squared-like test for the relationship between the attribute we split upon and the target attribute. If the relationship is statistically significant, then the split is preserved. If not, it is not. The "confidence factor" parameter found in most implementations of these algorithms corresponds to the $\alpha$ value used in determining whether the relationship is considered significant. This approach captures the idea that we should prefer to keep branches that have few datapoints, but a very strong signal, or that have a weak signal, but very many datapoints supporting the pattern, since both cases are less likely to be overfitting that cases where we have weak signals and small numbers of points.


text:
Deep Learning for radio signal classification with DeepSig dataset
I want to see if I can make my Software Defined Radio, SDR, to classify unknown radio signals with the help of an artificial neural network. That is, my SDR outputs a sequence of complex numbers (IQ-data), which I want to use to determine if the receieved signal is, for instance, FM or AM modulated. This approach was used in a paper (https://arxiv.org/pdf/1712.04578.pdf) and they created and used a freely downloadable dataset (https://github.com/sofwerx/deepsig_datasets/blob/master/README.md).
Being new to both SDR and Deep Learning I have now tried for a couple of months to create an LSTM network, train it on the dataset and then use it for classification, but have sadly failed. I have concluded that it is likely due to the fact that I do not seem to understand how the dataset is structured. I have not been able to find any documentation concerning this dataset (other than a text file with a list of the modulation forms used in the dataset) though. My hope is that someone on this forum has some prior experience to share about how to use it.
The dataset (DEEPSIG DATASET: RADIOML 2016.10A) is split in three matrixes, X: 2x1024x2555904 cells, Y: 24x2555904 cells and Z: 1x2555904 cells. My belief has up until now been that "X" contains the complex time series, i.e. one row for the real component and one row for the imaginary component both in a sequence of 1024 samples. The "Y" to contain the corresponding 24 classes, whereas the "Z" somehow contain the signal to noise level for each signal.
After training and deploying the network in Matlab I found out that it does not even manage to classify the local FM radio station. I then looked at the data I had used for training and plotted one of (what I thought to be) the FM sample sequences from the dataset in the complex plane, but I did not get the "circle" I had expected from a frequency or phase modulated signal. I am thus totally lost, the dataset has been used for a high class scientific paper so the problem lies with me, and my lack of understanding. I apologize for the long and still very unconcise question, but I don't want to infer any of my own misconceptions into the query. Thanks for any help!

This link shows an example of doing exactly what you are trying to do. https://medium.com/gsi-technology/residual-neural-networks-in-python-1796a57c2d7


text:
Has any research been done to solve word searches with AI?
A simple word search seems too simple to solve with a computer AI. But what I'm interested is how human's solve it. They build up strategies over the course of solving the puzzle. For example:
1) first just look to see if any words "jump out".
2) look for the horizontal words.
3) look for the letter "O"
4) look for the letter "p" next to a letter "o"
5) methodically look along rows and columns and diagonals.
6) cross off words
7) take the first letter of a word in the list and put it in memory.
Things of that sort. I would like to build such a program. With built in search capabilities, some of which are faster than others. And the AI can combine different search methods and try to solve the puzzle as fast as possible.
It should also store strategies that work. Or think about why strategies work sometimes and not others.
I would like to read a bit more about AI and strategies if you know any good references?


text:
Can gamma be greater than 1 in a DQN?
If I have a DQN, and I care A LOT about future rewards (moreso than current rewards), can I set gamma to a number greater than 1? Like 1.1 perhaps?

$ \gamma $ goes up to 1, but cannot be greater than or equal to 1 (this would make the discounted reward infinite).
The discount factor $ \gamma $  determines the importance of future rewards. A factor of 0 will make the agent "myopic" (or short-sighted) by only considering current rewards, while a factor approaching 1 will make it strive for a long-term high reward. If the discount factor meets or exceeds 1, the action values may diverge. For $ \gamma =1$, without a terminal state, or if the agent never reaches one, all environment histories become infinitely long, and utilities with additive, undiscounted rewards generally become infinite.
Source:https://en.wikipedia.org/wiki/Q-learning
https://cs.stanford.edu/people/karpathy/reinforcejs/puckworld.html

+1.  Exactly, simply set gamma to 1, or something very close to it.  Even the value of 0.99 generally works fine, and still avoids the terminal state search issue.

You can't! As if you have a $\gamma$ greater than $1$ the specified sum for the q-learning will diverge! (goes to infinity in the future steps for $\gamma^n$). To know more about that please more scrutinize on the specified formula for the q-learning.


text:
Extending a neural network to classify new objects
Suppose a model M classifies apples and oranges. Can M be extended to classify a third class of objects, e.g., pears, such that the new images for 'retraining' only have pears annotated and apples and oranges ignored? That is, since M already classifies apples and oranges, can the old weights be somehow preserved and let the retraining focus specifically on learning about pears?
Methods such as fine-tuning and learning without forgetting seem to require all objects in the new images annotated though.

Yes this is standard transfer learning. Using a trained model, we can freeze the first N hidden layers of a classifier, except for the last few. This will allow our previous relevant training to be retained whilst also being able learn new features and target new classes.
We will then initialize a new output layer that works our new context(i.e sigmoid, 1 node for binary classifier). Everything is now set to resume training on our new y_targets.
Take a look at this link for some more info on transfer learning.
If you want no perturbance to your past learning, I would recommend freezing all previous hidden layers and then tacking some additional ones on.

But doesn't fine-tuning require even the old objects annotated in the new training images? I'm trying to avoid having to annotate the old objects in the new images.


text:
How do we ensure that training GANs will fall in the desirable Nash equilibrium?
One Nash equilibrium of every GANs model has is when the generator creates  perfect samples indistinguishable from the training data and the discriminator just output 1 with probability 1/2. And I think this is the desirable outcome since we are most interested in the generator part of the GAN model. I know that we probably try to converge to this equilibrium with some hacks in training such as "mode collapse avoidance" and so on. But is there any theoretical work trying to go in another ways (say, by reduce the number of Nash equilibria somehow?)