text:
How do the achievements met in the gaming field (ex. AlphaGo Zero) impact other fields of application?
How can we use the ability of AlphaGo Zero computer, to do something in any other life important related field? Is it possible to make something important besides having created something so smart that can play mind games way better than humans?

Yes it's created something important. Until Alpha(Go) Zero all (or almost all) of Deep Learning approach to Reinforcement Learning was based on Time Difference loss function. The weakness of Time Difference loss function was that it was essentially training on itself, that is data produced by the same method was used as part of regression target. That was producing the problem of "extrapolation error" - solution would blow up, or oscillate wildly. There was attempts to mitigate that problem (n-steps algorithms), but they weren't helping much. Alpha Zero combined Deep Network with tree search instead (Monte Carlo Tree Search). Tree search algorithm produced wide and long fields of high precision data (value function), with network influence on value much diminished. That way network was training mostly not on itself, but on data produced by tree, and tree search itself was accelerated greatly by network (using it as heuristics). The whole happens to be much more then sum of parts.
This approach is not limited to broad game or RL theory. It may work for any problem for which high-precision simulator could be built. Essentially if problem allow Monte Carlo Tree Search, or other tree search which could be augmented with heuristic, Alpha Zero approach would probably work on it. Of cause Alpha Zero approach is computationally expensive, so it's not always efficient to apply it.


text:
Shortest route GA: One loop through one dataset vs multiple loops through subsets of the same data?
I've a rather simple question for a school project. We're developing a GA solution for the following problem:
Chromosome: A location with lat-lon coords. There are two types of locations - up to 15 waypoints from user input, and a dataset of about 3-400 stations.
Gene: A route consisting of all waypoints (incl. a fixed start and end) + 1 station.
Fitness function: Shortest path.
Stop condition: Run duration - configurable, default 3 seconds.
We're discussing two possible implementations:

A problem set of all waypoints and all stations, kinda like soccer team assignment or nurse rostering design. Run GA once on all of that.
A problem set of all waypoints and one station, do TSP. Run GA for number-of-stations-in-dataset iterations.

Which is better, in terms of design, efficiency, and performance?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How to make meaningful sentences from a set of words?
I have set of topics generated using LDA and  like {code, language, test , write, function}, {class, public, method, string, int} etc and I want to make meaningful sentence/sentences from these words using  api or libraries.  How do I implement this with NLTK and(or) Machine Learning? Any suggestions as to how I should go about this?

How do you define "meaningful"? Generally, you would start from concepts and meanings, and then realise them in syntactic structures using lexical items (words). You seem to want to start in the middle somehow.
For turning a semantic representation into a valid sentence, you would use a generator; these are often based on grammars. Examples exist which take a grammar, fill in random words, and create a syntactically well-formed sentence; often they will, however, be rather non-sensical or meaningless. Have a look at this site which describes the Syntax Construction Kit. The author, Mark Rosenfelder, links to a number of toy programs which do exactly that. Just substitute his lexicon with the list of words created by your LDA process. See for example this generator based on generative grammar.

Really appreciate your valuable suggestion it really helped me. The link of the API tool which you shared is a paid tool. I cannot pay for that as I am a student. Is there any free link or any student version?
Regards,

@shani Sorry, no idea! Might be an idea to contact the author?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What is the benefit of scaling the hyperparameter C of an SVM?
Please read the following page of the Sklearn documentation.
The figure shown there (see below) illustrates why C should be scaled when using a SVM with 'l1' penalty, whereas it shouldn't be scaled C when using one with 'l2' penalty.

The scaling however does not change the scores of the models examined within the GridSearch. So what exactly is this scaling-step good for?


text:
Why does precision-recall curve become more stable when neural net begins to overfit?
I am training a convLSTM with a dropout layer (with prob 0.5).
If I train over more than 5 epochs I notice that the network starts to overfit: my validation set loss becomes stationary while the train loss keeps going down with every epoch.
And if I train for 20 or more epochs the gap between the validation and train loss is quite substantial. At the same time precision-recall curve becomes much more stable (i.e. monotonic) if i train with a large number of epochs (e.g. 20). Why is that? Is this behaviour a common occurrence?

on the validation set

if validation loss does not bounce and start to get worse I would not call it overfitting...but what you are describing is common and not necessarily bad...the NN is memorizing the training examples, but if it is not impacting the validation loss adversely it might not be a bad thing...to get better validation loss you can try more/better data or regularization like L2.

yes, the precision-recall curve for the model trained with 20 epochs. When you train with less epochs it can have multiple dips (local minima).


text:
Deep Q-Learning agent poor performing actions. Need help optimizing
I'm trying to make deep q-learning agent from https://keon.io/deep-q-learning
My environment looks like this:
https://i.sstatic.net/EJHTD.jpg
As you can see my agent is a circle and there is one gray track with orange lines (reward gates). The bolder line is an active gate.
The orange line from the circle pointing to his direction.
The agent has constant velocity and it can turn left/right 10 degrees or do nothing
On the next image are agents sensors
https://i.sstatic.net/LqG8J.jpg
They are rotating with the agent.
The states are a distance from the agent to active gate and lengths of sensors.
In total there are 1+7 states and it is q-learning neural net input dimension.
Actions are turn left, turn right and do nothing.
Reward function returns 25 when the agent intersects reward gate; 125 when agent intersects the last gate; -5 if agent intersects track border
If none of this, reward function compare the distance from the agent to the active gate for current state and next state:
If current state distance > next state distance:
 return 0.1
else
 return -0.1
Also, DQNAgent has negative, positive and neutral memory.
If reward is -5, (state, action, reward, next_state, done) go to the negative memory,
if reward is >= 25, to positive
else to neutral
That is because when I'm forming minibatch for training, I'm taking 20 random samples from neutral memory, 6 from positive and 6 from negative.
Every time when agent intersects track border or when he is stuck for more than 30 seconds, I'm doing training (replay) and agent starts from the beginning.
This is my model
model = Sequential()
model.add(Dense(64, input_dim=self.state_size,activation='relu',
                  kernel_initializer=VarianceScaling(scale=2.0)))
model.add(Dense(32,
    activation='relu',kernel_initializer=VarianceScaling(scale=2.0)))
model.add(Dense(self.action_size, activation='linear'))
model.compile(loss=self._huber_loss,
                  optimizer=Adam(lr=self.learning_rate))
return model

I tried different kinds of model, a different number of neurons per layer, other activation and loss functions, dropout, batch normalization, and this model works the best for now
I tried different reward values
Also, I tried to use static sensors (they are not rotating with the agent)
https://i.sstatic.net/UCLGM.jpg (green lines on the photo)
Sometimes my agent manages to intersect a few gates before hits the border. Rarely he manages to traverse half of the track and once, with this settings, he traversed two laps before he stuck.
More often, he is only rotating in one place.
I think that the problem lays in state representation or reward function.
Any suggestions would be appreciated


text:
What is the best method to deal with heterogeneous multi agent system MAS?
Heterogeneity: Based on the heterogeneity of agents MAS can be divided into two categories namely: homogeneous and heterogeneous. Homogeneous MAS include agents that all have the same characteristics and functionalities, while heterogeneous MAS include agents with diverse features.
As I read in this paper that these methods can deal with the heterogeneity of agents MAS :
The dueling double deep Q-network (DDDQN) and Independent Deep Q-Network (IDQN): first approach to address heterogeneous multi-agent learning in urban traffic control.
deep Q-network (DQN): To handle heterogeneity,  each agent has different experience replay memory and different network policy.
The asynchronous advantage actor-critic (A3C) algorithm is used to learn optimal policy for each agent, which can be extended to multiple heterogeneous agents.  So, Can someone tell me What is the best method to deal with heterogeneous multi-agent system MAS?


text:
Deciding the rewards for different actions in Pong for a DQN agent
I am attempting to implement an agent that learns to play in the Pong environment, the environment was created in PyGame and I return the pixel data and score at each frame. I use a CNN to take a stack of the last 4 frames as input and predicts the best action to take, I also make use of training on a minibatch of experiences from an experience replay at each timestep.
I have seen an implementation where the game returned a reward of 10 for each time the bot returns the ball and -10 for each time the bot misses the ball.
My question is whether it would be better to reward the bot significantly for managing to get the ball passed the opponent, ending the episode. I was thinking of rewarding 10 for winning the episode, -10 for missing the ball and 5 for returning the ball.
Please let me know if my approach is sensible, has any glaring problems or if I need to provide more information.
Thank you!

Could you link the implementation you saw? I think the canonical version of Pong in RL just uses the game points scoring, i.e. +1 if you score, -1 if opponent scores. However, as a researcher you are free to come up with different ways of training an agent.

Do you have a specific reason for asking this question abstractly? If you can try both policies and compare them, why not just do this to directly answer the question?


text:
Evaluation metrics multi-class classification (ROC- PR curves)
Facing with a multi-class classification task,
my question is:
are ROC and Precision-Recall (One-vs-All-Rest) curves useful to evaluate and visualize the performance of a model?
or Confusion matrix, Precision, Recall, F-score (micro and macro) are enough?
What do you think about?


text:
Can AI help summarize article or abstract sentence keyword?
I'm wondering if AI now can help us abstract summary or general idea of long article, for example novel or historical stories, or abstract most important keyword from sentence;
Would you please tell me if any of this kind of project is done?
I wish I can improve my reading speed and effectiveness with AI help.

Yes. Text summarisation has been a research topic in (computational) linguistics for literally decades. Have a look at the Wikipedia page on Automatic Summarisation for an overview.
There are basically various different approaches: either, selecting salient sentences (or parts of sentences) which represent the gist of the text, or, trying to 'understand' the text and generating new sentences. The former is generally easier, and works on any text, while the latter would probably be able to produce better results, but is more complicated and would not work on any text, as it would be specific to a particular topic.

Thank you for your answer, It would be great if there's any available service on internet for text summarisation, what I need is just copy and paste article in text input, and it will return summarize result to me, then I can improve my reading speed soon

@gloomyson smmry is reasonably popular, and sounds similar to what you request.

Addition to dear Oliver Mason's answer
You can check here to see how people uses machine learning tools to summarize texts and also check some articles from here to understand its background If you are curious about.


text:
how to benefit from previous training weights in training again to increase accuracy?
I have trained  a modified VGG classification CNN, with random initialized weights; therefor the validation accuracy was not high enough for me to accept (around 66%).
now using the weights resulted from training the network, how can i use those weights in training the network again to improve accuracy? (e.g. using previous training weights with different learning rate, or increase epochs, ..)

First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.
Then, there are several methods and considerations:

You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use "initial" value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply "nothing change" in your architecture.
If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.
Or, you can use the concept of the selffer network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using "initial" value of hyperparameters

You can read more about the transfer learning concept (or selffer network) to get the most appropriate method for your case. There is also a paper about "incremental training on CNN" that I think is similar with the selffer network but with some modifications.
Hope it helps

"I assume you've tuned your hyperparameters", if i understood you correctly, yes, i reached this accuracy after trying different learning rates, and adding dropout layers.  (i have not change other parameters of vgg).
i tried using the previous weights into training the network again by only changing learning rate (learning rate of the new training is smaller than the previous training). but this method did not change the accuracy.


text:
Training LSTM with class imbalance
I need to train an LSTM on some light curves, in order to find a signal (there are 2 classes signal and background). However the signal (data points corresponding to signal) is around 100 times less frequent than the background so I have a huge class imbalance and in the end all points are labelled as background. I tried to use focal loss, but it doesn't help. Is there a way to make it work?


text:
Point Cloud Alignment using a Neural Network?
Having two point clouds, the second being a transformation of the first, how could I utilize a neural network in order to solve the pose (transformation in terms of x, y, z, rx, ry, rz) of the second point cloud?
Since the point clouds can be rather large (~200,000 points), I think it'd be best to first select regions from each point cloud and see if their geometries are similar (still researching the optimal method for this). If they are not similar, I'd choose two new points. If they are similar, I'd use those two regions when implementing the neural network to discern the pose.
My preliminary research has led me to believe that a Siamese neural network may work in this scenario but I'm not sure if there are better alternatives. One of my goals is to accomplish this without relying on Iterative Closest Point. Any and all insight is appreciated. Thanks.


text:
Code examples of controlling multiple units with RL
Anyone knows a resources (papers, articles and especially repositories) regarding controlling multiple units with RL.
The controlled units should not be fixed, for example in Real Time Strategy the agent builds various units (workers, soldiers ...) and later controls them. During the game various units could die and new ones are built.
I think good contemporary example is AlphaStar, while OpenAI Five controls just a single agent. This might be incorrect since I've never played those games.


text:
How to combine features with different temporal scale in machine learning
We have various types of data features with different temporal scale. For example, some of them describe the state per second while others may describe the state per day or per month from another aspect. The former features are dense on the time scale and latter features are sparse. Simply concatenate them into one feature vector seems not proper. Is there any typical method in machine learning can handle with problem ?


text:
Size of dataset for feature vector with rare event
I'm doing a model to detect duplicate in my database (there is a lot of features that can be different but mean same object in the end)
So I have my feature vector for my duplicate dataset which contains score, distance and relation between 2 identical object that I labelised
such as (0, 0, 123, 14000, 5, 10, 0, 0, -1)
Since duplicate are a rare event I was wondering what size of dataset should I use for non duplicate features, since I want my model to learn about the disparity of the multiple features I have I though I should have like 10 times the number of example of non-duplicate and in my model change the weight of duplicate class by multiplying by 10.
Is that a good thing to do or is it better to take 50%/50% of duplicate/non duplicate features for my model ?
Also, should I apply filter to chose my non-duplicate dataset in order to have like close object on some features but different on other. Or should I take them randomly among all the data I have ?


text:
Minimax combined with machine learning to determine if a path should be explored
I have an idea for a new type of AI for two-player games with alternating turns, like chess, checkers, connect four, and so on.
A little background: Traditionally engines for such games have used the minimax algorithm combined with a heuristic function when a certain depth has been reached to find the best moves. In recent days engines using reinforcement learning, etc (like AlphaZero in chess) have increased popularity, and become as strong as or stronger than the traditional minimax engines.
My approach is to combine these ideas, to some level. A minimax tree with alpha-beta pruning will be used, but instead of considering every move in a position, these moves will be evaluated with a neural net or some other machine learning method, and the moves which seem least promising will not be considered further. The more interesting moves are expanded like in traditional minimax algorithms, and the same evaluation are again done for these nodes' children.
The pros and cons are pretty obvious: By decreasing the breadth (number of moves in a position), the computation time will be reduced, which again can increase the search depth. The downside is that good moves may not be considered, if the machine learning method used to evaluate moves are not good enough.
One could of course hope that the position evaluation itself (from the neural net, etc) is good enough to pick the best move, so that no minimax is needed. However, combining the two approaches will hopefully make better results.
A big motivation for this approach is that it resembles how humans act when playing games like chess. One tends to use intuition (which will be what the neural net represents in this approach) to find moves which looks interesting. Then one will look more thoroughly at these interesting moves by calculating moves ahead. However, one does not do this for all moves, only those which seem interesting. The idea is that a computer engine can play well by using the same approach, but can of course calculate much faster than a human.
To illustrate the performance gain: The size of a minimax tree is about b^d, where b is the average number of moves possible in each position, and d is the search depth. If the neural net can reduce the size of considered moves b to half, the new complexity will be (b/2)^d. If d is 20, that means reducing the computation time by approx. 1 million.
My questions are:

Does anyone see any obvious flaws about this idea, which I might have missed?
Has it been attempted before? I have looked a bit around for information about this, but haven't found anything. Please give me some references if you know any articles about this.
Do you think the performance of such a system could compete with those of pure minimax or those using deep reinforcement learning?

Exactly how the neural net will be trained, I have not determined yet, but there should be several options possible.

This sounds similar to TD-Gammon I believe, which was written to play backgammon. You sound like you are describing a neural network that will bootstrap the values of state-action pairs and then only evaluate some top percentile of state action pairs. I think TD-Gammon 2.1 and forward really took this depth search further as well. But, as you might already know, TD-Gammon is an RL implementation with basic TD($\lambda$} and an ANN used to estimate state action values.

As one of the answers said, you are describing the AlphaZero paper. From your question you already seem to be aware of AlphaZero, and yet you ask if its ever been done before. My advice is to take a deeper look at the AlphaZero paper.

The use of of a neural network to push the search algorithm to continually only along a promising path is the same that was described in the AlphaZero paper. In AlphaZero, the NN loop contained the search function and would encourage the continued search of high probability moves that were then simulated by the same NN that now contained the Value Net. The use of alpha-beta specifically is not necessary. Just a search function aptly known as PUCT (Predictor + Upper Confidence Bounds applied to Trees)

There is a lot of related research out there.
You can look at contextual bandit problems, which is the basis for "Monte Carlo Tree Search". Here some clever bookkeeping is used to make sure that branches that looked bad but haven't been explored recently will still get explored eventually. This results in the UCT algorithm, which then got used in combination with deep learning in AlphaGo, but you can use it without deep learning too, if that makes more sense for your particular problem.
The exploration / exploitation trade-off is crucial for this kind of problem. Your original estimates of the value of a position will be very uninformed, so it should not be used to prune the search tree too aggressively. This is exactly what UTC does, with provable theoretical guarantees.


text:
What's the advantage of log_softmax over softmax?
Previously I have learned that the softmax as the output layer coupled with the log-likelihood cost function (the same as the the nll_loss in pytorch) can solve the learning slowdown problem.
However, while I am learning the pytorch mnist tutorial, I'm confused that why the combination of the log_softmax as the output layer and the nll_loss(the negative log likelihood loss) as the loss function was used (L26 and L34).
I found that when log_softmax+nll_loss was used, the test accuracy was 99%, while when softmax+nll_loss was used, the test accuracy was 97%.
I'm confused that what's the advantage of log_softmax over softmax? How can we explain the performance gap between them? Is log_softmax+nll_loss always better than softmax+nll_loss?

The short answer is yes, log_softmax + nll_loss will work better.
I don’t know the implementation details under the hood in PyTorch, but see the screenshot below from the documentation:

Yes, I know that log_softmax + nll_loss will work better, but I want to know why...

@user1024 this seems like a question for the development team since it could depend heavily on their implementation. But you mentioned you used softmax + nll_loss together. Note that since log is monotonically increasing, maximizing the log of the probability with respect to theta is the same as maximizing the probability with respect to theta. That is they both have the same argmax. But log likelihood expects log probabilities, this log_softmax is used. Also note that log(a/b) = log(a) - log(b). But since b is the same for all classes, it’s (probably) omitted in their implementation.

@user1024 omitted because it wouldn’t affect the maximization, so I assume it would make it faster and more numerically stable. But I also assume they have further optimized the routine implementation in ways that aren’t as obvious simply for speed and stability performance.

@user1024 and assuming that we are using a Gaussian prior, then maximizing the likelihood requires us to take an exponentiation, but taking the log cancels out the exponentiation, which would also make it faster and more stable.

Thank you @Hanzy, your comments helped explain the advantages of log_softmax in speed and numerical properties. As a beginner in AI, I have allodoxaphobia to determine so many super-parameters.


text:
Why overfitting is bad in DQN?
It is mentioned by Fu 2019 that overfitting might have a negative effect on training DQN. They showed that with either early stopping or experience replay this effect could be reduced. The first is reducing overfitting, the latter is increasing data.
It doesn't only have negative effects on the returns though, my test shows that it has a negative effect on value errors as well (diff. between predict V and ground truth V). I observed frequently with limited data that the training diverged almost 100% of the time (on small nets). Since increasing the amount of data could reduce the chance of divergence, I think this is an effect from overfitting.
Overfitting should mean low training loss, however, my observation is that there is a strong correlation between TD loss and value error. That is if I see a jump in TD loss, I could expect to see a jump in value error around that moment.
Or it is not overfitting because it is not really fit (i.e. high loss) but over-optimization that is for sure.
Now the question is why?
There are two points:

If it is overfitting, overfitting should have a positive effect because remembering values for all training states correctly is hardly a bad thing. (In fact, my training data is a superset of my testing data, so remembering should be fine.)
If it doesn't fit, this begs a question what over-optimization really does. It doesn't seem to fit, but it does have a negative effect. How could that be?

Is your training data really a superset of the test data? If so, that defeats the purpose of having the test data - you should never include your test data in your training set! Otherwise, you have no unbiased means of determining whether you're overfitting or not.

@NuclearWang I only use the test data to calculate value error. Anyways, overfitting or not could not be seen by looking at the value error since the optimization is done in TD loss. I expect to see low TD loss. Even in this training data, TD loss is also increasing contradicting to a general idea of overfitting.
I should point out that increasing TD loss could be a side effect of divergence.

Then what do you mean by superset? Simply memorizing the training set is usually a bad thing, since there's no generalization to unseen data. But you seem to be saying that it's not a problem because your training data points are part of the test set. Low value error on the training set doesn't necessarily indicate overfitting, that's only the case if you also see high value error on the test set. If value error on the train and test is similar then the model is not overfit, and that can happen with a both a low-error (good) and a high-error (poor) model.

@NuclearWang You seem to suggest that I cannot say the word "overfit" because I don't have a proper test data. You are right on that. I kinda assume that to be the case because I do train looooong enough. Saying it is overfitting due to loss is hard in RL with bootstrapping because I always see divergence before the end of training, not having a chance to see loss coverge to zero. I now hypothesize that maybe overfitting causes divergence. That's why overfitting is bad in RL. Not in a sense that generalization is bad in general.


text:
Is there any other rotated object detection datasets?
I have googled for a long time for rotated object detection datasets. Most of papers focused on rotated object detection using DOTA, HRSC2016 or coco text detection dataset. Some researcher also collect their own datasets but almost all of their theme is areal object detection. Is there any others dataset focus on rotated object detection?

Multiscale Rotated Bounding Box-Based Deep Learning Method
Here's a link for a reference


text:
Is it possible to use AI to find music that have a distinct tune?
Without considering lyrics, there are two type of songs: songs that have a distinct tune which you can easily remember, and songs that are the opposite. Can we design a network to identify the songs that have a distinct tune? It's very hard to find good music, I can't get a song that has a distinct tune for even 100+ songs which consumes a lot of time.

How do you define "distinct"? Unless you know what you're looking for, AI will not help you.

@Oliver Mason Yes, part of the question is about is there a model to describe distinctness/creativeness/boringness of a music. I'm not even sure if these are subjective things that varies a lot between people.


text:
Should noise (such as OU) be decreased over time in actor / critic algorithms?
In most of RL algorithms I saw, there is a coefficient that reduces actions exploration over time, to help convergence.
But in Actor-Critic, or other algorithms (A3C, DDPG, ...) used in continuous action spaces, the different implementation I saw (mainly using Ornstein Uhlenbeck process) is correlated over time, but not decreased.
The action noises are clipped into a range of [-1, 1] and are added to policies that are between [-1, 1] too. So, I don't understand how it could work in environments with hard-to-obtain rewards.
Any thought about this ?

what is stopping you from multiplying noise value with some decaying coefficient ?

I tried on some openAI environments (Pendulum and BipedalWalker) but could not get any convergence, where non-decreased OU noise made perfectly the job. I can barely understand this, so I should try a lot of different decay coefficient to be sure. May take a long time, so I was expecting this question had an already known answer

I'm not sure I don't think it's set in stone it probably depends on environment. You could try experimenting with different setups. Maybe you were decaying exploration too fast, try having it non-decayed for some fixed number of steps and then decayed slowly from that point, or maybe when you see your performance is getting better  and you are getting a lot of good episode outcomes try decreasing it from that point


text:
What are a list of board game environments for RL practice?
Recently OpenAI removes their board game environments. (It may be possible to install an older version to get access to them, but I haven’t downgraded).
Is there a list of repositories or resources of board game or similar environments that can be used to practice RL implementations? Things like checkers, chess, backgammon, or even grid world and mazes would be excellent.
A running list might be useful for many in this community.

Maintaining a list of resources would be useful, but it is not something the Stack Exchange Q&A format is good for. Some introductory material - such as textbooks on major AI topics - does end up as Q&A here (and often upvoted Q&A, which shows users appreciate it). But a minor topic like this is unlikely to attract a decent answer that will be maintained.


text:
Are self-driving cars using single frame or multiple frame to make decision?
This might be a trivial question but I couldn't find any reliable answers on the internet.
Almost all the neural network architectures for self-driving cars that I have seen on the internet have a feedforward network, previous frames will not help in making the current decision.
I have read somewhere that Tesla uses two last frames captured to make a decision, even then 2 frames will not be that useful in this case.
This might not be very helpful when predicting things ie.. lane cut-ins, as the system needs to observe the vehicle (that is going to cut in) behavior such as turn indicator, vehicle veering towards center lane over time in order to predict.
Can someone explain if this is the way production self-driving card such as Tesla work?

Or Is it something like the below?

Or are they using something like Many to one Recurrent net where inputs are CNN vectors of previous few frames and output is the control?

It varies quite substantially between different self-driving paradigms(rather obviously) but for the most part the vast majority of implementations are using a variety of different reference frames in order to make predictions.
For example, Tesla's Autopilot is being fed many different camera feeds as well as radar and ultrasonic signals that are processed in a variety of temporal contexts.
While, for the most part, all of these programs are very tight-lipped, we can make a variety of assumptions based on the information available and educated assumptions.
As with many large, complex ML/AI systems, there is a large amount of compartmentalization where many different connectionist(or sometimes classic) models are combined(à la youtube recommendation system). Tesla is likely utilizing recurrent and convolutional networks where particular modules(combinations of models) are deciding on specific contexts(temporal or signal-based). These outputs are then most likely fed into an actor network which makes real time decisions.

You say "For example, Tesla's Autopilot is being fed many different camera feeds as well as radar and ultrasonic signals that are processed in a variety of temporal contexts.". It would be nice if you could provide a reference for this.

Visualization Appreciated
The diagrams are nicely thought out. As you refine your comparative design visualizations, you might use something like Inkscape to draw them for web publication, whether or not you decide to submit a paper to a publisher or license your ideas Creative Commons.
Web Research Realities
The reliability of answers from the internet are a function of the author, how much time she or he took in researching the question themselves, what search terms are used, and where the search term is entered. This question falls under the more challenging to research, cases where the must reliable answers are in the results sets of experiments run in proprietary government and corporate research centers.
If the GM or Tesla or Toyota or DoD research results were posted on line, someone would likely be fired, sued, and possibly jailed and a team of lawyers would hunt down all references to the posted information, employing international agreements and backbone level content filters to eliminate the dissemination of secrets.
A Better Research Approach
We can determine with a fairly high confidence that a decision based on a single frame is much less likely to lead to collision avoidance, beginning with a simple thought experiment.

Two kids are playing creatively and decide to make up a game that involves a ball. One of the rules of the game is that each player can only run backward, not forward or sideways. That's what makes the same fun for these kids. It's silly because everyone wants to run forward to get to the ball. The rest of the rules of the game are not particularly relevant.
A machine driver is processing an image and goes to make the decision. In this case, the term decision indicates the use of a rules based system, whether fuzzy and probabilistic or of the former pure Boolean type, but that is not particularly relevant either. This thought experiment applies to a learned response arising from the training of an architecture built primarily of artificial networks as well.
Consider now the case where no decision or learned response is invoked on the basis of the direction of the ball travel or its proximity to the street and path of the vehicle being driven because the kids are facing away from the ball. It would not be reasonable to assume that the training data included this spontaneously made-up game. The result of the combination of child creativity and single frame selection leads to a delayed collision avoidance tactic, at best.

In contrast, if two or three frames are included in the analysis, the feature of the ball moving toward the street and the children, regardless of their orientation with respect to the ball and the street, may be detected as a feature of the entire system through which the vehicle is driving.
This is one of astronomical number of examples where the training without the temporal dimension will lead to a much higher likelihood of improper trajectory projection from pixel data on the basis of any reasonable training set than had training and use of the pixel data included the temporal dimension.
Mathematical Analysis
When the results of trials with real vehicles move from the domain of corporate intellectual property and the domain of governmental national security artifacts into the public domain, we will see our empirical evidence. Until then we can rest on theory. The above thought experiment and others like it can be represented. Consider the hypothesis of
$$ P_{\mathcal{A} = \mathcal{E}(S)} > P_{\mathcal{A} = \mathcal{E}(\vec{S})} \; \text{,}$$
where $P_c$ is the probability given condition $c$, $\mathcal{A}$ is the actuality (posteriori) and $\mathcal{E}$ is the expectation function applied to instantaneous sensory information $s$ on the left hand side of the inequality versus the recent history of instantaneous sensory information $\vec{s}$ on the right hand side.
If we bring intermediate actions (decisioning after each frame is acquired) into the scope of this question, then we may pose the hypothesis in a way that involves Markov's work on causality and prediction in chains of events.

The accuracy of decisioning based on optical acquisition in the context of collision avoidance is higher without the Markov property, in that historical optically acquired data in conjunction with new optically acquired data will produce better trajectory oriented computational collision avoidance results than without historical data.

Either of these might take considerable work to prove, but they are both likely to be provable in probabilistic terms with a fairly reasonable set of mathematical constraints placed on the system up front. We know this because the vast majority of thought experiences show an advantage to a vector of frames over a single frame for the determination of actions that are most likely to avoid a collision.
Design
As is often the case, convolution kernel use in a CNN is likely to be the best design to recognize edge, contour, reflectivity, and texture features in collide-able object detection.
Assembly of trajectories (as a somewhat ethereal internal intermediate result) and subsequent determination of beeping, steering, acceleration, breaking, or signaling is likely best handled by recurrent networks of some type, the most publicly touted being b-LSTM or GRU based networks. The attention based handling and preemption discussed in many papers regarding real time system control are among the primary candidates for eventual common use among the designs. This is because changes in focus is common during human driving operations and is even detectable in birds and insects.
The simple case is when an ant detects one of these.

A large insurmountable object
A predator
A morsel of good smelling food
Water

The mode of behavior switches, probably in conjunction with the neural pathways for sensory information, when a preemptive stimulus is detected. Humans pilot and drive aircraft and motor vehicles this way too. When you pilot or drive next, bring into consciousness what you have learned unconsciously and this preemptive detection and change of attention and focus of task will become obvious.


text:
How to identify whether images contain driver's licenses or ID cards
Suppose I have a lot of scans of hardcopy documents, in the form of jpegs. Some of them are potentially scans of driver's licenses or identification cards. I wonder what would be a good way to identify those scans that contain driver's license/ID cards.
One thought I had was training a model or use an existing pretrained model that can detect faces. However, if the data set I have has a lot of scans of photos of people, it would cause false positives. So I am not sure how I might approach this problem.
Any thought would be much appreciated!

what was your approach to solve this problem.

Why not make a detector/classifier to look for the text "Drivers Licence" or some very generic keywords related to license? As you mentioned images of people may be present in any sort of documents. Looking for text which is super specific to IDs/drivers license seem to be a better way.


text:
Is the Turing test still relevant, as of 2019?
Given the Chinese room argument, and given the development in chatbots and machine learning, isn't Turing test superseded by some other way of evaluating AI's inteligence? Would an positive result of a Turing test provide any value, besides telling that a machine is good at conversations (but possibly nothing more)?

The Turing test is a good test for AI applications like Siri and Alexa (or, in general, intelligent personal assistants), but it doesn't test a lot of features (e.g. vision) that an artificial general intelligence requires. There are several arguments against the Turing test (and it is definitely not flawless), but is it a necessary (but insufficient) test for AGI. The Turing test is still quite relevant: e.g. if IPAs passed the test, people would use them more often and they would be a lot more useful.


text:
what are the similarity measure use for both continuous and categorical data?
I have searched but found that some similarity measures are for continuous data and some are for categorical data. But i want to know the similarity measures which are use for both data, continuous and categorical?

Sometimes continuous data can be represented as parametric distribution with distribution parameters as variable, essentially continuous stochastic process. In that case cross-entropy would work on that type of continuous data.


text:
N fold Cross Validation
I want to make 8 fold cross validation from the dataset. The dataset is the musical onsets annotated which has txt format at each songs.

How to make the another folder like called splits (the name of folder) which is contains 8 fold of songs. Each fold will be written as a txt format.

I need some code references to make 8 fold like that. Do I need use sklearn Kfold? And how to save each song like stated in the third picture?
Thank you.

data science stack exchange will be a better fit for this question.

This is off topic here, but there is example code already in the documentation for sklearn's kfold: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KFold.html#sklearn.model_selection.KFold


text:
Transposed convolution as upsampling in DCGAN
I read several papers and articles where it is suggested that transposed convolution with 2 strides is better than upsampling then convolution.
However implementing such model with the transposed convolution resulted in heavy checkboard effect, where the whole generated image is just a pattern of squares and no learning takes place. How to properly implement it without totally messing up the generation? With the upsampling+convolution I got okay result but I want to improve my model. I am trying to generate images based on the CelebA dataset.
I use keras with tf and I used the following code:
model.add(Conv2DTranspose(256, 5, 2, padding='same'))

model.add(LeakyReLU(alpha=0.2))

model.add(BatchNormalization(momentum=0.9))

model.add(Conv2DTranspose(128, 5, 2, padding='same'))

model.add(LeakyReLU(alpha=0.2))

model.add(BatchNormalization(momentum=0.9))

model.add(Conv2DTranspose(64, 5, 2, padding='same'))

model.add(LeakyReLU(alpha=0.2))

model.add(BatchNormalization(momentum=0.9))

Here I try to turn a 4x4 image into a 32x32. Later it will be turned into a 64x64 image with 1 or 3 channels depending on the image. However I get the following pattern always. Some tweaking usually leads to some other pattern but it does not really change:
Checkboard effect
Thank you for your answers in advance


text:
Creating a zero element in embedding space
I have some variable length input vectors for my own use case of a 'stylistic transfer'-esque process, and I am wondering if anyone knows of a way to engineer an input that maps to a 0 element in embedding space. This would be an element that simply holds space but would be readily overlaid with vector addition of another embedded input.
My rationale is that I could pad the inputs with these zero elements to mask what I don't care about and have a semantically meaningful vector addition in the embedding space.
I wonder if I could permute some training examples with a chosen value which all map to the same output and this would allow a neural net to learn such a feature.


text:
DQN not able to learn in a game where other agents perform random walks
I am making a school project where I should develop any kind of game where I can have one reactive agent and one agent based on machine learning competing with each other.
My game consists of a salesmen problem. Basically, I have 3 types of entities, consumers, salesmen, and hotspots.
The consumers are represented by the person with a green background. There are 8 of them. They basically move around the whole game using random walks and they tend o aggregate on the HotSpots (the orange icon with the router in it).
The salesmen are represented by the person with the dark grey background. One of them is controlled by a reactive agent that has in it some rules that I programmed and the other one is controlled by my DQN model.
The salesmen have 5 available actions, move up, right, down, left or sell.
When they choose to sell the simulation will try to sell to the closest consumer in a predetermined max range. If no consumers exist in that range or if the consumer rejects to buy then the sell fails.
I started training a Deep Q Network that I built using TensorFlow. As input features, I am giving the agent current position, the position of each consumer and a boolean saying if the consumer was recently asked to buy or not (consumers that were asked to buy something will reject future offers for a determined amount of time with 100% probability).
For the output layer, I have 5 nodes, one for each available action.
Here is a screenshot of the game:
The red number on the right-bottom corner of each agent represents their total utility.

I decided to give the agents the following rewards:

SELL_SUCCESSED_REWARD = 3 - The agents receives 3 points for each success sell.
SELL_FAILED_REWARD = -0.010 - The agent loses 0.01 points for each failed sell
MOVING_REWARD = -0.001 - The agent loses 0.001 points for each move
NOT_MOVING_REWARD = -0.0125 - The agent loses 0.0125 points for standing in the same position (ie. not moving or trying to move against a wall)

I started training my agent but I seems to do not learn anything! I left it training for around 3 hours and I could not see any improvement. I tried different activation functions, batch sizes, exploration rates etc but no noticiable effect.
My question is: Can a DQN learn in this type of enviroment where there are a lot of random walks?
If yes what could be my problem? Not enought training time? Bad input features? Bad implementation?
Here are the files with my implementation of DQN:
Agent: https://github.com/daniel3303/aasma-project/blob/master/src/Agent/DeepLearningAgent/DeepLearningAgent.py
Training: https://github.com/daniel3303/aasma-project/blob/master/train_dqn.py
Thanks.

First thing that seems weird to me is your reward function. You give negative reward for moving and negative reward for not moving ? So how is the agent supposed to know what to do ? Reward function should reflect your final goal, that is to sell a lot of items, so you should give positive reward when item is sold and negative one when you failed. This may or may not help but I think it's worth to try at least

I don't think there is anything wrong with the reward scheme here. Instead I would suspect an implementation issue. Could you clarify that the grid of houses is the same throughout (as you don't represent positions that the agents cannot occupy in the state)? It may help if you give a more thorough description of your inputs/features with one or two examples, as that could be key, and I don't really want to invest in reading your code to find that out


text:
Speech to text models
I'm currently using google speech to text api to transcribe audio in real time (police scanner audio dispatches). The audio quality isn't great and I've been putting in key words to try to help train it. Is there a way to either use google, amazon aws, IBM Watson, to create a model based on past audio dispatches where I can manually type in what was said to help train it? It seems like putting in key words won't really cut it. Any other suggestions to help make it more accurate?


text:
A3C fails to solve MountainCar-v0 enviroment (implementation by OpenAi gym)
While I've been able to solve MountainCar-v0 using Deep Q learning, no matter what I try I can't solve this enviroment using policy-gradient approaches. As far as I learnt searching the web, this is a really hard enviroment to solve, mainly because the agent is given a reward only when it reaches the goal,which is a rare event. I tried to apply the so called "reward engineering", more or less substituting the reward given by the enviroment with a reward based upon the "energy" of the whole system (kinetic plus potential energy), but despite this no luck.
I ask you:

is correct to assume that MountainCar-v0 is beyond the current state of the art A3C algorithm, so that it requires some human intervention to suggest the agent the policy to follow, for example adopting reward engineering?
could anyone provide any hint about which reward function could be used, provided that reward engineering is actually needed ?

Thanks for your help.

I don't know about your first question, but I got a  basic policy gradient approach with the kinetic energy as reward working on MountainCar-v0.
You can implement it based on this blog and the notebook you find there. It uses an MLP with one hidden layer of size 128 and standard policy gradient learning.
The reward engineering boils down to replacing the reward variable with the kinetic energy $v^2$ (no potential energy and no constant factor, the reward itself is not used). I takes $>1000$ episodes to solve the environment consistently.
I'm afraid the solution is not very satisfactory and I don't have the feeling there is much to learn from it. The solution is originally for the cartpole problem and it stops working for me if I change hyperparameters/optimizer or the specifics of the reward.

Thanks for your reply, Christoph. At the very end, I was able to solve the problem using a custom reward (like the one you suggested). I think that without a reward providing the agent some "hint" about the quality of its policy, the original problem could be solved only if the agent reaches the top of the hill basically by chance.


text:
DQN ANN input vs Linear function approximator feature vector
So when using semi-gradient td(0) you need to convert your state representation into a feature vector that represents the state and as far as I know, should not be correlated.
Is the input on the ANN of a DQN the same? should it be a feature vector that represents the state? What considerations should one have when creating such vector?


text:
Would this NN for my chip outputs work?
I'm a grad student from EE.
So, basically, there's an electrical circuit that is supposed to output "0" or "1" by exactly 50 to 50 chance. It generates a number of big arrays of 0s and 1s, each of which amounts to more than 4,000 of the numbers.
But because these arrays are physically generated in a fab, I assume it might develop some dependencies among numbers and some output could be predicted by more than 50% chance. For example, due to some variations in the process, "1" can be more likely to come than "0" after a sequence of "001100".
Then let's say I make a simple deep neural network which takes 7 inputs and gives 1 output. I simply slice my array by 8 numbers, 7 of which are given to the input and the last one is used as a label (the true answer). I train my simple DNN using all these sliced numbers and it will learn some sequences. Finally, I apply my NN to a test set, and if it predicts the next number with an accuracy of more than 50%, that proves my assumption, and if it doesn't that is also good for me because it says my circuits are good.
Would it work?

It could work yes but given the described problem an RNN (Recurrent Neural Network) would probably be a better option since you would be able to process sequences with arbitrary length.

I'm afraid my desktop can't take too much load. Would it be lighter than a simple 3-layer NN?

@DuttaA if the system is purely random he will get around 50% accuracy in a big enough test set, this is independent of the NN predictions. If he gets much higher or much lesser accuracy values he can infer that the system is biased.

@Hoon predicting with an RNN is not much heavier than predicting with simple Feedforward neural network. Only the training part will be heavier for the RNN due to backpropagation through time. I recommend that you start by training a simple NN and check how it performs. If it does not work then try a more complex model. If it still does work maybe your system is purely random and there are no patters no learn.


text:
Training actor-critic algorithms in games with opponents
I am wondering how am I supposed to train a model using actor/critic algorithms in environments with opponents. I tried the followings (using A3C and DDPG):

Play against random player. I had rather good results, but not as good as expected since most interesting states cannot be reached with a random opponent.
Play against list of specific AIs. Results were excellent against those AIs, but very bad with never seen opponents
Play against itself. Seemed the best to me, but I could not get any convergence due to non-stationary environment.

Any thought or advice about this would be very welcome.

I've recently had a paper accepted at a conference (COG 2019) which would be very relevant to any answer I'd write to this question. Since it only went through peer review relatively recently, it's not publicly available yet. But I do plan to put it up on arXiv very soon (during the coming week somewhere). In case I forget to come back to this question afterwards... feel free to poke me here in a few days to remind me.

I look forward to reading this, thank you ! I am surprised of the difficulties I had to find any relevant paper on this question.

@DennisSoemers poking you on the paper!

@DukeZhou It's in my answer down below, though that link is still the arXiv version. The PDF link from the conference itself is here: http://www.ieee-cog.org/2019/papers/paper_91.pdf . We barely missed out on Best Paper Award for this paper, got second place!

Lol the Nash-Equilbrium theorem guarantees convergence. You just need to improve stability (e.g. with a greater diversity of starting agents & more compute).

The specific approaches you mentioned (A3C, DDPG), and usually also other Actor-Critic methods in general, are approaches for the standard single-agent Reinforcement Learning (RL) setting. When trying to apply such algorithms to settings that are actually multi-agent settings, it is indeed common to encounter the problems you describe. They can all be viewed as your agent "overfitting" to the opponent they're training against:

Play against random player. I had rather good results, but not as good as expected since most interesting states cannot be reached with a random opponent.

In this case, your agent will "overfit" against random opponents. It will likely become capable of beating them relatively easily, and to some extent this may also still translate to relatively weak non-random agents (like simple heuristic approaches), but it is unlikely to generalise to strong opponents.

Play against list of specific AIs. Results were excellent against those AIs, but very bad with never seen opponents

Here you pretty much exactly give a description of what I mean when I say "overfitting" against the opponent you're training against.

Play against itself. Seemed the best to me, but I could not get any convergence due to non-stationary environment.

When training using self-play like this, there is still a danger of overfitting against "itself". This may turn out to work well in some games, but in other games, there are indeed risk of instability / lack of convergence. This can, for example, be due to the agent "rotating" through a circle of strategies that beat each other. Intuitively, you could think of Rock-Paper-Scissors. A randomly-initialised agent may, for example, have a slight tendency to play Rock more often than the others. Self-play will then lead to a strategy that primarily plays Paper, to beat the current "Rock" strategy. Continued self-play training can then figure out that a tendency to play more Scissors will be strong against itself. Etc., these strategies can keep rotating forever.

The standard approach for self-play training in games is the one popularised by AlphaGo Zero and AlphaZero, and also around the same time independently published with results on the game of Hex. That last paper coined the term "Expert Iteration" for the approach, which I like to use. The basic idea of Expert Iteration is to have:

A policy that we're training (parameterised by some function approximator like a Deep Neural Net)
A tree search algorithm, like Monte Carlo tree search (MCTS).

These two components can then iteratively improve each other. The trained policy can be used to guide the search behaviour of MCTS. MCTS can be used to generate a new distribution over actions which, thanks to additional search effort, is typically expected to be a little bit better than then trained policy. That slightly better distribution is then used as a training target for the policy, using a Cross-Entropy loss. This means that the policy is trained to mimic the MCTS search behaviour, and also used to improve that same MCTS search behaviour.
This training procedure has been found to lead to strong results. One of the prevailing hypotheses is that the additional search performed by MCTS can help to stabilise training, and avoid overfitting to the "self" opponent; the search algorithm can actually "reason" about what a strong opponent (not necessarily the "self" agent) could do.

The Expert Iteration training procedure I described above often works well, but it doesn't really answer your question about training Actor-Critic approaches in games with opponents... because it's not really an Actor-Critic algorithm!
In Learning Policies from Self-Play with Policy Gradients and MCTS Value Estimates, we (I'm first author on this paper) propose an approach where the policy in self-play is trained using a training target based on the value estimates of MCTS, rather than the visit counts of MCTS (which is what we use in the standard Expert Iteration framework).
I don't know if it necessarily should exactly be called an Actor-Critic algorithm, but you could intuitively view it as an "Actor-Critic" approach where MCTS is the critic. As mentioned right before Section IV, the resulting gradient estimator also turns out to be very similar to that of the "Mean Actor Critic".

If Expert-Iteration-style solutions are not an option (for example because tree search is infeasible), I would suggest taking a look at some of the following Policy-Gradient-based approaches (and possibly later research that cites these papers):

Learning with Opponent-Learning Awareness
Policy Gradient Search: Online Planning and Expert Iteration without Search Trees

Great breakdown of common overfitting pitfalls and expert iteration!  Thanks for this.


text:
Convolutional Sequence to Sequence Learning kernel parameters
I am reading the paper Convolutional Sequence to Sequence Learning by Facebook AI researchers and having trouble to understand how the dimensions of convolutional filters work here. Please take a look at the relevant part of the paper below.

Let's say the input to the kernel X is k*d (say k=5 words of d=300 embedding dimenisonality). Therefore the input is 5*300. In a computer vision task a kernel would slide over parts of the image, in NLP you usually see kernel taking up the whole width of the input matrix. So I would expect kernel to be m*d (e.g. 3*300 - slide over 3 words and look at their whole embeddiings).
However, the kernel here is of dimensionality 2d x kd which in our hypothetical example would be 600*1500. I don't understand how this massive kernel would slide over an input that is by far lower dimensional (5*300). In computer vision you could zero-pad the input, but here zero-padding would basically turn the input matrix into mostly zeros with only a handful of meaningful numbers.
Thanks for shedding some light on it!

They are doing a matrix multiplication: consider $y = Ax, y \in \mathbb{R}^m, x \in \mathbb{R}^n, A \in M_\mathbb{R}(m,n)$. In the paper $x$ is a concatenation of $k$ elements of $\mathbb{R}^d$, so $x$ is long $kd$; $y$ is long $2d$.

First for clarity I want to differ "convolutional kernel" and "filter" here, let's say a filter has several convolutional kernels. Second in NLP 1D-convolution is mostly used, so different from CV, which typically use a $k \times k$ convolutional kernel, in NLP generally the kernel size is only $k$
Having listed above hope things could become a little clearer: If we adopt the same concepts as we seen in typical CV scenario, per my understanding, the size of convolutional kernel is only $k$, but the count of kernels, or the dimension of new "channel", is set to $2d$. One kernel maps $k \times d$ elements into one element, $2d$ kernels together convert the input to a vector of which the dimention is $2d$
As the calculation process is mapping a matrix $\mathbb{R}^{k \times d}$ to a vector $\mathbb{R}^{2d \times 1}$, if expressed in a form of matrix multiplication, or in a full-connected view, we can firstly expand the input matrix, concatenate all rows together to get a new big vector of which the dimension is $\mathbb{R}^{kd \times 1}$, then multiply a weight matrix $\boldsymbol{W} \in \mathbb{R}^{2d \times kd}$ could get the final vector
The key problem that roots the confusion is beause, in my opinion, the "kernel" imported here is a little different from the same concept used in a typical CNN. Again, if we describe in a traditional CNN terminology, the kernel size should be $k$, and there are $2d$ kernels applied.
Hope my understanding is correct.


text:
Learning Features from a Pre-trained Network
I am currently working on learning the features provided by a pre-trained network for image retrieval. Currently I take the features provided by the pre-trained network, use global max pooling to essentially provide me with a vector and then use fully connected layers to learn the feature vector. This has provided good results, although prone to over-fitting, particularly without dropout.
Is it possible/would it be beneficial to use a 1D convolutional layer instead of the fully connected layers to learn the features? Bearing in mind this is essentially still image data that has just been transformed.
model.add(GlobalAveragePooling2D(input_shape=input_shape))
model.add(Dense(256, activation="relu"))
model.add(Dropout(0.2))
model.add(Dense(256, activation="relu"))

I'm not sure how to try this practically in Keras as 1D convolutional layers only seem to accept a 3 dimensional input tensor.
Any suggestions welcome!


text:
Choice of inputs features for Snake game
I am designing a neural network using Deep Q-Learning, which teaches an agent how to play Snake (The classic Nokia game from the 90'ies). The goal of the game is to navigate the snake on a playing field (2D), and to eat a randomly placed fruit. As the Snake eats the fruit, it grows in length. The game ends if the snake hits the game border, or it self, so as the number of fruits consumed increases, so does the difficulty of navigating without hitting something.
I have trained the Snake game on a 10x10 playing field using the following inputs:

x direction of the snake
y direction of the Snake
The fraction of playing field occupied by the snake itself
A bool which says if the fruit is in front of the snake (from it's current direction)
A bool which says if the fruit is to the left of the snake (from
it's current direction)
A bool which says if collision (game over) in front of the snake is possible
A bool which says if collision to the right is possible
A bool which says if collision to the left is possible

With this choice of inputs I am getting the Snake agent to work reasonably well, and score until it plateaus out. I have examined the cases where the snake dies, and it all happens when it has no way of escaping, for example, it turns around and blocks its own path, until it finally has no where to go. This is more likely to happen as the Snake increases in lenght.
I was thinking on how I could improve this performance. It seems to me, that the reason the Snake can make a self-blocking turn is because of the inputs. Since the path it takes is clear, there's no information, that the next path is not, or that continuing further down will eventually lead to game over. If the Snake agent was aware of all obstacles in each step, i.e. the entire state-space, then I could imagine that would help train the network towards finding the optimal path without ending up blocking itself.
Since I have made the Snake game myself, I can return to the agent a matrix, or an unfolded vector, which contains the inputs for each column / row on the playing field. A blocked cell would be set to 0, a free cell 1, the fruit cell has value 0.75 and the snake head (which moves) could be assigned value 0.25. After trying this approach, I have to say I was unsuccesful. The snake ends up just turning in circles, even if I use the same reward system as the 8 input case shown above.
I am therefore trying to understand what is happening here. Am I missing information? I would think the full 10x10 state space would give me exactly enough information to lead to the correct evaluation of the next path. I would very much like to hear someone elses input to this approach.
Thanks a lot

Before we start to tweak with you Agent-Environment setup, there are couple of important things to note

Q-Learning
Q-learning as a fundamental is a greedy approach based on Action Value functions. When I say action value what I mean to say is this
$q_\pi(s,a) = p(\hat{s},r | s, a)[r + \gamma(V(\hat{s}))]$.
This means according to $\pi$ policy the Action value for taking action "$a$" at state "$s$" is the probability of taking reaching state "$\hat{s}$" and getting a reward "$r$" given the current state is "$s$" and action taken is "$a$" multiplied by sum of immediate reward "$r$" and expected future reward given by future state value(s) "$V(\hat{s})$".
Please note the $p(\hat{s},r | s, a)$ is only available in deterministic modelling technique (model based methods) and this part is not available for stochastic modelling techniques (model-free methods). For model-free methods we execute multiple experiments and take the expected value in accordance with Law of Large Numbers. Which is what we do in common Deep Reinforced Learning implementations.
The above formulation for future rewards can be expanded in series using the state-value formulation as $V(s) = \pi(a,s)q_\pi(s,a)$, where $\pi$ is the policy we are following aka probability of taking action $a$ when in state $s$.
Why is it termed Greedy?
This is because while using the fundamental of multiple experiments, with every action I take the immediate reward, and the max q_value ($q$) associated with the next_state to update my current q_value.

In Q-learning the future state reward is what you calculate by submitting the next_state to the network and taking max of the q_values attained. (Hence, Greedy!!)
Crux of the above
Having said all the above (Duh!), what is in it for us?
In Q-Learning, your reward is key for your model learn, i.e. "$r$" immediate reward and the future reward ($V(\hat{s})$)

When the state-space is limited
For example, a numerical tic-tac-toe game, in such cases it is ideal to have discrete reward system, such as the one suggested by you.
When the state-space is very large (the problem statement you are tackling)
In such cases, it is ideal to have a continuous reward mechanism, i.e. rewards should be more variable in nature, like continuous values.
Suggestions

You can define the reward function in the environment has the 10$\sqrt{2}$ - Euclidean distance between the face of your snake and the fruit it is trying to reach. The reason I am doing it (10$\sqrt{2}$ minus) is because the maximum distance could be 10$\sqrt{2}$ (considering it is 10X10 board). This would help you to have a more variable/continuous rewards for your model to learn with (since you are using neural networks for training them). As your Snake goes closer to the fruit this value will keep on increasing. One cavet, if by taking an action, your snake reaches fruit, you can provide it a specific constant (extra) reward, and if it hits its own self, or the borders you penalize it with a specific constant. So that the model understand better.
You can define your state (input vector to your NN) as 100 dimension array (10X10 grid) with each element signifying if there a object in the that area or not. 1 means it is fruit, 0 means it is snakes own body and 0.5 means it is free for movement. The idea is to limit the input values of the vectors between 0~1.  As your agent takes action, the values in the state will keep on changing.
The output of your network should have 3 neurons, each signifying the $q$ of each of action (front/right/left movement). For get_action function you will get the action pertaining to max(q_value), and keep on adding things in your memory d_que. Subsequently you can trigger model_train methods at every episode/or even at after every step.
Lastly, the hyper parameters would be the discount factor $\gamma$ you choose and the learning rate $\alpha$ you use for the NN network. That's something you will have to experiment. Along the depth of your ANN. I think a single hidden layer should be enough.
For the last layer of your NN, please use a linear activation function, and not a relu activation function.
Please run many-many-many experiments, something in millions for optimum training. There could be multiple ways of monitoring your model's learning process. How many games won, how much fruit eating it is able to achieve in 1 game etc.

Hope the above helps.

Thanks for your reply. I have already added reward functions similar to what you're suggesting, and as I mentioned, I tried with an input state vector (10x10 = 100 elements), which is similar to what you say in point 2. I will try to run for longer time, but currently, after 300.000 steps, I did not improve one bit in the reward pr game, but the loss, calculated as the sum of the errors, does not change at all. I will try to upload my code to github and provide a link later if someone is interested.

Looking forward to the code link!

You can find the code on my github repo. You can run the code from the main script (DDQN_algorithm_StackExhange.py))

I am wondering why you're suggesting to use a linear output function. I consider the problem to be a classification problem ("go straight", "go left", "go right"), and the NN to take the choice of the path with highest probability for maximizing the Q function. This post suggest the softmax function: https://stats.stackexchange.com/questions/218542/which-activation-function-for-output-layer

The ref given by you is a general suggestion for activation function for any type of NN. There is different between regression based problems and RL based problems. There are a couple of issues as to why you should not be using a softmax. It would be a little difficult to explain here, I will build markdown on github and share the link to explain this.


text:
Predict best price using neural network?
Our customer runs a tour agency. He has an excel spreadsheet containing the following information for people that have contacted them:
Customer name, country, tour duration (requested by customer), tour date, number of people in the tour (usually from 1 to 3), price given to the customer, answer: accepted/rejected (indicates if customer accepted or rejected the price given by the tour agency).
My customer wants a predictor or tool that can let him enter the details given by future customers, e.g:
Number of participants,
Tour duration,
Country (not sure if necessary?)
And the system will return the best price to charge the customer (so he won't reject the proposal but pay the maximum possible).
Another option would be that the tour agency owner will enter the price and the system will answer "Customer will accept that price" or "Customer will reject that price".
Is this even possible? I think it may be done using neural networks trained with the previous answers from customers that the tour agency owner has in his excel spreadsheet?

This is possible.... but there's no reason to use a neural network! Your best bet on a problem like this is likely to use a logistical regression for the yes/no aspect of the question and a linear regression (or combination of linear regressions) to answer the pricing question - there are also ways of simply using linear regressions and setting up cutoffs to answer the yes/no question.
The reality is that the accuracy of such a model/series of models would depend entirely on the quality and quantity of the data, but it's unlikely that in this case a neural network would provide a better result than smart usage of simpler models.


text:
How to debug and find neurons that most influenced a pixel in the output image?
I'm building CNN network of Image to Image.
After training, I have some bad results in part of the Image.
I would like to find the neurons that most influenced those pixels and do retraining only for them.
I have seen some previous works about visualizing networks, like here:
https://github.com/utkuozbulak/pytorch-cnn-visualizations
But they are only finding activations maps or visualizing with softmax in the output layer.
How can I do that for Image to Image?


text:
Drone training, how to train without training data?
I setupped a small drone simulator using PhysX, the time step is at 200 hz, while motors update like regular ESCs (at 50 Hz). I computed the inertia matrix, tweaked a bit mass of components to be real, air drag, gravity etc. After a first partial success in tuning a PID algorithm I got bored to find and hunt perfect values, and started thinking to tune it with a NN, but then I thinked, why using PID at all if NN can find better solutions?.
I started playing with NN, but then I realized I have no traning data.
Sure I could do
NN.Train(input,expectedOutput);

but what is actually the expected output? To be useful it has to be the thrust force of propellers tuned to keep input (position and orientation) stable in the desired place.
But actually I don't (Want to) know in advance the thrust that every single propeller has.
Since it is a simulation it is ok spending some computing time between each simulation step (eventually I'll try to optimize it later to fit it in a microcontroller)..
So, is it possible given a regular NN implementation where I can select number of:

Input neurons
Hidden neurons
Output neurons

find a way to train my model live? I need a way to tell the NN

hei this time you performed X keep going or go the opposite way.

It's debatable whether neural networks can find a better solution than PID, if your goal is to simply keep the output around a certain reference point PID should do a perfect job pretty much. If you really want to use "intelligent" control with NN you can look into reinforcement learning.
Few interesting papers that directly adress your problem:
Autonomous helicopter flight via reinforcement learning
An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Autonomous Helicopter Control Using Reinforcement Learning Policy Search Methods
You can also find more in the references of these papers.

Probably worth adding something that more directly answers the title question: e.g. Reinforcement Learning addresses the issue of no supervised learning training data, by generating label data (predicted utility) or gradients (changes to control policy) from observations of interactions with the environment.


text:
Divide classes into truncated and non-truncated objects
At the moment I am working on a vehicle counting & classification project.
For a specific part in the project I need to get back only the completely visible vehicles from my input data (images).  I am wondering if this could be done (more) automatically in the following way:

zoom in such that only approximately one van would be visible
divide the vehicles into two categories: truncated and non-truncated
train on these two classes
After training and testing, use the model to find the completely visible vehicles.

So the main question is, is it possible that this would give sufficient results or should I try to find another solution?

It could work. I think itll be hard to find labelled bounding box data describing full/truncated cars so this is a good idea for self labelling.
Theres probably also some other ways for you to get labelled data without the crowd sourcing or self labelling. Take a large labelled car dataset with bounding boxes (COCO has some, and if your good with aerial imagery theres alot more out there), and just enforce that any car bounding boxes that are touching the boundary are truncated, while ones fully encompassed in the imaged are not, since with high probability that will be the case.
Good luck!

Thank you for you response, good to hear that it might be possible. Do you think it will have a negative impact if I label mixed vehicles as truncated of non-truncated? (So not only cars, but also vans for example)

@JCRogier assuming your model can handle it, and theres enough training data, definitely! btw there exist open source pkgs with retinanet, yolo, faster rcnn, etc to save you some time!

Your approach should work well, you just have to test different neural networks architectures to see the most accurate one. I suggest you focus on CNNs architectures and you'll be good to go.
If you don't want to spend much time labelling your data you can only label non-truncated vehicles. Everything else is not a "non-truncated vehicle". This is even better because some images may contain persons, roads, animals... and your model will get confused if he's only trained to know truncated vehicles and non truncated vehicles. The other model will instead know how non-truncated vehicles look like and consider everything else as "not a non-truncated vehicle". And this is exactly what you need in your use-case.

Ah great, thank you! I was afraid that CNNs would automatically still recognize the objects if it was only partly visible, good to know that that is not necessarily the case and that I do not need to label both classes.

When some parts of the object are recognized this will activate some neurons but not all of them, and the overall activation will not be strong enough to classify the object as "non-truncated". This is of course after your model is trained.


text:
How to implement a neural network for Flappy Bird in Python?
I am new in the field of AI. I am working to create the flappy bird using Genetic Algorithm. After reading and seeing some examples, I saw that most implementations use a Neural Network + Genetic Algorithm and after certain generations, you achieve a very good agent that survives very long.
I currently struggling to implement the Neural Network since I have never taken a Machine Learning course.
On many examples that I have read neural networks require training inputs and outputs. For the flappy bird, I can't think of output since you don't really know if the action of flapping will benefit you or not.
In the example that I followed, Synaptic.js is used and it is pretty straight-forward. However, in Python, I can't find a simple library that will initialize randomly and adjust the weights and biases depending on the good agents that survive longer.
What would be the right way to implement this Neural Network without having a training dataset?
Is there any way to create Flappy Bird without using Neural Networks, just Genetic Algorithm?
The example in Javascript that I am referring to: Flappy Bird Using Machine Learning

What you are trying to achieve, is a game that learns to play flappy bird. For doing this you need a neural network AND a genetic algorithm, those two things work together.
About your concerns on the output, you don't have to know if the action will benefit or not, i will soon explain why.
The neural network part
So, what you need is to know how to build a neural network, i don't know your knowledge about it, but i suggest starting from the basics. In this scenario, you need a feed forward neural network, because you just take the inputs from the current flappy bird scene/frame (such as the y position of the bird, the distance from the closes pipe ecc..) and feed it through a network that outputs either 1 or 0 (jump or don't jump) in the only output neuron we just decided it has.
In python you can implement a neural network from scratch, or using a neural network framework that does al the dirty work for you.

From scratch you would need to use numpy for matrix calculations, and you would need to learn matrix multiplication, dot products and all that fancy stuff (You can just let numpy taking care of the matrix calculations, but understanding how it works behind the scenes always helps understand new problems that you might come across when doing more advanced stuff)
Using a framework like Tensorflow for python, the only thing you need to do is find the right structure for the network you want to use. You will not have to worry about how activations work, or how the feed forward is performed (But again, it's a good thing to know when working with neural nets)

The genetic algorithm part or ""learning""
I say ""learning"" because at first sight it might look like learning, but really it is not.
The genetic algorithm works like "the survival of the fittest", where the "smarter" birds, which are the ones that reached the higher score on the current generation, will have a chance to have their child little birds, that have the same brain as their parent, with either some minimal modifications, or a mix of their parent brains.
The process of this ""learning"", so the genetic algorithm, works like so:

Create a generation of let's say 200 birds, every bird has a brain with random weights, so at the first run, they are all very...not smart
The game starts, and every frame of the game, the brain of the bird recieves as input some data that is taken from the current frame ( y pos of the bird, distance from pipe...)
The brain ( neural network ) of each bird, performs a feed forward with that data, and outputs what at the beginning is a very random result, let's say 0.75 for one bird
At this point you decided that 0.75 is greater than 0.5, so you take that as a 1, which stands for "jump", while if it was 0.3, so 0, the bird does nothing and keeps falling
Shortly the bird will die cause he has no idea of what he is doing, so he most likely collides with a pipe or the ground.
After all birds met their fate, you see that some birds reached further than others, so you choose, for example, 5 of the best performing ones.
Now you try to create a new generation of 200 birds using only the brains of those 5 that were choosen, by mixing and modifying theyr brains
Now the new birds have a brand new brain, that in some cases might be better than the previous one, so chances are that some of those birds will reach a higher score, therefore flap further into the level.
Repeat from point 6

In practice your "perform_genetic_algorithm" function in python, will have to choose the birds with the highest score, and as wild as it sounds, mix their brains and modify them, hoping that some modifications will improve the performance of the bird.

I can't think of output since you don't really know if the action of flapping will benefit you or not

The mechanism above explains why you basically do not care at all about the output, except saying to the game engine: "hey the bird decided to flap, do it". Whether it's the right action or not, doesn't matter, as the smarter birds are naturally gonna get further and so be more likely to be choosen for next generation.
Hopefully now it's all more clear.

Here is some useful links for building a neural network and for understanding the genetic algorithm:

How to build a neural network: I am linking this because it contains all useful information about how to build a very basic neural network in python. In your case, you would have to ignore all the part about backpropagation, loss & error calculation and SGD, and just look at the feed forward part.
How to build a neural network - 2: This is another example of building a neural network that i found really useful, probably it's simpler and more straight forward than the previous link, but again, the backpropagation part is not needed for this genetic based learning.
Video tutorials on genetic algorithm: This is a very long but very explanatory playlist of videos that dives into the nature of genetic algorithms and how to implement one
Genetic algorithm optimization: Other source about genetic algorithms

I agree a neural net is a good start and you might want to add a constitutional neural network to list of models you want to test and evaluate.
However your question was really on getting something up and running using python. I don't have enough time for me to play games, much more let my computers do it for me... but I know someone who has done such a project.
Hopefully that helps.

I would recommend learning about Reinforcement learning first. You don't need a dataset as you train your network by letting it play the game over and over again. but knowing how to do so doea mean finding out about markov decision process and how you can use the neural network to solve this.


text:
How much Physical memory does Alpha Zero's Neural Net require?
I appreciate that there are many ways to arrange the memory in a NN, and that the numerical representations may be with bytes to floats depending on an implementation.  What is typical amount of memory required for the "application side" for the better NN programs such as Alpha Zero or an automated driving AI?  How much does it matter?


text:
Learning utility function for AIS data
I am trying to learn utility functions for ships through their AIS data.
I have a lot of data available and plan on focusing on fishing boats.
So far I've researched a lot of IRL algorithms but I'm not sure if I missed a important one that could be applied.

I've found this paper https://journals.sagepub.com/doi/citedby/10.1177/0278364917722396, but I'm not sure if this is really applicable.
I would need to transform the ais data as trajectorys, add material from openseamap and plot this to images of trajectories. Or did I misunderstand the paper completely?
My other found approach would be selecting features as position, distance to other vessels and others. And then try to apply continous max entropy deep inverse reinforcement learning.

Is there another approach that may be easier in your eyes?


text:
How to deal with Neural network input data with different length and type
I'm trying to make use of sensor data from VOC, Humidity, Age, Sampling rate and use them as NN input data. Below are the technical questions I'm struggling to find out.
For each training set, I have 2500 data points for VOC and Humidity. However, for the age and sampling rate, I have only one for each. I'm wondering if it would work to just put 5002(=2500(VOC)+ 2500(hum)+1(age) + 1 (sampling rate)) input data points to the layer.
*Note: sampling rate is in the data set because the VOC and HUM data have different data points for each training set. I reduced the data points by sampling from the original data points. However, I know for sure that timespan does matter.
Please help me out! A good reference is also a huge welcome!


text:
Action spaces for an RTS game
I think reinforcement learning would be a good fit for this problem, but I am not sure of how to deal with a seemingly infinite number of actions. In the beginning of each game (generic RTS game), the player places down units anywhere on the map. Then as the game progresses, the player can move units around by selecting on them and clicking on a valid location on the map. They must take into consideration things like distance and travel time. An AI agent must do the same.
How would I represent these actions? It’s not as simple as selecting ‘up’, ‘down’, ‘right’,...etc. Should the agent just randomly pick locations on the map?
Are there any papers or implementations I can look at to help me get started?

Randomly selecting positions on the map will be problematic with RTS because of the combinatorial explosion--although the space is not formally continuous, it is effectively continuous in terms of game tree tractability.  (Have you thought about trying to abstract a little by reducing the map to a still-intractable-but-more-manageable number of cells or regions?

(how much computing power do you have?)

You should probably check out https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/

you could have an entire 2d map represented as a coordinate system and then actions would be coordinates of where you want to send the units on the map. This is just an idea, didn't try it and don't know if it would work.


text:
Recognize pattern in dataset
I'm currently working on a group project where we need to find a pattern in a given dataset. The dataset is a collection of X, Y, Z values of a gyroscope from someone who is walking. If you plot these values you'll get a result like this.

And this is how our dataset looks like.

We are new to AI and ML so we first did some general research like understanding how matrices work and how to do some basic predictions with frameworks like TensorFlow and PyTorch. Now we want to start on this problem. What we need to do is to find a pattern in the dataset and count how many times this pattern appears in this dataset.
We started of with some none AI functions to count, we managed to do that but the way we counted will probably only work on this specific dataset. So that's why we decided to do this with AI.
We would love to hear as many different approaches as possible to count the steps since we are still learning.

Wouldn't a spectral analysis be more appropriate in this case?

For such time-series data that has a significant amount of periodicity, I would recommend converting data to the frequency domain and performing various spectral analysis methods as @firion has already mentioned. For example, you could perform Fourier Analysis and study the individual components and identify patterns there.
Also, it generally not recommended to perform the normal pattern extraction approaches to time-series data as they fail to understand the temporal relationship between subsequent data points.
Hope this helps!

since you are on ML forum, recognition of sequences does RNN.
you wond belive, i currently work on similar stuff. Start about thinking of algo to find repititions in string, like : "ababcab" returns ('ab' : 0,2,5)
and yes you could do Fourier Analysis but thats not ML method at all

You can also fit trigonometric functions like sin and cos. eg check this doc


text:
Sails size recognition
Is it possible to recognize the height and width of the sails of different kitesurfers and windsurfers taken from public webcams? And show these information on video in real time? Or on screenshots?

It is likely possible, at least within some level of accuracy (computer vision systems can be applied to all sorts of measurement problems, but are not perfect). Is that really your question, or are you looking for something more specific and technical about the subject? In general the site doesn't like to just say "yes, here's a link" to this sort of question, but to have some useful detail on the topic of AI to add. Is there any part of such a project where you are stuck or want to know in more detail before you attempt the work?

@NeilSlater thank for your reply. Yes, I'm going to scope and make a feasibility study for an App development project. We want to create an App that improve the Kitesurfers, Surfers and Windsurfers travel experience. We can access several Webcams and we would like to apply some sort of Machine Learning to send notification to Surfers setting different triggers: 1) detect surfers movement, 2) detect and determine kitesurfers sails sizes,3) detect and determine kitesurfers/surfers/wind surfers velocity.
I would need techinal specifications to complete a draft App product requirements document.


text:
Algorithm that creates new images based on other images
Are there any open sourced algorithms that can take a couple of images as an input and generate a new, similar image based on that input? Or are there any resources where I can learn to create such an algorithm?

I'm not an expert on that so you could probably get a better answer.
I'm not sure to understand what you're looking for. Are the couple of images about the same thing? Like pictures of cats and you want to generate a new cat based on these pictures? If that's what you want, you could probably take a look at Generative Adversarial Network (GAN) : Introduction.
A GAN is made up of a Generator and a Discriminator. The goal of the discriminator is to distinguish the real data from the generated data. And the goal of the generator is to improve its generated data to look similar to real data. Then, if there are different cat images in your dataset, the generator will learn to create a new cat based on that dataset.
If what you're looking for is to take different images like a cat and a dog and generate a "catdog", you can take a look at Variational AutoEncoder (VAE). For example you can train two different VAE (Encoder/Decoder). One for cats, and one for dogs. Then you take the encoder of dogs and the decoder of cats. That what I saw one day, not sure if it really works.
Correct me if I'm wrong

Thank you, that sounds like a good starting point. Actually it is about similiar images. Specifically, I want to feed the algorithm with pictures of stellar nebula and I am hoping to create new pictures of nebula by the algorithm. It's for a game project that I am working on.

It probably exists some other methods that I don't know but GAN seems to be good here. To get a better idea, you can read this article ( https://towardsdatascience.com/image-generator-drawing-cartoons-with-generative-adversarial-networks-45e814ca9b6b) and replace Homer Simpson by stellar nebula.

This was exactly what I was looking for;-) will be following!
I do have something that might be of interest to you, an algorithm that generates pictures from textual input.
Here is the link to the site where you can try it out:
https://hypnogram.xyz/
Having played with it a little it seems that the text is used to search online and the pictures found online are then merged together according to a rule set.
But that is just a wild guess.
It also seems that the images generated by the algorithm are unique and even when you use the same string twice the generated picture does not repeat itself. But again, that is what it seems like to someone who does not actually know,  I only recently stumbled on the site and have yet to read more about the process.
Enjoy!

But this doesn't allow to use an image as input ...

I'm not an expert and I'm aware this is an old question so you've probably found something of interest since.
For latecomers, an interesting lead could be ControlNET, an extension for Stable Diffusion. This extension includes various models to analyze an input image and control the output. Some of the models include detection of pose, depth, contours... It's extremely useful to create iterations of an existing image, which seems to be your objective.


text:
Any good resources for learning programming GPU level operations?
I want to be able to improve my lower level device specific programming abilities to assist in future endeavors. Examples would be learning to write custom tensorflow operations in C++ optimized to work on GPUS. Does anyone know good resources to find tutorials showing which packages to use for which devices and etc.?
An approach other than just reading source and replicating until understanding would be nice.

take a look at cuda toolkit by nvidia


text:
Binary classification for a series of data (using Keras) to tell if it is a straight line or not a straight line
I am new to machine learning and I would like to seek some advice/help for directions on implementing a binary classification for a series of data and tell if it is a straight line or a not?
for example I have the following data e.g.
Training data:

0,2,4  -- straight line
0,10,5  -- not a straight line
0,99,99  -- not a straight line
0,1,2  -- straight line

My Validation data would be:

0,60,120 -- straight line
0,120,60 -- not a straight line