text:
How many parameters are being optimised over in a simple CNN?
Okay so here's my CNN (simple example from a tutorial) along with some arithmetic to get the total number of free parameters.
We've got a dataset of 28*28 grayscale image (MNIST).

First layer is a 2D convolution using 32 3x3 kernels. Dimensionality of the output is 26x26x32 (kernel stride length was 1 and we have 32 feature maps of 26x26). Running parameter count: 288
Second layer is 2x2 MaxPool with a 2x2. Dimensionality of the output is 13x13x32 but then we flatten so we got a vector of length 5408. No extra parameters here.
Third layer is Dense. A 5408x100 matrix. Dimensionality of the output is 100. Running Parameter count: 540988
Fourth layer is Dense also. A 100x10 matrix. Dimensionality of the output is 10. Running Parameter count: 541988

Then we're supposed to do stochastic gradient descent on a 541988 parameter space!
That feels like a ridiculously big number to me. And this is meant to be the hello world problem of CNNs. Am I missing something fundamental in my understanding of how this is meant to work? Or maybe the number is correct but it's not actually a big deal for a computer to crunch?
In case it helps. Here is how the model was built in Keras:
def define_model():
    model = Sequential()
    model.add(Conv2D(32, (3,3), activation = 'relu', kernel_initializer = 'he_uniform', input_shape=(28,28,1)))
    model.add(MaxPooling2D((2,2)))
    model.add(Flatten())
    model.add(Dense(100, activation='relu', kernel_initializer='he_uniform'))
    model.add(Dense(10, activation='softmax'))
    opt = SGD(lr=0.01, momentum=0.9)
    model.compile(optimizer=opt, loss='categorical_crossentropy', metric=['accuracy'])
    return model

I do not think the number is that high, but you can always lower it by making 1 more convolution early for example

Is there something wrong in my working? Also, wouldn't adding another convolution simply increase the number of free parameters?

Depends on the kernel size, if you use a bigger one without padding no. I do not see anything wrong with it but I'm no expert. If you are worried about the number of parameters probably adding another pair Conv2D + Pool(2,2) will reduce them. Although it adds a new layer with parameters it will reduce the size of ur current third layer by a lot in terms of number of parameters.

Thanks @MiguelSaraiva. I'm not so worried about reducing the dimensionality. I was more interested in consolidating my understanding of the optimisation problem. I think it's about right. I used print(model.count_params()) and got 542230. I just need to figure out where the extra 242 parameters came from.

Neural networks can have a lot of different structures. CNNs can have a number of parameters that ranges from a few thousands to several millions.
In general you aim to increase the number of filters and reduce the first 2 dimensions, as you go deeper in the network.
So if you had Conv -> pool -> Conv -> pool -> ... , you could do for example first conv with kernel size = 5 and 8 filters and the second conv with kernel size = 5 and 16 filters. And both pools being (2,2).. But this is just an example.
In your network you start with a 28*28 image, and you use 32 3*3 filters. so number of parameters is (3*3 + 1) * 32 = 320.
In the dense layer you have as input a 13*13*32 and use a 100 FC layer. so n_parameters is (13*13*32 + 1)*100 which is 540900.
Then you get (100+1) * 10 FC which is 1010 more.
total = 320 + 540900 + 1000 which is 542230, as expected.
The +1 that shows up in every layer is the Bias neuron. Basically you add a bias neuron per output in a connection between 2 layers. In the FC1000 to the FC10 it is easy to understand, you have a bias per output neuron. In the Convolutional layers you have a bias term per each of the filters applied, so for each filter you have the filters weights plus 1 for the bias.
Apart from that you also had a small math mistake when adding the 288 at the beginning, you added 188. So your missing 242 parameters were: 32 from the conv layer, 100 from the first dense, 10 from the second dense and 100 from the sum. 100+100+32+10 = 242.


text:
How can we get a differentiable neural network to count things?
Imagine I have images with apples in them. I want to train a neural network which can count the number of apples in each image.
BUT, I don't want to use a detector, then count the number of bounding boxes. I'm interested in knowing if there's a way to bake this sort of logic into a differentiable neural network.
The simplest variation of this problem might be: I have an input vector $x \in \{0, 1\}^N$ and I want to count the number of 1s in it. I can make a single layer neural network, setting all the weights to 1, bias to 0, and linear activation. And my answer pops out. But how would I train the network to do this from scratch? Sure, I could regress to an output in $[0,1]$ and multiply the result by $N$, then the network is differentiable. But did the model really learn how to count? If so, would this behaviour be generalisable to counting multiple types of objects at once? Would it generalise to inputs where there can be any number of said object (like an image can have many apples in it, despite the size of the image)?
I want to know if there's a model which can learn how to count.
Here's another way I'm thinking about it: I can look at an aerial view of pine trees and say "yeah maybe there are 30 trees", but then I can do the task of looking at and identifying each tree individually, incrementing a counter, and making sure not to revisit that tree. The latter is what I consider truly "counting".

Could you clarify what you mean by "really learn to count"? You seem to want to make the distinction between estimating a total and counting

@NeilSlater you ask a good question. Maybe one way is to say it's certainly not like "estimating" the total. I can look at an aerial view of pine trees and say yeah maybe that's 30, but then I can do the task of looking at and identifying each tree individually, incrementing a counter, and making sure not to revisit that tree. Something feels very different about those tasks and I can't put my finger on formalising the difference. I also can't formulate why I believe a NN is not equipped to do the "true" counting task.

If you could [edit] some variant of your comment into the question, I believe I can answer that part. I am not confident that I can answer definitely whether there is a model that "learns to count". Whether it can be found depends on how much of the process you expect it to learn (as opposed to be given by the problem forumulation), and how you would present the learning task to it. Counting is a surprisingly complex thing when you start to think about it in detail

It certainly is. That statement is touching on where I'm coming from.

I've been looking for a network to do that type of counting too. Have you seen this?
https://stanford.edu/~jlmcc/papers/SabathielMcCSolstad20LearningToCountMultiModalEnvironment.pdf

@numiri I haven't seen it but thanks for sharing

Very interesting problem. Maybe you can find or create some labeled gaming data set with moving objects. And if you will, please let me know;).

Estimating from an observation is a function, but "really counting" is a process. Feed-forward neural networks can learn arbitrary functions from training examples, but they cannot represent (and therefore cannot learn) processes. They can attempt to estimate the results of completing a process as a function, but that is not the same thing as actually performing the process.
To learn arbitrary processes from examples requires a model with some concept of state and evolution over time in addition to any functions that are required. Recurrent neural networks (RNNs) are suitable models for those kinds of learning problems, but so are other AI learning constructs.
This rabbit hole goes deep, and for any model you could build it is possible to ask:

Is the system really performing a counting task similar to how a human might attempt the same task?

Has the sytem really learned to count, or has it been constructed by the developer so that some parts of the process are inevitable?

Has the system learned to count in general within any sub-component, or are the things that it can count tightly coupled across components?

There are other questions you could ask too, depending on your goals for producing such a model.
It is worth noting that for small quantities and certain patterns, that human "counting" does more closely resemble the estimation process, or perhaps is more akin to an NLP problem. For example, consider how you "count" the pips on a die - although you can use a counting process to confirm what you see, typically you do not do so. Instead, some part of your brain is supplying a very accurate answer quickly and unconsciously:

More generally, counting in humans likely consists of multiple related strategies for converting observations into symbolic and/or sensory representations of quantity. Some may be instinct, some are learned conscious behaviours and some appear to be in-between, perhaps originally learned but turned into subconscious skill through repetition.
Ignoring whether or not you are implementing your AI project for counting items in an image wholly within a neural network for now, I think you need the following components:

A state that represents and tracks the start, progress so far and end of the counting process with respect to the input being processed.

An accumulator that represents quantity counted so far.

A detector that can trigger a counting event when it observes something that needs to be counted.

A strategy or planner for processing input so that detection events are separated and only triggered once for each valid event.

The last component can vary a lot, and humans will use a range of different strategies for counting depending on the difficulty of the task. For instance, you might use working memory (a limited resource in humans) to track a few key points in an image to help segment an image into smaller sections as you work across it. Or you might make visual marks on an image to track each object that had already been counted. For a human, counting strategies can be mixed and matched, and switched between sometimes even during the same counting task.
All of these components could be represented in learnable parts of an AI process, but they do not have to be. When you suggest in your question that

I don't want to use a detector, then count the number of bounding boxes.

then you are most likely saying that you don't want to use a fixed hard-coded strategy. You want to somehow create a neural network that can discover at least one strategy for processing the input.
The trouble you will face is that giving a large RNN model (e.g. LSTM) a dataset of examples with input images and output correct counts will likely be too much of a challenge. Discovery of a robust object counting system from scratch will be too hard.
There are a few things that may help you construct something that "really" learns to count. Here are a couple of ideas:

Curriculum learning, where you start training the neural network with some hand-holding examples, and slowly ramp up the complexity. This is analogous to how we teach children to count.

Designed-in modelling for processing strategy. For instance, you could add a virtual fovea and artificial visual saccades and require that the neural network output a sequence of locations within the image that it picks out to run the detector against. This will be a constraint that allows certain types of human-like counting strategy to work, and could simplify the problem to the point that the network has a chance to learn it.

A paper that uses a "fovea" model for object detection: Object detection through search with a foveated visual system

Thanks Neil. I didn't want to anchor answers, but the reason I thought about this question was because of this Kaggle competition. One component of it is counting the number of different atom types in a diagram of a molecule, and then it gets more complicated because each atom needs a unique integer label and then some sort of graph representation to be written down. Top teams are getting very impressive results and it's mostly with a image encoder, sequence decoder architecture (you mentioned LSTM).

People appear to be using transformers instead of LSTMs though. I just couldn't help but wonder if the network was learning the concept of counting, labelling, and writing down the graphical representation, or just learning what the output string should be without going through those intermediate logical steps.

@AlexanderSoare I do not know enough about those kinds of models, but suspect they do fit at least in general terms with this answer. I would also suspect that there will be careful use of designed-in constraints when it comes to handling the image-to-sequence part of the challenge, making it feasible in the first place and more accurate at the single task in the competition.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Unbalanced dataset in regression rather than classification
Assume that we have a labeled dataset with inputs and outputs, where the output range is $\left[0, 2\right]$, but the majority of outputs is in $\left[0, 1\right]$. Should one adopt some kind of over- or undersampling approach after compartmentalising the output space to make the dataset more balanced? That would usually be done in classification, but does it apply to regression problems, too?
Thanks in advance!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What does AI software look like, and how is it different from other software?
What does AI software look like? What is the major difference between AI software and other software?

Code in AI is not in principle different from any other computer code. After all, you encode algorithms in a way that computers can process them. Having said that, there are a few points where your typical "AI Code" might be different:

A lot of (especially early) AI code was more research based and exploratory, so certain programming languages were favoured that were not mainstream for, say, business applications. For example, much work in early AI has been coded in Lisp, and probably not much in Fortran or Cobol, which were more suited to engineering or business. Special languages were developed to make it easy to program with symbols and logic (eg Prolog).

The emphasis was more on algorithms than clever/complex programming. If you look at the source code for ELIZA (there are multiple implementations in many different languages), it's really very simple.

Before the advent of neural networks and (statistical) machine learning, most AI programming was symbolic, so there hasn't been much emphasis on numerical computing. This changed as probabilities and fuzziness were increasingly used, but even if using general purpose languages there would be fewer numerical calculations.

Self-modifying code is inherently complex; while eg Lisp made no difference between code and data (at least not in the same way as eg C or Pascal), this would just complicate development without much gain. Perhaps in the early days this was necessary when computers had precious little memory and power and you had to work around those constraints. But these days I don't think anybody would use such techniques anymore.

As modern programming languages evolved, Lisp and Prolog (which were the dominant AI languages until probably 20 to 30 years ago) have been slowly replaced by eg Python; probably because it is easier to find programmers comfortable in an imperative paradigm rather than a functional one. In general, interpreted languages would be preferred over compiled ones due to speed of development, unless performance is important.

The move to deep learning has of course shifted this a lot. Now the core processing is all numeric, so you would want languages that are better with calculations than symbol handling. Interpreted languages would now mainly make up the 'glue' code to interface between compiled modules, and be used for data pre-processing. So current AI code is probably not really that different from code used in scientific computing these days.
There is of course still a difference between R&D and production code. You might explore a subject using an interpreted language, and then re-code your algorithm for production in a compiled language to gain better performance. This depends on how established the area is; there will for example be ready-made libraries available for neural networks or genetic algorithms which are well-established algorithms (where performance matters).
In conclusion: I don't think AI code is any more complex than any other code. Of course, that's not very exciting to portray in a film, so artistic licence is used to make it more interesting. I guess self-modifying code also enables the machines to develop their own conscience and take over the world, which is even more gripping as a story element. However, given that a lot of behaviour is nowadays in the (training/model/configuration) data rather than the algorithm, this might even be more straight forward to modify.
Note: this is a fairly simplified summary based on my own experience of working in AI; other people's views might vary, without either being 'wrong'.
Update 2021: I now work at a company that extracts business information/events from news data on a large scale using NLP methods. And we're using Lisp... so it's still in active, commercial use in AI.

Oliver Mason's answer is quite good, but I think it can be expanded upon a bit.
I think there are extra factors that could be popularly interpreted as making AI code difficult to read (as compared to other code):

AI code actually is more complex than most code that is written. When we work in AI, we often lose sight of this, but most code ever written does one of two things: turn data in one standard format into data in another standard format; display something to a user. Both of those are conceptually easy to understand. Neither of them is likely to require knowledge of mathematics. This is very unlike most code written in AI, where understanding what was written, and why, requires extensive knowledge beyond the knowledge needed to read and write computer programs. So, reading AI code requires more knowledge of mathematics or of complicated AI-focused Algorithms.
The "programs written by AIs" are really our models in the modern context. Our algorithms "program" a template model to make it work for a specific application. This is especially true if you think of it in the senses in which programming is also used in "linear programming", "quadratic programming", and even "dynamic programming". Our models really are hard to understand. Often even their creators cannot explain or characterize the model's behavior on specific inputs without running the model. The reason for this is that our models do not represent simple enough concepts that humans can easily understand or simplify them.
Self-modifying code is rare, but does exist within AI. However, as with other AI-generated models, AI-generated code tends to be comparatively difficult for humans to interpret, because (unlike most human-generated code), it is not written with the intention that humans are going to try to read and understand it. There actually are some efforts to generate code that conforms to human styles, but usually the code that is generated does not work well.

Regarding your first point, I'd say it requires more knowledge of AI, not necessarily mathematics. It has shifted towards the latter more recently, but a lot of AI programming in the NLP area did/does not need a lot of maths.

This may be a much simpler explanation than you're looking for, but in Machine Learning Zero to Hero, Google engineer Laurence Moroney summarized it in a way that I thought was brilliant. Paraphrasing from a presentation slide:

In traditional programming, you input rules and data and the program outputs answers. In machine learning, you input data and answers and the program outputs rules.

There's an algebra-like symmetry to this. And the program doesn't even know what it's coming up with rules for. It just randomly evolves the rules until the data produces the correct answers. You can then take those rules, apply them to different data, and hopefully get correct answers.

AI has been redefined recently to machine learning.
All programming except machine learning (and we'll come back to this) is embodying human knowledge in terms a computer can follow.
EG A text editor has user interface rules, user expectations, a contract with the OS that it has to follow. A programmer puts it all together. This applies to text editors, expert medical systems, banking software, accounting software (and the programmer needs to know accounting to program it).
Machine learning is training software with data and outputs allowing it to determine the link between them. No human knowledge. Nor can it explain what it is doing.
Of course they actually work far better when human knowledge surrounds them as part of their data. A AI that routes incoming invoices etc works better when told where things should actually go (accounts payable).

I don't think many people will agree with your first statement.


text:
What does a consistent heuristic become if an edge is removed in A*?
For the A* algorithm, a consistent heuristic is defined as follows:
if, for every node $n$ and every successor $m$ of $n$ generated by any action $a$, $h(n) \leq c(n,m) + h(m)$.
Suppose the edge $c(n,m)$ is removed, how do we define consistency for nodes $n, n'$ ? Since the $n'$ is not generated from $n$.

Consistency is a property of heuristics. You can think of consistency as the common sense idea that our guess at the time to go from $A \rightarrow B \rightarrow C$ cannot be more than the time to go from $A \rightarrow B$, plus our guess of the time to go from $B \rightarrow C$.
Supposing we remove a given edge $c(n,m)$ from our graph, but that our heuristic function gives the same value for $h(n)$ as before. Can the function be inconsistent? Let's try a proof sketch:

Since the function was consistent before we removed this edge, it must already be the case that $h(n) < c(n, m') + h(m')$ for every other node $m'$ adjacent to $n$.
Removing the edge $c(n,m)$ doesn't change these relationships, because the other edges still have the same costs as before, and the heuristic function still gives the same value for every node.
The same argument may be applied to node $m$.
Since none of these relationships have changed, $h$ is still consistent.

if c(n,m) edge is removed how would you know that h(n) < c(n,m) + h(m) ? since we don't know how long it takes to go from n to m ? I understand intuitively why h is still consistent but how would the inequality for consistency apply for node n and m ? does c(n,m) need to be a direct edge ?

@calveeen It ends up that if the heuristic is locally consistent (between all neighbors of states) then it is also globally consistent (where n and m are not directly connected and c(n, m) represents the cost of the shortest path between them). This can be proven by induction. Removing the edge between n and m can only make c(n, m) larger, in which case the inequality still holds.


text:
One Softmax or two separate logistic regressions for the task of classifying pictures as a/b and c/d
Simply put, the question 11 in chapter 4 of Aurélien Géron's book "Hands-on Machine Learning" asks:

Suppose you want to classify pictures as outdoor/indoor and daytime/nighttime. Should you implement two logistic regression classifiers or one softmax regression classifier?

which he gives the answer in the jupyter notebook that accompanies the book (the given answer quoted below is exactly how he gave in the book and is complete):

If you want to classify pictures as outdoor/indoor and daytime/nighttime, since these are not exclusive classes (i.e., all four combinations are possible) you should train two Logistic Regression classifiers.

And I am not sure if I got it. My answer was direct: one softmax!
I was surprised the the answer was different from that and I don't know if I understand it.
What I thought:
Isn't the idea of softmax regression: multiclass classification? When he says "all four combinations are possible" doesn't he means outdoor daytime, outdoor nighttime, indoor daytime, and indoor nighttime? If those four classes are possible and I want to classify pictures as one between those four, this is the task I expect to be more appropriate for a Softmax regression than two separate logistic regressions.
More than that, I think that Softmax regression allows for joint modeling instead of treating each classification task independently, which would be the case of using two logistic regressions. Considering all classes together and learning the relationships between the features can capture dependencies between then, right?
About the parameter estimation: isn't estimating a single set of parameters that map inputs to probabilities of each class more efficient than training two separate models?
Generalization: isn't sofmatx possibly better to generalize to unseen combinations of outdoor/indoor and daytime/nighttime since it learns a unified decision boundary?
Am I wrong? There are more things to consider or two logistic regressions separate will be always better than one softmax model?

Softmax gives you a probability distribution, thus its output are positive and sum to 1... if you want "outdoor" AND "daytime", you want a $1$ on the "outdoor" neuron and a $1$ on the "daytime" neuron... clearly, that's not possible if you use a single softmax, as the sum should add up to 2
What you can do, is to create a cartesian product of your output, and thus having an output like:

neuron 1: outdoor and daytime
neuron 2: outdoor and night time
neuron 3: indoor and daytime
neuron 4: indoor and night time

However, this solution works only because you are in a very small example, if instead you have 10 options for the first classification, and 10 for the second one, in the case of 2 softmax, you would have 20 neurons, instead with this "joint" representation, you would need 100...

Ok, I got it. So, one logit to classify as indoor OR outdoor and another one to classify as daytime OR nightime. At first, I thought that would be more effective to look at the whole picture and classify with the x AND y, but I got it now. Thanks.

It is possible to have an output for every combination of independent classes.  As another poster has noted this scales poorly.  It is also possible to have a network which diverges into two sets of outputs, each softmax-ed among themselves.  This reduces some overhead compared to two separate networks.
The question about whether either will generalize well is born out by the data.  The first few hidden layers may perform some transformation which makes  indoor/outdoor and daytime/nighttime separable and only decision boundaries need to be drawn by the final weights/activations.  This is the best case scenario.  The worst case scenario is that there are no common transformations which separate all of these classes and the divergent network begins at the input layer at which case one may as well have two independent networks.


text:
input annotations quality check for large scale image data
while dealing with image data at very large scale, there are different sources where data is coming from. Often, we do not have any control over quality of labels/ annotations. I already do use sampling quality checks method to manually check the quality of annotations but as the volume of data has increased,even sampling QC become an inefficient job. Are there other methods to automate / simplify this task for data at large scale?


text:
Would AlphaGo Zero become perfect with enough training time?
Would AlphaGo Zero become theoretically perfect with enough training time? If not, what would be the limiting factor?
(By perfect, I mean it always wins the game if possible, even against another perfect opponent.)

We cannot tell with certainty whether AlphaGo Zero would become perfect with enough training time. This is because none of the parts (Neural Network) that would benefit from infinite training time (= a nice approximation of "enough" training time) are guaranteed to ever converge to a perfect solution.
The main limiting factor is that we do not know whether the Neural Network used is big enough. Sure, it's pretty big, it has a lot of capacity... but is that enough? Imagine if they had used a tiny Neural Network (for example just a single hidden layer with a very low number of nodes, like 2 hidden nodes). Such a network certainly wouldn't have enough capacity to ever learn a truly, perfectly optimal policy. With a bigger network it becomes more plausible that it may have sufficient capacity, but we still cannot tell for sure.

Note that AlphaGo Zero does not just involve a trained part; it also has a Monte-Carlo Tree Search component. After running through the Neural Network to generate an initial policy (which in practice turns out to already often be extremely good, but we cannot tell for certain if it's perfect), it does run some MCTS simulations during it's "thinking time" to refine that policy.
MCTS doesn't benefit* from increased training time, but it does benefit from increased thinking time (i.e. processing time per turn during the actual game being played, rather than offline training time / self-play time before the evaluation game). In the most common implementation of MCTS (UCT, using the UCB1 equation in the Selection Phase), we can prove that it does in fact learn to play truly perfectly if it is given an infinite amount of thinking time. Now, AlphaGo Zero does use a slightly different implementation of the Selection Phase (which involves the policy generated by the trained Neural Network as prior probabilities), so without a formal analysis I can't tell for sure whether that proof still holds up here. Intuitively it looks like it still should hold up fine though.

*Note: I wrote above that "MCTS doesn't benefit from increased training time. In practice, of course it does tend to benefit from increased training time because that tends to result in a better Network, which tends to result in better decisions during the Selection Phase and better evaluations of later game states in the tree. What I mean is that MCTS is not theoretically guaranteed to always keep benefitting from increases in training time as we tend to infinity, precisely because that's also where we don't have theoretical guarantees that the Neural Network itself will forever keep improving.

Assuming you mean a mathematically perfect player, similar to what we can achieve trivially in Tic Tac Toe, then the answer is "maybe". The underlying reinforcement learning algorithms that it uses do have some convergence guarantees, but there are some caveats:

Theories of convergence that apply to value and policy functions learned by RL assume unrealistic timescales and decays of learning parameters. If you have actual infinite time and resources then it is possible to explore all board states and learn their values accurately. But then, if you have those resources at your disposal, a brute-force search would work too.
Using neural network approximations to true value functions can put bounds on how well value functions are learned, as they rely on generalisation, and are characterised by an error metric. The values that they calculate are in fact guaranteed to not be mathematically perfect, and in part that is by design (because you want learning from similar states to apply to new unseen states, and as part of that need to accept the compromise that most state values will be slightly incorrect). This is especially true of the fast policy network used to drive Monte Carlo Tree Search (MCTS).
Running longer MCTS during play improves performance at cost of spending more time per move decision. Given infinite resources, MCTS can play perfectly, even from very crude heuristics.

The difference between AlphaGo Zero and a more traditional game search algorithm is to do with optimal use of available computing resources, as set by available hardware, training time and decision time when playing. It is orders of magnitude more effective to use the RL self-play approach combined with the focused MCTS as in AlphaGo Zero, than any basic search. At least for similar puzzles and games, it is more efficient than any other game playing technique that has been explored. We are likely still more orders of magnitude of effort away from a perfect Go player, and there is no reason yet to suspect that this will ever become practical, and Go become "solved".

Yes AlphaGo Zero could become undeniably perfect.
It has won 100:0 against AlphaGo Lee (which won 4:1 against 18-time world champion (human) Lee Sedol) and 89:11 against AlphaGo Master (which won 60 straight online games against human professional Go players from 29 December 2016 to 4 January 2017).
From the official AlphaGo website:

"AlphaGo's 4-1 victory in Seoul, South Korea, in March 2016 was watched by over 200 million people worldwide. It was a landmark achievement that experts agreed was a decade ahead of its time, and earned AlphaGo a 9 dan professional ranking (the highest certification) - the first time a computer Go player had ever received the accolade.".

From AlphaGo's webpage: "AlphaGo's next move":

"We plan to publish one final academic paper later this year that will detail the extensive set of improvements we made to the algorithms’ efficiency and potential to be generalised across a broader set of problems. Just like our first AlphaGo paper, we hope that other developers will pick up the baton, and use these new advances to build their own set of strong Go programs.
...
Since our match with Lee Sedol, AlphaGo has become its own teacher, playing millions of high level training games against itself to continually improve.".

Could AlphaGo be generalised to solve real life problems?


text:
Should Q values be changing within an epoch/episode or should they change after one episode/epoch?
I am trying to use Deep-Q learning environment to learn Super Mario Bros. The implementation is on Github.
I have a neural network that Q values update within an episode for a very small learning rate (0.00005). However, even if I increase the learning rate to 0.00025, the Q values do not change within an episode as they are predicting the same Q values regardless of what state it is in. For example, if Mario moves right, the Q value is the same. When I start a new episode, the Q values change though.
I think that the Q values should be changing within an episode as the game should be seeing different parts and taking different actions. Why don't I observe this?

Learning rate should only have an effect on the magnitude of updates to your Q-values, it should not influence when updates are triggered. What makes you think that it does? If you are certain that you have correctly observed this phenomenon, where did you get the code from that does this?

@DennisSoemers if I had a larger learning rate, would the magnitude of updates to my Q-Values be smaller or bigger? As in my own code, the higher learning rate results in the Q-values not updating or staying the same. I've done something wrong, was just trying to find the root of it

@NeilSlater the environment is Super Mario Bros. I am using Deep-Q learning

@rtz Typically, we compute some sort of error (something like target_output - current_output), multiply that with the learning rate, and then update accordingly (that's the basic idea). So, a larger learning rate should result in larger updates. A learning rate of $0$ would mean no updating whatsoever. All of this is of course you mean the same thing as I do when you say "learning rate"... though I don't think it's really an ambiguous term generally

@DennisSoemers definitely talking about the same thing and that's what I'm doing but unsure what could be going wrong. If you were really interested, I could send you the code but that definitely helps. Thank you

@rtz You can put the code somewhere and I'll probably have time somewhere today to have a quick look through it (though not right now). Inspecting a DQN implementation line-by-line would probably be a lot of work, but can at least see if I happen to notice something in a quick glance.

@DennisSoemers if you don't have the time it's okay too. Thank you very much!
https://github.com/rtang23/FYP/blob/5abfccf3e86742b8487080493789e13b6f812388/training2.py

Can you edit the question to elaborate in more detail exactly what issue you are observing? What do you mean when you say that "Q values do not change within an episode"? Do you mean that the network's parameters are not being updated? Or do you mean that it keeps predicting the same values regardless of what state it's in (or is it maybe stuck in the same state)? Exactly what is it that you observe?

@DennisSoemers I am thinking the learning rate would not affect what state it is in. However, that does seem to be the case. I also checked with my code if it stayed in the same state when this happened and this was not the case.

I didn't really see anything strange in your code. I can say certainly that modifying the learning rate should not have the kinds of effects you're describing. That said, it's still not 100% clear exactly what you are observing though, my bet is on a misunderstanding/misinterpretation there. Before I can try to find an explanation for what you observe, I really need to know every single tiny detail of exactly what it is you are observing.

@DennisSoemers sorry for the delayed reply. So what is happening is when I use a learning rate of 0.00025, the Q values throughout the episode stay the same. I.e. if Mario is at the start of the screen and moves to the right, the Q values are not being updated as I am printing them constantly in a test episode. I hope that clears it up

I was attaining negative values out of my convolutional layers and then using relu on them resulted in the gradient of the activation being 0. Hence, my Q values were not being updated. I've since updated my activations to be ELU. Thanks for the help.


text:
Is text preprocessing really all that necessary for NLP?
As a first step in many NLP courses, we learn about text preprocessing. The steps include lemmatization, removal of rare words, correcting typos etc. But I am not so sure about the actual effectiveness of doing such a step; in particular, if we are learning a neural network for a downstream task, it seems like modern state of the art (BERT, GPT-2) just take essentially raw input.
For instance, this ACL paper seems to show that the result of text preprocessing is mixed, to say the least.
So is text preprocessing really all that necessary for NLP? In particular, I want to contrast/compare against vision and tabular data, where I have empirically found that standardization usually actually does help. Feel free to share your personal experiences/what use cases where text preprocessing helps!

It all depends on the quality of data. Due to old rule "Garbage in, garbage out" link
, if you have bad quality data(data redundancy, unstructured data, too much memory, etc) your results won't be spectacular.
In other cases, everybody could be a Data Scientist, because its only task was "put raw text into classifier". Also, you should remember that BERT or GPT-2 it's deep learning algorithms so they not need too much processing. Using preprocessing in machine learning is more that needed(prediction of sentiment for example).
Shortly, preprocessing is optional, but highly advisable.

Depends, I agree in some tasks it can help. But I don't think in general we should think about raw data as "garbage' necessarily.

It depends on the dataset we have and algorithm we use, usually text preprocessing can help your model perform better. But, some preprocessing method can have no significant impact on accuracy. We need to choose which preprocessing method that can help us make better quality dataset to give to the model.


text:
Snake game: snake converges to going in the same direction every time
This is a q-learning snake using a neural network as a q function aproximator and I'm losing my mind here the current model it's worst than the initial one.
The current model uses a 32x32x32 MLPRegressor from scikit-learn using relu as activation function and the adam solver.
The reward function is like following:

death reward = -100.0
alive reward = -10.0
apple reward = 100.0

The features extracted from each state are the following:

what is in front of the snake's head(apple, empty, snake)
what is in the left of the snake's head
what is in the right of the snake's head
euclidian distance between head and apple
the direction from head to the apple measured in radians
length of the snake

One episode consists of the snake playing until it dies, I'm also using in training a probability epsilon that represent the probability that the snake will take a random action if this isn't satisfied the snake will take the action for which the neural network gives the biggest score, this epsilon probability gradually decrements after each iteration.
The episode is learned by the regressor in reverse order one statet-action at a time.
However the neural network fails too aproximate the q function, no matter how many iterations the snake takes the same action for any state.
Things I tried:

changing the structure of the neural network
changing the reward function
changing the features extracted, I even tried passing the whole map to the network

Code (python): https://pastebin.com/57qLbjQZ

Can you provide a snippet of your code.

the pastebin link to the code its in the post

@orenrevenge I've updated my answer to address the code you posted too.

There are two problems here.

The code you posted doesn't incrimentally train your multilayer perceptron. Instead, it effectively re-randomizes the weights, and then re-fits the model each time you call .fit() at lines 35 & 54. Using SKLearn's _fit() function with Incremental=true might solve this, or you can package up the data into a larger batch, and train on that offline instead after several episodes.
Your reward function makes it painful to be alive, and doesn't give enough benefits through the Apples to make up for this. There are 100 squares that could contain the apple. On average, the apple will spawn about 5 squares from the snake in each direction. Since the snake can't move diagonally, that's 10 moves (5 left/right, 5 up/down). That means that if the snake plays perfectly, then on average, it might be able to get zero reward total. In practice, the snake will not play perfectly. This means living gives negative expected reward.

In contrast, if the snake can kill itself, it will stop getting negative rewards. The reward function you've used is maximized by getting big enough to run into your own tail as fast as possible. The snake should be able to do this after eating 3 apples I think. There is some incentive to hunt for food well, but not much compared with hitting your own tail as soon as possible.
If you want the snake to learn to hunt for the food, reduce the penalty for being alive to -1, or even -0.1. The snake will be much more responsive to signals from the food.

there are no walls in the game, the moment the snake goes on the edge of the map it reappears on the other end so basically it learns to never die which seems a little contradictory since he gets negative rewards for being alive

@orenrevenge, you say that an episode lasts until the snake dies. How can it die? Only by hitting itself?

yes thats correct

@orenrevenge  This is really more of an implementation question I think, and you might have better luck on the main StackOverflow site. Looking at your code, I think your mistake is in fit_episode, on lines 35 & 53. MLPRegressor.fit() promises to produce a trained model for the data you give it, but makes no promises about  preserving the weights learned in past fits. See the SKLearn sourcecode: https://github.com/scikit-learn/scikit-learn/blob/f0ab589f/sklearn/neural_network/multilayer_perceptron.py#L604 By default, the fit is not incremental. Using _fit with incremental=true might fix this.

@orenrevenge Even after you fix that though, the reward function you've picked is optimized by having the snake try to kill itself as quickly as possible. The rest of my answer still applies.

that's it, the scikit-learn fit method was the problem I had to use partial_fit instead thanks a lot now I seem to get some results, the snake finally learns to go after the apple, I still need to modify the reward function for better performance according to your suggestions

Assume you are the snake.
In front of you is empty. Left of you is empty. Right of you is empty. The distance to the apple is 4. The apple straight in front of you. Your length is 20.
Can you make a good decision with this input? In which direction would you go to achieve maximum score?
From the given input, you could go straight forward to the apple. But that might be a failure and lead to death.
IMHO, the input state is simply not enough to make a good decision, because
a) the snake doesn't even know in which direction it's currently moving.
b) the snake has no idea about where its body is
The situation could look like this:

The only way for the snake to move out of this trap is as indicated by the arrow, so that the tail frees the way out just in time. Your neural network does not have the necessary input to make that decision.


text:
is a "word prediction" problem, applicable using HMMs?
I'm learning HMMs and decided to model a problem for learning purposes. I came to this idea of word predicting by letters.
here is the model :
while typing, the word is typed letter by letter, so we can consider them as series of observations.
let's say we have just 4 words in our database:

Tap
Trip
Trap
Trigger

and we want to predict the word after 1,2 or 3 written letters.
we have to define states and HMM parameters (state transitions, emissions and priors).
our [hidden ?] states would be :

[ ][ ][ ][ ][ ][ ][ ] : no observations. I chose 7 [ ] because of the longest word
T [ ][ ][ ][ ][ ][ ]
T A [ ][ ][ ][ ][ ]
T R [ ][ ][ ][ ][ ]
… .

and we have to learn the transition probabilities of each state pairs (the pairs which come after each other like T,TR and TA. but not  for TA and TR).
our prior probabilities are 1/3 because we have 3 words. but we may change it by learning which word is used more frequently.

Now, I have these questions :

is HMM suitable for this kind of problem ?
are my assumptions (about states and prior probabilities) correct ?
the states get out of hand when the word count increases. making the model very complex. is that deduction true?
how the emission probabilities are defined in this model ?
what do I miss in case of parameters or definitions ?

I'm a new contributor, be nice to me. regards :)

I think a HMM is overkill for this problem. You kind of have 'hidden' states, but they are very limited and dependent on the full sequence of previous states, which you probably want to avoid to make best use of the HMM's features. It also, as you rightly say, leads to a proliferation in states: each dictionary item adds as many states as it has letters to the model.
The way I would approach this it to use a trie which contains your dictionary, and then you traverse the trie as the user types characters. At each point you have a sub-trie which gives you the possible completions of the word. You could even (if available) augment this with probabilities to guess the most likely word (though this does not take into account the previous word in the sentence, which might be more useful).
If you want to learn about HMMs, one possible application would be a weather forecast (I thought this was an example Lawrence Rabiner uses in his tutorial, but it's actually on the HMM Wikipedia page). Or, if you want to work with texts, use parts-of-speech tagging. You want to have something where each observation can belong to several possible states (eg light can belong to adj, noun, and verb). Here you would have the words as observations and the part-of-speech tags as states, which keeps the model at a reasonably small size (and is thus easier to train).

actually I'm planning to use HMMs for limited dictionary speech recognitions (one word commands) and tweak it for very low power microcontrollers. so I have to write it from ground up with many hardware side optimizations. I thought using this learning model would point me in the right direction. the tutorial you provided seems good for my purpose. and for a general opinion, do you think if HMMs are suitable for that low-vocab speech recognitions on low power hardware ?

Yes, should be suitable.


text:
Should I use Monte Carlo or a classifier for this Decision Making problem?
I want to build a model to support decision making for loan insurance proposal.
There are three actors in the problem: a bank, a loaner applicant (someone who ask for a loan) and a counselor. The counselor studies the loaner application and if it has a good profile it will propose to him loan from banks that fits his profile. Then the application is sent to the bank but the bank could refuse the applicant (based on criteria we don't know).
The counselor has also to decide whether or not he will propose to the loaner applicant a loan insurance.
The risk is that some banks reject loan applicant who accepts a loan insurance and other banks accept more applicants with a loan insurance. But there aren't rules regarding banks since some banks accept or reject applicants with loan insurance according of the type of acquisition applicants want with their loan for example.
Thus, the profile of the applicant can matter in their rejection from banks but all criteria influencing the decision are quite uncertain.
I've researched online and found several scholarly articles on using Monte Carlo for decision making. Should I use Monte Carlo or a simple classifier for this Decision Making problem ?
I saw that Monte Carlo (possibly Monte Carlo Tree Search) can be used in Decision Making and it is good when there is uncertainty. But it seems that it would forecast by producing some strategy (after running a lot of simulations) but what I want is an outcome based on both the profile of the loaner applicant and the bank knowing that criteria from banks (to accept loaner applicant from could change every six months. And I would have too model banks which seems quite difficult.
A classifier seems to me to not really fit the problem. I am not really sure. Actually, I don't see how a classifier like a decision tree, for example, would work here. Because I have to predict decision of the counselor to propose or not based on the decision of banks (and I don't know their criteria) to refuse or accept applicants who were proposed loan insurance and accepted it.
The data I have is former applicants profile who were sent to banks and if they were accepted or not by the bank, if they wanted a loan insurance or not and the type of acquisition they wanted to make with their loan.
I am new to Decision Making. Thank you!

A classifier seems to me to not really fit the problem. I am not really sure. Actually, I don't see how a classifier like a decision tree, for example, would work here. Because I have to predict decision of the counselor to propose or not based on the decision of banks (and I don't know their criteria) to refuse or accept applicants who were proposed loan insurance and accepted it.
The data I have is former applicants profile who were sent to banks and if they were accepted or not by the bank, if they wanted a loan insurance or not and the type of acquisition they wanted to make with their loan.

Why does this seem to you like something where a classifier wouldn't fit? Unless I'm missing something, it sounds like a prototypical example of a classification problem to me.
You have:

Input features (applicants' profile)
A clear (binary?) prediction target: propose or don't propose (equivalent to predicting whether or not the bank would accept, because you'll always want to propose if the bank would accept, and never propose if the bank wouldn't accept).
Old training data containing both the input features and the matching prediction targets.

Approaches like Monte-Carlo Tree Search can only be used if you have a forward model or simulator. In your setting, you could view the features (applicants' profile) as a "game state", and model the problem as a game with two actions (propose or not propose). However, you don't have a forward model (a function that, given a current state and action, generates a possible reward and subsequent state).
In applications where MCTS is often used (such as games), you do have such a forward model: for a game like Go or chess, you can easily program the game's rules, program how you transition from one state into another when you select an action, etc. This does not appear to be the case for you.


text:
What is the actual learning algorithm: back-propagation or gradient descent?
What is the actual learning algorithm: back-propagation or gradient descent (or, in general, the optimization algorithm)?
I am reading through chapter 8 of Parallel Distributed Processing hand book and the title of the chapter is "Learning internal representation by error propagation" by PDP research Group.
https://web.stanford.edu/class/psych209a/ReadingsByDate/02_06/PDPVolIChapter8.pdf
If there are no hidden units, no learning happens.
If there are hidden units, they learn internal representation by propagating error back.
Does this mean back propagation[delta rule] is the learning rule and gradient descent is an optimization algorithm used to optimize cost function?

See also the discussion here: https://stackoverflow.com/q/37953585/3924118.

"If there are no hidden units, no learning happens. If there are hidden units, they learn internal representation by propagating error back.", this is a property of the model, not of the learning algorithm.

You can run gradient descent without back propagation, in some cases:

Simple structures such as linear or logistic regression, where the gradients can be calculated directly from the inputs and cost function value.
In "black box" gradient-based learning algorithms where you don't know how (or don't want to) calculate gradient analytically, so you choose to measure it. This is a slow process, similar to gradient checking of neural networks, i.e. adjust each parameter in turn by a small $+\epsilon$, measure your loss function, and divide difference by $\epsilon$ to get gradient estimate for that parameter.

In those cases, you can calculate the gradient, and then use the optimisation function of your choice - e.g. SGD, Adam, Nesterov Momentum - to make changes to the learned parameters. The model will be changed and should learn its task based on the cost function.
You can also run back propagation without gradient descent. In that case, the model will not change and nothing is learned.
That makes the different flavours of gradient descent technically the "core" learning algorithm in my opinion, and back propagation the preferred method for calculating gradients in neural networks. These gradients are most often estimates of an unknown "true gradient" due to training in batches of small samples of training data.
In practice, the two terms back propagation and gradient descent are rarely separated when discussing neural network training. So a lot of people will say that they train a network "using back propagation" or that they are using a gradient descent method without mentioning how they are calculating the gradients using back propagation (because that's pretty much a given). Or they might just say they are "using the Adam optimiser" which in the library they are using automatically runs an NN forward on a mini-batch of data, performs back propagation*, and then changes weight parameters based on the gradients, but heavily adjusted by previous gradient measurements.

* With auto-differentiating frameworks such as TensorFlow, PyTorch etc, it is not really clear that the algorithms are performing "classic back propagation" - although they work using the same chain rule, they may well not perform calculations in the exact same groups that you learned when you studied neural networks and had to write the code for calculating gradients by hand. Instead of back propagating gradients layer-by-layer, they are propagated through an abstract computation graph that includes the learnable parameters.

Gradient descent (GD) is an optimisation algorithm, that is, it is used to find a (local) minimum of a multi-variable and differentiable function $f$. GD is an iterative and numerical optimisation algorithm. It is iterative because it proceeds in iterations. It is numerical because it is not an algorithm which produces an exact solution, due to numerical errors (e.g., round-off errors).
GD is based on the idea that, if a (multi-variable) function $f(\mathbf{x})$ (where $\mathbf{x} \in \mathbb{R}^N$ is a variable representing the input of $f$) is defined and differentiable in a "neighbourhood" of a point $\mathbf{\theta} \in \mathbb{R}^N$, which is a point (in the loose sense of the word "point") in the domain of $f$ (that is, $f(\mathbf{\theta})$ is defined and differentiable), then $f$ decreases fastest if one goes from $\mathbf{\theta}$ in the "direction" of the negative gradient of $f$ at $\mathbf{\theta}$, that is, $-\nabla f(\mathbf {\theta})$. (Do not be scared of the symbol $\theta$, we could have denoted it by $\mathbf{a}$!). Let us break this dense definition down!
Note that here we assume that $f$ is a multi-variable function (that is, a function whose input is not just one real number, but it is a function of $N$ real numbers, which can be grouped in a vector, e.g. $\mathbf{\theta} \in \mathbb{R}^N$), hence $\mathbf{x}$ and $\mathbf{\theta}$ are vectors of real numbers. To emphasize,  $\mathbf{\theta}$ is some specific (and "fixed") point in the domain of $f$, whereas $\mathbf{x}$ is a variable that can take any of these specific points.
The basic statement/assignment which is iterated in GD, which captures the definition above, is the following:
$$\mathbf{\theta}_{n+1} \gets \mathbf{\theta}_{n} - \gamma \nabla f(\mathbf{\theta} _{n}) \label{gd-assignment}\tag{1}$$
where the $\gamma$ is a hyper-parameter which represents the "strength" of the step we take in the direction of the negative gradient (i.e., $-\nabla f(\mathbf{\theta} _{n})$). Note the "minus" (i.e., $-$) in front of $\nabla f(\mathbf{\theta}_{n})$, hence the expression "in the direction of the negative gradient". Note also that here "direction" is implemented as a subtraction. The subscript $_n$ represents the current iteration number.
As simple as it looks, the GD algorithm consists in the execution of the assignment \ref{gd-assignment} until some condition is met (e.g., ${\theta}_{n+1}$ is "close enough" to ${\theta}_{n}$). In a nutshell, GD is an iterative algorithm which updates $\theta$ by iterating \ref{gd-assignment} multiple times.
In the context of machine learning (ML), in particular supervised learning, we initially have an untrained model $\mathcal{M}$ (e.g., a neural network), that is, a model whose parameters (sometimes also called "weights") are e.g. randomly initialised. To train $\mathcal{M}$ (i.e., to find the most appropriate parameters of the model with respect to the training data), we first define a so-called "loss function" (which is differentiable), denoted by $\mathcal{L}$, which represents the discrepancy between the current output of $\mathcal{M}$ and the expected output (with respect to the training data) of $\mathcal{M}$. The process of training a model consists in minimising this loss function, that is, it consists in finding the minimum of $\mathcal{L}$.
To do this, as we have just seen above, we can use GD. If you consider the learning process the one of minimising this loss function $\mathcal{L}$, then any algorithm which minimises $\mathcal{L}$ should be called a "learning algorithm". Hence GD should be called a "learning algorithm".
Note that the parameters of the model $\mathcal{M}$, which need to be found during the training process, are often denoted, in ML, by $\theta$. So, the reason why I used the strange symbol $\theta$ in the assignment $\ref{gd-assignment}$ should now be clearer.
Back-propagation (BP) is just an algorithm, proposed by Seppo Linnainmaa in his master's thesis, to compute the derivative of a differentiable (composite) function, which can be represented as a graph.
In ML, back-propagation is often used to compute $\nabla f(\mathbf{\theta}_{n})$ in the assignment \ref{gd-assignment}. So, BP is used as a sub-routine in the GD algorithm (or any other optimization algorithm) to find the parameters of the model $\mathcal{M}$.
Note that, in the context of ML, $\gamma$ in \ref{gd-assignment} is often called the "learning rate". There's a reason for this: GD is the actual learning algorithm. BP is just a sub-routine used to compute the gradient, but we could have used another sub-routine. We could say, in the context of ML (and, in particular, NNs), that the combination of GD and BP is the actual "learning algorithm", but, if by "learning" we mean "optimisation", then the actual "learning algorithm" is GD and not BP, because BP does
not optimise anything alone.

Gradient descent is a very general optimization algorithm. Backpropagation is a special case of auto-differenciation combined with gradient descent. So backpropagation is a clever way to do gradient descent.
The idea of Gradient descent is to calculate the gradient (derivative) and use that to descent on the error surface. In 3D, you could imagine 2 parameters (x,y) and the 3rd dimension (z) gives you the training error the model makes with those two parameters. You are at some (random) starting point in this error surface and make a step in downwards direction. Then you calculate the error again and do again a step in downwards direction. Note that I didn't write "downwards" but "in downwards direction" as your step might be too big.
Backpropagation consists of two steps: The forward-pass and the backward-pass. In the forward pass, the output of the network for a given input is calculated. In the backward pass, the gradients are calculated with the chain rule.
See http://cs231n.github.io/optimization-2/ for a good reference for back propagation.
Please note that machine learning is way broader than neural networks. For example, decision trees don't use gradient descent at all to train.

What is the actual learning algorithm: back-propagation or gradient descent?

Gradient descent is the learning algorithm. It is realized in backpropagation in the weight-update of the backward pass.
Notes and References
When I write "backpropagation algorithm", I refer to Tom Mitchells "Machine Learning", Chapter 4.5.2.
To cite Tom Mitchells "Machine Learning":

Algorithms such as BACKPROPAGATION use gradient descent to tune network parameters to best fit a training set of input-output pairs.
[...] gradient descent provides the basis
  for the BACKPROPAGATION algorithm [...]
[The section 4.5 MULTILAYER NETWORKS AND THE BACKPROPAGATION
  ALGORITHM] discusses how to learn such multilayer networks using a gradient descent algorithm similar to that discussed in the previous section.
[...]
As shown above, the BACKPROPAGATION algorithm implements a gradient descent search through the space of possible network weights

Comments are not for extended discussion; this conversation has been moved to chat.


text:
What is the approach to deduce formal rules based on data?
We have data in text format as sentences.
The goal is to detect rules which exist in this set of sentences.
I have a limited set of contextless sentences that fit a pattern and want to find the pattern.
I might not have sentences that don't fit the pattern.
What should be an approach to do that?

This I really to vague to answer. What’s the context? Where’s the data coming from? What kind of rules do you want to find? The answers for “I have a bunch of books and want to infer topics” and “I have a set of contextless sentences that fit a pattern and another set of contextless sentences that don’t fit that pattern and want to find the pattern” are extremely different.

If you don't have non-examples of your pattern and don't have some kind of heuristic guide, unfortunately the answer is that you can't. "All sentences" will always be 100% compatible with your examples, and you'll never be able to collect evidence that disconfirms (or even decreases the likelihood) of that hypothesis. Even if you rule out that hypothesis by fiat, the hypothesis that the only non-accepted sentence is

The answers for “I have a bunch of books and want to infer topics” and “I have a set of contextless sentences that fit a pattern and another set of contextless sentences that don’t fit that pattern and want to find the pattern” are extremely different.

will be impossible to rule out
With a good enough heuristic guide, you might be able to do something, but that would require knowing statistical facts about examples and non-examples and then sampling from the world IID, and just happening to not get any non-examples. That doesn't seem like the situation you're in. If you really have nothing to quantify the non-examples in any way, there's nothing that can be done.


text:
Label arrangement for custom Keras image generator
I am trying to generate 90 and 270 degrees rotated versions of my sample images on the fly during training. I found an example and modifying it. But I am confused about what should be the order? For instance in one batch I have 32 images and my image generator should return total 64 images. Let's say upper case letters are 90 degree and lower case letters are 270 degree rotated images. Should the order be AaBbCc or ABCabc? I apply the same to the validation set. Here is the related code fragment:
Edit: Code fragment added.
    def _get_batches_of_transformed_samples(self, index_array):
    # create list to hold the images
    batch_x = []
    # create list to hold the labels
    batch_y = []
    # rotation angles
    target_angles = [0, 90, 180, 270]
    angle_categories = list(range(0, len(target_angles)))
    self.classes = target_angles
    self.class_indices = angle_categories
    # generate rotated images and corresponding labels
    for i, j in enumerate(index_array):
        is_color = int(self.color_mode == 'rgb')
        image = cv2.imread(self.filenames[j], is_color)
        if is_color:
            image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

        for rotation_angle, cat_angle in zip(target_angles, angle_categories):
            rotated_im = rotate(image, rotation_angle, self.target_size[:2])
            if self.preprocess_func: rotated_im = self.preprocess_func(rotated_im)
            # add dimension to account for the channels if the image is greyscale
            if rotated_im.ndim == 2: rotated_im = np.expand_dims(rotated_im, axis=2)
            batch_x.append(rotated_im)
            batch_y.append(cat_angle)

    # convert lists to numpy arrays
    batch_x=np.asarray(batch_x)
    batch_y=np.asarray(batch_y)

    batch_y = to_categorical(batch_y, len(target_angles))
    return batch_x, batch_y

I actually rotate them as 0, 90, 180 and 270 degrees. As seen in the code, for each batch I return all the rotated versions of all the images in the batch. But is this correct or should I return first 0 degree rotated versions, second 90 degree rotated versions so on?
Edit2: I checked my previous work which I use the built in Keras ImageDataGenerator. generator.classes returns [zeros(100,1); ones(100,1)]. In that study I only have two classes. I understand that Keras indexes the images as [class1, class2, ...]. I think I have to do the same.

I am not allowed to comment, so I am adding it here. Not sure I understand, are you asking if the training data must be ordered?
If so, training data must be random, and not AaBbCc or ABCabc.
Here the input to the NN will be the source image, and a category 90 degrees or 270 degrees as too separate inputs. i.e. inputs [0...k][k+1][k+2], where 0 - k are normalised pixel data, k+1 is 1 for 90 degree rotation else 0, and k+2 is 1 for 270 degree rotation else 0. Alternatively for more rotation options lets make a = input [k+1] and b = input [k+2]. If a = 0 and b = 0 then 0 degrees, if a = 0 and b = 1 then 90 degrees, if a = 1 and b = 0 then 180 degrees if a = 1 and b = 1 then 270 degrees.
From your image set, draw 3 samples. Sample 1 [training set] must be say 50% of the images, sample 2 [validation set] say 30% of the images and sample 3 [out of sample set] 20% of the images.
Use the training images from sample 1 to update the weights. Read in images from sample 1 at random, the rotation must be random, use your image rotator to rotate the image, and compare it to the output of the neural network, calculate the sum of the mean error and perform gradient descent to update the weights. Repeat the process. After every x weight updates randomly select a few images from the validation set and calculate an error value. If the average error value of the validation images are above some threshold then stop, or if the error begins to get worse. Do not use the validation set to update the weights
At the end use the out-of-sample set of images to verify the performance of the neural network.
The process above does not include hyper-parameter optimisation, or any other optimisation techniques, but does describe the basic way of training a NN using supervised learning.
Edit: I assumed that you wanted to NN to learn to rotate the image. After your updates, I assume you want rotated images to increase generalisation of your NN. In which case randomising or shuffling the input batch is preferred. i.e. send your images to Keras to rotate them, get the batch back any order is fine ABab or AaBb and then shuffle them. Pass the shuffled training and validation set to the NN and compare it to the known label for weight updates.

Actually my question is about how should the image generator return the images. A, B are my images and I rotate them 90 and 270 degrees. My question is should the generator return a batch as [90, 90, 90, 270, 270, 270] deg or [90, 270, 90, 270, 90, 270] deg. I added the related part of the code. Thank you for your help.

I understand. Passing in A, B into the generator can rotate the images in any way, thats not really that important. However to get the best results the batch of images both original and rotated should be shuffled. If you pass the images to the NN ordered during training ie AaBb or ABab it will effect learning.


text:
Calculation of GPU memory consumption on softmax layer doesn't match with the empirical result
I'm training a language model with 5000 vocabularies using a single M60 GPU (w/ actually usable memory about 7.5G).
The number of tokens per batch is about 8000, and the hidden dimension to the softmax layer is 512. So, if I understand correctly, fully-connected (softmax) layer theoretically consumes 5000*8000*512*4=81.92GB for a forward pass (4 is for float32).
But the GPU performed the forward and backward passes without any problem, and it says the GPU memory usage is less than 7GB in total.
I used PyTorch. What's causing this?
EDIT: To be clearer, the input to the final fc layer (256x5000 matrix) is of size [256, 32, 256].

GPU DRAM capacity - 7.5G
the below link explains how nVIDIA GPU cUDNN does memory optimization.
https://devblogs.nvidia.com/optimizing-recurrent-neural-networks-cudnn-5/
the below link has detailed steps to calculate memory required by parameters and data.
http://cs231n.github.io/convolutional-networks/#case
one data point missing in question is number of output classes in softmax layer.
The above two links will help you to calculate memory required and how software handles large matrix multiplication.

Thank you. The number of output classes is 5000, which is the number of vocabulary. Apparently, I didn't need to multiply 5000, so the memory consumption is at the order of 10MB.


text:
Flatten image using Neural network and matrix transpose
I have read a lecture note of Prof. Andrew Ng. There was something about data normalization like how can we flatten an image of (64x64x3) into a (64x64x3)*x1 vector. After that there is pictorial representation of flatten

As per the picture height, length and width of the picture is 64 , 64, 3. I think nx is a row vector which is then transpose to a column vector. If there is 3 pictures I think nx contains {64,64,3,64,64,3,64,64,3}. Am I right?
To use a 64x64x3 image as an input to our neuron, we need to flatten the image into a (64x64x3)x1 vector. And to make Wᵀx + b output a single value z, we need W to be a (64x64x3)x1 vector: (dimension of input)x(dimension of output), and b to be a single value. With N number of images, we can make a matrix X of shape (64x64x3)xN. WᵀX + b outputs Z of shape 1xN containing z’s for every single sample, and by passing Z through a sigmoid function we get final ŷ of shape 1xN that contains predictions for every single sample. We do not have to explicitly create a b of 1xN with the same value copied N times, thanks to Python broadcasting.
As per my understanding, Wᵀ = nx and x= nxᵀ.
Is it Wᵀ= [64,64,3,64,64,3,64,64,3] and x = [64,64,3,64,64,3,64,64,3]ᵀ?
In that case there product will be a symmetry matrix.
Is there any significance of symmetry matrix?
I just messed up all the things while flatten the image. If anyone has any idea please share with me.
Thank you in advance.

Yes, if you have 3 images (and by images I assume you mean samples) the flatten layer will be of the shape $12288*3$ ($64*64*3=12288$). The size of $W$ however does not change, and nor does the size of $b$ as these are parameters and are independent of the amount of samples passed through the network.
ETA: I only answered the "Am I right?" part of your question because that's the only part of your questions that's actually a question. I don't know what you're trying to ask in the second half of your question

Thank you. Yes image means samples. In the second half my question I want to the input for W^T  and x. Is it W^T= [64,64,3,64,64,3,64,64,3] and x = [64,64,3,64,64,3,64,64,3]^T?


text:
About the requirement to compute the gradient at layer $l$
I'm trying to understand a line of my note.
Let's say there is a simple feedforward neural network that has $N$ layers, and for a given layer $l$, it has weight $W^l$, and $g^l$ is the gradient to update it. Now the problem is:

From the Wiki page of Backpropagation, to compute $\nabla_{W^l}C$(or simply $g^l$), i.e. the gradient of the cost function with respect to the weight of layer $l$, you will need two things:

A sub-expression $\delta^l$, which is the gradient of the weighted output of the current layer $l$, i.e. $\nabla_{z^l}C$. (there is no such symbol in the link, but I believe my usage is correct.)
The activation of the previous layer $l-1$, i.e. $a^{l-1}$.

This is why it says $\nabla_{W^l}C=\delta^l(a^{l-1})^T$. My reasoning of this is that: the weighted output of layer $l$, i.e. $z^l$, is the result of multiplication between the output of the previous layer, i.e. $a^{l-1}$, and the weight of the current layer, i.e. $W^l$.

But now I found my note saying something different: it says that to compute the gradient $\nabla_{W^l}C$(or simply $g^l$), it would require $W^{l+1},g^{l+1},a^l$,  that is: the weight and gradient of the next layer and the output of the current layer.

Is my note wrong?

A sub-expression $\delta^l$, which is the gradient of the weighted output of the current layer $l$, i.e. $\nabla_{z^l}C$. (there is no such symbol in the link, but I believe my usage is correct.)

First of all, you're correct that the symbol $\delta^l$ represents the gradient of the cost w.r.t. the weighted output (so the activation function has not been applied) of the layer $l$. This reasoning for this is simply chain-rule.
But you have to be aware of that the auxiliary function $\delta^l$ involves the value of $(f^l)'$ at $z^l$. This means that you will need to save $z^l$ for computing $\delta^l$.

But now I found my note saying something different: it says that to compute the gradient $\nabla_{W^l}C$(or simply $g^l$), it would require $W^{l+1},g^{l+1},a^l$,  that is: the weight and gradient of the next layer and the output of the current layer.

By $\nabla_{W^l}C=\delta^l(a^{l-1})^T$, it's clear that you will need both $\delta^l$ and $a^{l-1}$ to update the weight $W^l$. For $\delta^l$, you will need:

the $\nabla_{a^l}{C}=(W^{l+1})^T\delta^{l+1}$, which is the gradient of the cost w.r.t. the activated weighted output of the current layer $l$.
the weighted output $z^l$ to compute $(f^l)'$ at $z^l$, which is mentioned in my first reply.

To compute the gradient $\nabla_{W^l}C$, the value of these symbols are required:

$W^{l+1}$.
$\nabla_{a^l}{C}$ instead of $g^{l+1}$, which is the gradient of the weight as you have said.

this involves $z^l$, as stated above.

$a^{l-1}$ instead of $a^{l}$, since the latter is for $\nabla_{W^{l+1}}C$.

So yes, your note is incorrect.

reference: https://towardsdatascience.com/back-propagation-simplified-218430e21ad0


text:
How should I label the classes in RNA?
I have a project, which is the keyboard biometrics of users.
suppose I have 3 users,
I do not know how to label in two types of class, (+ 1, -1).
If I want to verify the identity to user1, my idea of class designation would be:
       TIMES                LABEL
user 1
9.4  9.2  1.0  3.4  0.5      1
9.4  9.2  1.0  3.4  0.5      1
9.4  9.2  1.0  3.4  0.5      1
9.4  9.2  1.0  3.4  0.5      1
9.4  9.2  1.0  3.4  0.5      1

user 2
0.1  3.2  1.0  1.2  1.7      -1
3.4  1.2  3.0  1.1  2.8      -1
2.4  2.2  3.0  1.6  2.9      -1
1.4  3.2  2.0  2.6  3.6      -1
3.4  0.2   3.0  2.7  3.5     -1

user N
0.2  1.4  4.5  3.7  2.9      -1
9.2  1.5  7.6  2.6  2.6      -1
9.3  1.6  7.5  2.9  3.4      -1
9.8  3.8  6.6  2.8  2.5      -1
9.8  2.8  1.7  3.8  1.6      -1

but as my system has more and more users classes -1 will be too many compared to classes +1,
How should I label the classes?

Just as a quick clarification, are you wondering how to add more than two class (-1 and +1) to your model?  Specifically what type of classifier are you using?  A neural network? An SVM?  Logistic regression?  There are many ways to do this, some of them specific to the model and some more general to any classifier.

@juicedatom i used network neural, but my data is inbalanced data

From a purely dataset-development perspective I would just label the classes with numbers starting from 0 (ID of the first person) to N (ID of the last person.
During training however, what you do with those classes will vary depending on the type of architecture you are training.  For example, if you are building a neural-network classifier you could simply have multiple output nodes.  When running backprop with a label for the ith person you could put a 1 on the ith output layer and -1 on the rest of them.  This is similar to this answer as well
Whatever you end up doing, try not to label your data to the benefit of the algorithm you're trying to run unless you really know what you're doing.  You don't want to accidentally lose information or make your training data less interpretable and harder to deal with in the long run.


text:
Using 3D Points as Inputs to a Neural Net
I am currently looking to use a neural network to classify gestures. I have a series of Dx,Dy,Dz readings that represent the differences across the three axes made during the gesture. About 10 movements for each example of the gesture. Basically a 10x3 matrix and then classify the training data into about 15 classes. I plan to use a CNN classifier to do this because, while the time domain is relevant this problem the difference in the movements can be differentiated when presented with as a discrete matrix.
I'm used to using images with a neural net so I instinctively want to just convert the matrices into a 2D tensor and feed them into a CNN, but I was wondering if there was a better way to do this? For example, I have seen 1D tensors passed to a fully connected neural network for classification which seems like it could be more appropriate for this data input type?
Any tips on general architecture would be really appreciated as well!
Thanks!

Welcome to AI.SE ! It sounds weird when you say that you "flatten the matrix into a vector and feed it into a CNN" . CNN need tensors, not 1D vectors... are you sure you used this technique ?

Sorry, I was getting two methods mixed up - one using a CNN and one using a fully connected network - my bad. I'll update the question.

A few thoughts :

10x3 matrix for each example is really a small amount of data. FCNN could do a good job on that.
As a result, I'm not sure CNN is appropriated. The smallest dimension is 3, so it'll force you to have really small kernels.
Have you thought about LSTM? since that your data is sequential, LSTM may be useful. However, I'm not sure they could be really effective on this little amount of data. but it could be nice to try.

Thank you so much for the suggestions, really appreciated! I'll check out the FCNN approach first then and after that see if I can try an LSTM


text:
Recognising Noise in Simple Classification
I have created a classifier for some simple gestures using an input layer, a hidden layer with tanh activation and an output softmax layer. I'm also using the Adam optimiser. The network classifies perfectly with validation data. However, I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently. For example:
One gesture input looks like this and is correctly classified as gesture 'A':

However, when I pass this 'noise', which is clearly differentiable to the human eye, as input it still classifies it with 100% confidence that it is the same gesture 'A'.

I assume it's because the inputs are still very close to 0? My instinct is to scale up the inputs perhaps to increase the differentiation between the noise and the input. However, in real operation the noise will all be on a similar scale to the inputs and I won't know what is noise and what isn't so I will still have to apply the same scaling to that noise. Will I run into the same problem?
On a more general note is there a teaching approach to prevent misclassifications, particularly if we know what they might look like? For example, in this case I thought I could perhaps generate some noise and use it at training time to create an extra noise class, or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold? For example, if the network only produces 50% classifcation confidence for an input then I can discard it as noise. Any suggestions much appreciated!

The network classifies perfectly with validation data, however I'd like it to be able to take in random noise that looks nothing like the shapes and not be able to classify it confidently.

You need to train the network on noise well if you want it to be able to recognize it.

or is it just best to come up with such a well-trained network that you can use some sort of confidence threshold?

You can use spot to get a confidence score for clarification. But training the network on a noise class will work way better
See:

Gal: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning


text:
Is it possible to state an outliers detection problem as a reinforcement learning problem?
To me it seems to be ill defined. Partially because of absence of knowledge which points are to be considered outliers in the first place.
The problem which I have in mind is "bad market data" detection. For example if a financial data provider is good only most of the time, but about 7-10% of data do not make any sense.
The action space is binary: either take an observation or reject it.
I am not sure about the reward, because the observations would be fed into an algorithm as inputs and the outputs of the algo would be outliers themselves. So the outliers detection should prevent outputs of the algorithm going rouge.
It is necessary to add that if we are talking about the market data (stocks, indices, fx), there's no guarantee that the distributions are stationary and there might be trends and jumps. If a supervised classifier is trainer based on historical data, how and how often should it be adjusted to be able to cope with different modes of the data.

Reinforcement learning is not a category of problems. It is a solution method for Markov decision processes (MDPs). The question should be "Can outlier detection be formulated as an MDP?".

Thank you for the comments. The description was expanded to address question from @NeilSlater.

@HaiNguyen: The additional factor that makes something a "RL problem" as opposed to an MDP formulation is the learning/optimisation goal, such as the definition of return, discount factor, whether to use average reward, whether we aim for optimal control or measuring a fixed policy. Otherwise agreed, a large part of the OP's problem would be about formulating their scenario as a MDP (or more likely framing it as one at a higher level due to consequences of choices). However, I don't see a need to re-word that part of the question

It is quite often possible to frame a problem as a Reinforcement Learning (RL) problem at some level. However, this may turn out to be for no benefit, or a net cost towards solving the problem. Casting parameter or hyperparameter searches as RL can be adding a layer of complexity and reduce efficiency.
One key thing to bear in mind is that any classification or regression that occurs within a RL framework will end up using effectively the same models and approaches that could solve the same problems directly. These models would either appear directly as the function approximators in RL that implement policies or value functions, or they would be an implied part of them. If you have labeled data for classification - even delayed until some time after you collect data, then you are usually going to be better off using supervised learning directly.
For hyperparameter searches (e.g. cutoffs for anomaly detection) then you may not need labelled data, but just need a good way to test the model offline.
The first point at which supervised learning or classic anomaly detection might fail for you is if you never receive any feedback about individual records, only a measure of overall performance. In other words, if you can measure consequences of good performance, but never measure or check  correctness.
Your characterisation

about 7-10% of data do not make any sense.

does not appear to fit that. It looks like you could detect this, maybe manually labelling a few thousand records, and train a classifier using supervised learning techniques. That is likely a much better use of your time than trying to restructure the problem at a higher level and trusting a trail-and-error approach to discover the same rules.
Putting that to one side, assuming you do have a problem where

data to be classified is arriving as a stream, and needs to be processed online, item by item or in small batches
you have reason to think that an accept/reject stage before processing further would be useful
you have no way to label training data for accept/reject
you have a way to measure performance of the remaining system after the accept/reject phase

then you could use RL to frame the accept/reject phase as an action. There are some challenges there, but essentially you would use RL along with measurement feedback to sample errors or gradients - typically using TD Error or policy gradients. This could wrap almost any model that does classification or anomaly detection etc, provided it could be trained using those gradients.
From comments, if the underlying distributions for accept/reject are non-stationary, this may point you more towards a RL solution. However, that may come with a cost to performance - you will need to balance exploration rate (which will reduce the performance of the model against stationary data) versus speed of learning new distributions. This is a problem for all online learners; the main advantage of a RL approach here is that it will not require generating new labelled data. If you can use a recency-weighted anomaly detection algorithm instead, then you won't need the labeled data either - whether that is better requires testing, personally I'd take a working anomaly detection as the baseline and only use RL if it proved itself better.
The specific items that you turn into states and rewards are not clear to me from the question, and you would need to work on these things carefully. It is possible you will need more than the current data item in order to define state, and that will depend a lot on how the feedback loop works that establishes reward.

Regarding classification and labelling, there still might be a problem with data which have a trend or which might have non-stationary distribution. RL approach would help to adapt to changes in the underlying data.

@ArtemOboturov: It might, but at the expense of needing to increase exploration, which would impact performance of the system as the agent would need to make more deliberate misclassifications over time in order to test whether distributions had changed. Most RL agents do at least have mechanisms for that - e.g. some form of Upper Confidence Bound - whether it is worth doing depends on the degree of drift you expect, and how happy you would be to allow this routine testing to affect your data stream.

Also, if you find that you can rely on unlabelled anomaly detection, then provided there is always a reasonable distance between acceptable and reject-able data, you can probably just use a simple online version of that - e.g. base the acceptance scores for mean and standard deviation on recency-weighted statistics.

@ArtemOboturov I have added a paragraph about non-stationary problems to the answer.


text:
How to approach this handwritten digit recognition?
I have multiple pictures that look exactly like the one below this text. I'm trying to train CNN to read the digits for me. Problem is isolating the digits. They could be written in any shape, way, and position that person who is writing them wanted to. I thought of maybe training another CNN to recognize the position/location of the digits, but I'm not sure how to approach the problem. But, I need to get rid of that string and underline. Any clue would be a great one. Btw. I would love to get the 28x28 format just like the one in MNIST.
Thanks up front.

I think one approach you can try to segment the digits and Connected Components Labeling (https://en.wikipedia.org/wiki/Connected-component_labeling).
With it, you'll end up with a label for each letter and then you can try to find the convex hull of the letter.
After that, just crop a square for each convex hull and input it to your CNN.
Notice that it will only if there is at least one pixel between the letters...

@Júlio César Batista Isn't that just looping over contours in OpenCV and removing them. Just this time, I have some labels thanks to the CCL. I really don't think it should work. What if there is some other element in front of that string, and It is big enough so that I can't close it with dilation or erodion without removing the parts that I care about.

Use of CNNs to recognize digits is a reasonable approach as of this answer's writing, the effectiveness of which can be enhanced via Sensitive Error Correcting Output Codes, (2005, John Langford, Alina Beygelzimer) according to Shuo Yang et. al. in Deep Representation Learning with Target Coding.
Given a sufficient CNN depth, it can be trained to recognize the digits with the field name and underscore intact. Automatically removing them would generally only be possible after they are recognized using the CNN approach anyway.
Isolating the digits from the field name and the underscore would be accomplished via the same approach used to isolate the digits from one another. There is no reason to perform these two conceptual tasks in series and consume additional development time and resources, when a single deep CNN can locate the digits and dismiss the field name and underscore. This is similar to animal vision. A mosquito's visual pathway distinguishes an oncoming object from the background to avoid a swat using the same network it uses to recognize the object.
What would be helpful would be to normalize the input by finding a two dimensional plateau (much like a geological one) representing the distribution of black pixels in horizontal and vertical dimensions. Summing rows and columns of pixels and using a heuristic algorithm to find a rectangle outside of which there is only noise may be sufficient. Then trim and scale, which removes redundancy and may improve CNN training speed, accuracy, and/or reliability by distributing the image more widely over the input layer and likely reducing the layer count by at least one.
Another approach is to use overall features of the form in which the field resides to rotate, position, and scale the image so that the field name and underscore is within a fraction of a pixel from an expected location on the form. In this case training a separate CNN is less redundant and has a higher return on development investment. In such a design, the field name and underscore can be removed by subtraction, however this may disturb field value recognition because the handwriting may overlap items blanked. It will require experimentation to determine the accuracy and reliability hit (diminution) from such disturbance.
Addendum in Response to Comment
For the first approach, a Field Name and Underscore Superimposer will need to be designed and implemented to transform the training example set features, the pixel arrays, but preserve the labels corresponding to each example. The hyper-parameter values that work best without the superimposition may need modification to work best with it.

What would be your preferred way of solving the problem. I'm interested in your first approach. So, if I build network big enough it could understand field name and underscore like some kind of a constant, that would not impact on the result? In general one bounding box that will hold filed name, underscore and number in one scale and normalize that as input and use it for training. I'm not sure in this approach would it ask for more training examples and what else would change? But it sounds great. By the way, labels would still be only exact number values?

I guess that by Field Name and Underscore Superimposer will need to be designed and implemented to transform the training example set features, the pixel arrays, but preserve the labels you really mean: the data set will have different features because images would include a Field Name and Underscore Superimposer but labels would stay the same of course. On the other hand, the new feature of data set would not have such a great impact on the hyper-parameters because basically, every picture would have the same new feature.

Also if a model is trained on 28x28 image, wouldn't that approach maybe had its pitfalls as scaling to 28x28 would make the image more unreadable. The dimensions would need to go up?