text:
Is it possible to create a fair machine learning system?
I started thinking about the fairness of machine learning models recently.
Wiki page for Fairness_(machine_learning) defines fairness as:

In machine learning, a given algorithm is said to be fair, or to have
  fairness if its results are independent of some variables we consider
  to be sensitive and not related to it (f.e.: gender, ethnicity,
  sexual orientation, etc.).

UC Berkley CS 294 in turn defines fairness as:

understanding and mitigating discrimination based on sensitive
  characteristics, such as, gender, race, religion, physical ability,
  and sexual orientation

Many other resources, like Google in the ML Fairness limit the fairness to these aforementioned categories and no other categories are considered.
But fairness is a lot broader context than simply these few categories mentioned here, you could easily add another few like IQ, height, beauty anything that could have a real impact on your credit score, school application or job application. Some of these categories may not be popular in existing datasets nowadays but given the exponential growth of data, they will be soon, to the extent, that we will have an abundance of data about every individual with all their physical and mental categories mapped into the datasets.
Then the question would be how to define fairness given all these categories presented in the datasets. And will it even be possible to define fairness if all physical and mental dimensions are considered as it seems that when we do so, all our weights in, say, the neural nets should be exactly the same, i.e., giving no discriminator in any way or form towards or against any physical or mental category of a human being? That means that a machine learning system that is fair across all possible dimensions will have no way of distinguishing one human being from another which would render these machine learning models useless.
To wrap it up, while it does make perfect sense to withdraw bias towards any individual given the categories like gender, ethnicity, sexual orientation, etc., the set is not closed and with the increasing number of categories being added to this set, we will inevitably arrive at a point where no discrimination (in a statistical sense) would be possible.
And that's why my question, are fair machine learning models possible?
Or perhaps, the only possible fair machine learning models are those that arbitrarily include some categories but ignore other categories which, of course, if far from being fair.

You appear to have deliberately misconstrued the definition of fairness in this question to always include all factors, making it impossible to answer without a frame challenge to your definition. Most people would not extend "fair" to always include all categories for all uses, and it results in a reductio ad absurdum argument against automated fairness which is unlikley to be the goal of any AI decision-making system. Is that the question you want to ask?

I see that you are not quite agreeing with how I phrased fairness. Well, these definitions, as you see were not coined by myself. I merely refer to them. Are they clear and concise. Clearly not. That's why i even highlighted the word etc. such as should also be highlighted.There is a huge room for manoeuvre with regard to which categories should or should not be treated fairly. If I am reading you correctly fair in AI should be include just some categories and exclude others? Is that your point?

My point is this: We cannot do philosophical debates on Stack Exchange. If your question resolves to "Why is fairness not usually considered all-inclusive of all traits?" then it may be better for Philosophy Stack Exchange. If your question is about how fairness is handled by AI systems, you may want to step back from your reductio ad absurdum (to paraphrase "In an absolutely fair system, no measurable trait may be used to make a decision, so what's the point?") because no-one treats fairness like that in practice

Or another way to put it: I would not want to post an answer briefly dismissing the over-broad definition of "fairness" you have used and start talking about fairness in AI systems, only to enter into a debate with you about the definition you have used. However, if you are looking for guidance/correction on the definition itself, it could be answered (perhaps better on Philosophy SE though)

Well, unthinkable as it is, (some) software/data engineers do make these decisions daily.  You might be aware of it. Or not. We technical people do that. Now, you may say, I am just a small cog in the wheel, let someone else do the thinking what fair in ML means. Fine. But Engineers could and should be aware of the impact of their work. The challenge with your point is that you avoid making a point what fair in AI is. You challenge the definitions I brought up but do not provide any other definitions of what fair in ML is?  You simply avoid making statements about fairness in ML.

@matcheek I would reformulate your question "are fair machine learning models possible?" as "Is there a definition of fairness applicable to ML models that takes into account all possible factors that can affect 'fairness'?" (or something like that). In general, I suggest you formulate your question so that it can be answered more objectively and that it avoids debates.

Correct I am avoiding a definition, because I do not want to answer in comments, or enter into a long debate here. I am pointing out that starting with your definition could be problematic, and trying to figure out if this is a hard line in the question. Would someone saying "That isn't a useful definition of fairness, but this is . . ." result in a useful answer or cause you to want to debate with the answerer?


text:
How to use one-hot encoding for multiple columns (multi-class) with varying number of labels in each class?
I am a beginner in TensorFlow as well as in AI. I am basically from Pharma background and learning AI from scratch.
I have data with 5038 input (Float64) and 826 output (Categorical - Multi Labels in each column). I have utilized one-hot encoding but the neural network tackles only one output at a time.
[1]How to process all 826 output (which give 6689 one-hot output) at once in a neural network. Here is the code that I am using.
[2] I am getting only 31% accuracy. I get this accuracy just in the second epoch. From second or third epoch onwards accuracy and other parameters become constant. Am I doing the wrong code here?
dataset = df.values
X = dataset[:,0:5038]/220
Y_smile = dataset[:,5038 :5864]

from sklearn.preprocessing import OneHotEncoder
enc = OneHotEncoder(handle_unknown='ignore')
enc.fit(Y_smile)
OneHotEncoder(handle_unknown='ignore')
enc.categories_
Y = enc.transform(Y_smile).toarray()
print(Y,Y.shape, Y.dtype)

from sklearn.model_selection import train_test_split
X_train, X_val_and_test, Y_train, Y_val_and_test = train_test_split(X, Y, test_size=0.3)
X_val, X_test, Y_val, Y_test = train_test_split(X_val_and_test, Y_val_and_test, test_size=0.5)

import numpy as np
X_train = np.asarray(X_train).astype(np.float64)
X_val = np.asarray(X_val).astype(np.float64)
X_test = np.asarray(X_test).astype(np.float64)

filepath = "bestmodelweights.hdf5"
checkpoint = [tf.keras.callbacks.ModelCheckpoint(filepath, monitor='val_accuracy', mode='auto', save_best_only=True, Save_weights_only = True, verbose = 1),
              tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=5, verbose =1)]

model = tf.keras.Sequential([
                             tf.keras.layers.Dense(1024, activation='relu', input_shape=(5038,)),
                             tf.keras.layers.Dense(524, activation='relu'),
                             tf.keras.layers.Dense(524, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(1024, activation='relu'),
                             tf.keras.layers.Dense(6689, activation= 'softmax')])

model.compile(optimizer=tf.keras.optimizers.Adam(), loss=tf.keras.losses.BinaryCrossentropy(from_logits = True), metrics=['accuracy'])

hist = model.fit(X_train, Y_train, epochs= 200, callbacks=[checkpoint],validation_data=(X_val, Y_val))

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Deep learning techniques with time-fixed, time-dependent and imaging data
I have a question about the use of deep learning techniques with time-fixed features and images (setting 1) and time-dependent features (setting 2). (I am pretty new to the deep learning world so please excuse me if it's a basic question.)
Setting 1:
Imagine having a training dataset composed of

some time-fixed features such as height, weight, and age of an individual at the first medical visit (these features are recorded once and therefore time-fixed, i.e., they do not change in time in the dataset).
some medical images for each individual, such as for example a CT scan.
a label defining if the patient has or not a specific disease.

Setting 2:
Same as setting 1 but with some features that are repeated over time (time-dependent, longitudinal), such as for example blood pressure recorded twice a day for each individual for several days.
Let say that the goal is to classify if an individual has or not a specific disease given the aforementioned features.
I have seen zillions of papers and blogs talking about convolutional neural networks to classify images and a few million about recurrent neural networks for time-dependent features. However, I am not very aware of what to use in case I have time-fixed, time-dependent, and imaging features altogether.
I am wondering how you would attach this problem.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Visualisation for Features to Predict Timeseries Data
I have a course assignment to use an LSTM to predict the movement directions of stock prices. One of the things I am asked to do is provide a visualization to compare the predictive powers of a set of N features (e.g. 1-day return, volatility, moving average, etc.). Let's assume that we use a window of 50 days as input to the LSTM.
The first thing that came to my mind is to use a RadViz plot (check below image taken from https://www.scikit-yb.org/en/latest/api/features/radviz.html).
However, I soon realized this will not work for the features since each sample will have 50 values. So if we have M samples, the shape of the input data will be something like Mx50xN. This, unfortunately, is not something RadViz can deal with (it can handle 2D data).
Given this, I would be grateful if someone can point me to a viable way to visualize the data. Is it even possible when each feature comprises 50 values?


text:
Fitness function in genetic algorithm based on an interval
I am writing an app, where when a ball is shot from a canon it is supposed to land in a hole that is on a given distance. The ball is supposed to land between the distance of the beginning of the hole and the end of the hole. The size of the hole is 4m and the size of the ball is 0.4m. My problem is that I am not sure how to write the fitness function for this. The place where the ball falls should be close to this interval of [D, D+3.6], where D is the distance of the hole. If anyone could give me a hint on how to approach this problem, I would be grateful.

Have you definitely been asked to solve using a GA? This would be OK for a toy/learning example, but at this level of complexity is far easier to solve analytically that applying a GA.

yes, I must do it using GA

Genetic algorithms work best when given a scalar fitness value that increases smoothly, so that you can compare two population members regardless of whether they failed or succeeded at the task.
That usually requires you to analyse the problem, and come up with a measure that would improve as an individual gets closer to solving a task. It generally helps if you score better for an individual if they solve a problem "better". You want to avoid simple boolean success/fail metrics.
A simple measure of how well an individual has done is to use the absolute distance it was away from a "perfect" shot (where the middle of the ball hits the middle of the hole). The only issue with this is that a perfect shot scores 0, whilst a miss scores 2+, and you want the best result to have the highest fitness. This can be fixed simply, take the negative of the absolute distance:
$$F = -|D_{hole} - D_{ball}|$$
where $D_{hole}$ - and $D_{ball}$ are horizontal distances from origin to centre of each object.
There is no requirement for fitness score to be positive. This will score $F \lt -2$ for a miss, $-2 \lt F \lt 0$ for hit, and $0$ for a perfect shot.
For GAs you don't have to care as much about differentiability or gradient of the fitness function, provided it gives reasonably good ranks between quality of individuals. So there is no point using e.g. a squared error metric here, although you could if you wished.


text:
zonal or template ocr invoices reading
I'd like to explore the possibilities of applying artificial intelligence to ocr reading.
Basic ocr invoices processing let me convert 30% of them only.
The main purpose is defining invoices areas by training an ai, then process those areas with ocr.
So I am looking into ai to define and recognize a document topology first, then apply ocr locally.
From a brief search, it is classified as zonal or template ocr.
Any chance of a premade open source library?


text:
Detect root cause across many event occurrences
Suppose there are sensors which supply numerical metrics. If a metric goes above or below a healthy threshold, an event (alert) is raised. Metrics depend on each other in one way or another (we can learn the dependencies via ML algorithms) so when the system is in alerting state only one or a few metrics will be a root cause and all others will be simply consequences.
We can assume there is enough historical metric data available, to learn dependencies but there are just a few historical malfunctions. Also, when malfunction happens there is no one to tell what was the root cause, the algorithm should learn how to detect root causes by itself.
Which algorithms can be used to detect the root cause events in the situation above? Are there any papers available on the subject?

Having worked in a chemical analysis group which tracked down  problems in a complex manufacturing process, I can say that always being able to prevent manufacturing problems is hopeless. Controls are put in place to prevent known problems, but there seem to be an infinite number of unknown ways to foul up complex processes. Also the cause of some problems is too complex to detect. So it is just impossible to prevent all manufacturing problems.


text:
Dyna-Q algorithm, having trouble when adding the simulated experiences
I'm trying to create a simple Dyna-Q agent to solve small mazes, in python. For the Q function, Q(s, a), I'm just using a matrix, where each row is for a state value, and each column is for one of the 4 actions (up, down, left, right).
I've implemented the "real experience" part, which is basically just straightforward SARSA. It solves a moderately hard (i.e., have to go around a few obstacles) mazes in 2000-8000 steps (in the first episode, it will no doubt decrease with more). So I know that part is working reliably.
Now, adding the part that simulates experience based on what it knows of the model to update the Q values more, I'm having trouble. The way I'm doing it is to keep an experiences list (a lot like experience replay), where each time I take real action, I add its (S, A, R, S') to that list.
Then, when I want to simulate an experience, I take a random (S, A, R, S') tuple from that list (David Silver mentions in his lecture (#8) on this that you can either update your transition probability matrix P and reward matrix R by changing their values or just sample from the experience list, which should be equivalent). In my case, with a given S and A, since it's deterministic, R and S' are also going to be the same as the ones I sampled from the tuple. Then I calculate Q(S, A) and max_A'(Q(S', A')), to get the TD error (same as above), and do stochastic gradient descent with it to change Q(S, A) in the right direction.
But it's not working. When I add simulated experiences, it never finds the goal. I've tried poking around to figure out why, and all I can see that's weird is that the Q values continually increase as time goes on (while, without experiences, they settle to correct values).
Does anyone have any advice about things I could try? I've looked at the sampled experiences, the Q values in the experience loop, the gradient, etc... and nothing really sticks out, aside from the Q values growing.
edit: here's the code. The first part (one step TD learning) is working great. Adding the planning loop part screws it up.
def dynaQ(self, N_steps=100, N_plan_steps=5):

    self.initEpisode()
    for i in range(N_steps):
        #Get current state, next action, reward, next state
        s = self.getStateVec()
        a = self.epsGreedyAction(s)
        r, s_next = self.iterate(a)
        #Get Q values, Q_next is detached so it doesn't get changed by the gradient
        Q_cur = self.Q[s, a]
        Q_next = torch.max(self.Q[s_next]).detach().item()
        TD0_error = (r + self.params['gamma']*Q_next - Q_cur).pow(2).sum()
        #SGD
        self.optimizer.zero_grad()
        TD0_error.backward()
        self.optimizer.step()
        #Add to experience buffer
        e = Experience(s, a, r, s_next)
        self.updateModel(e)

        for j in range(N_plan_steps):

            xp = self.experiences[randint(0,len(self.experiences)-1)]
            Q_cur0 = self.Q[xp.s, xp.a]
            Q_next0 = torch.max(self.Q[xp.s_next]).detach().item()
            TD0_error0 = (xp.r + self.params['gamma']*Q_next0 - Q_cur0).pow(2).sum()

            self.optimizer.zero_grad()
            TD0_error0.backward()
            self.optimizer.step()

I cannot tell from your description what could be wrong. It may be some part of the implementation that you have not described or even a coding error. Perhaps link your code, in case someone feels like taking a look.

I have looked through the code, and cannot see any obvious mistake, although I am only just a beginner with PyTorch. It's a bit unusual to be using full gradient descent with loss functions etc on the tabular version of Q learning. Have you tried just doing self.Q[x.s, x.a] += self.params['alpha'] * (xp.r + self.params['gamma']*Q_next0 - Q_cur0) ? That's usually how tabular DynaQ would be formulated . . .

@NeilSlater thanks, I'll try it (that's what I used to do before using torch), but I really don't think it's that -- if I use torch to do the SGD without the experience/planning section, it works great.

What optimiser are you using? If you have something more complex than basic SGD it could cause problems - even something simple like momentum.

@NeilSlater I'm using RMSProp. I don't pass it any parameters besides the Q matrix to optimize. I just do self.optimizer = optim.RMSProp([self.Q]).

RMSProp tracks recent gradients and adjusts step sizes depending on what it expects to see on average across recent mini-batches. It might have trouble here, since your gradient is across the whole Q table, you don't use mini-batches, and only one item ever receives a non-zero gradient at a time. That might do odd things where RMSProp starts accumulating all those zeros for "batches" of not-yet visited state/action pairs and then suddenly receives a large update when it is finally visited.

I still suggest replace your RMSProp-based optimiser with the simpler update I showed in earlier comment, and see if the problem goes away. For a deterministic problem you can set a high value of $\alpha$, maybe even $1$


text:
Reinforcement learning for segmenting the robot path to reflect the true distances
I have a grid of rectangles acting as blocks. The robot traverses through the inter-spaces between these consecutive blocks. Now I have sensor data streaming in representing Right and left wheel speeds. Based on the differences in the speeds of the left and right wheels, I infer the robot's position and the path it has threaded. I get the associated individual segments of the total distance when it travels straight, left, or right.
These distances are a function of the actual speed of the robot and the time interval elapsed before the end of that activity. These computed distances for the segments though don't map and fit-in well when projected on the grid layout of the environment. The segments are rather not adhering to the boundary limitations.
I wanted to know if I can use RL to force the calculated distances to fit in with the layout given certain knowledge (or conditions, if you will): the start and end position of the robot and the inter-space distances.
If not RL, do you know how can I solve this problem? I suspect my function computing the distances is off and wondering if RL can help me figure out the right mapping of sensor data to the path traveled adhering to the grid layout dimensions.

If you consider the illustration above you will notice S, D, and D' signifying the starting position, the true destination, and the destination location computed by adding together the calculated distances for each of the segments representing right(r), left(l) and straight(s) along the path towards the destination. Inter-space length is given 7m and dimensions of the blocks are (27m x 15m). If you look at the data presented on the left side you will notice 18m left and consecutive 24m right represents the activity, in the grid, as the passage through the blocks. Granted -- perhaps the car negotiates the edges and corners through this passage in a protracted left(l) and right(r) movements, without necessarily going straight(s) straddling and linking the turns as one would expect.
The question arises, however, when taken into account these individual segment lengths and stitch them together you end up in a destination, not in the ballpark range of the expected value. How can we design this problem so as to employ RL methods to, sort of, impose these grid dimensional constraints on this distance calculation methodology to yield better results? Or, probably best to re-imagine the whole problem so it is amenable to the application of RL.
Any advice/ insights would be appreciated.

When I compute the segment distances as a function of speeds of the wheels, I get the values not really conforming to the dimensions of the grid blocks. I suspect the function employed is a little off (plausibly compounded by the rate of data streaming) and want to explore RL for its ability to learn from the given conditions such as the start, the destination and the grid inter-spaces in getting these distances in segments, the cars inferred to be undertaking, sit nice with the grid organization.

Thanks, I think I already understood that from your question. the function is not correct, and you want to have a method to correct it. However, you are not explaining how the discrepancy is being detected. How do you know the value is wrong, and is this something that could be automated? How are you detecting it - by watching the robot? How do you intend to measure the error in the function for learning to work? A learning process needs data

Thank you @NeilSlater. I tried illustrating the problem by adding the picture and the explanation in words. I hope it addresses some of the questions you raised and helps you understand the problem better.

I intend to get data on the start point and the true destination for a few cycles the car traversals. I shall then look for the dynamics and the patterns in the movement of the destination so as to simulate data, by extrapolating, for some more cycles as time progresses. So, we can suppose we have the true destination for quite a few cycles to enable RL to learn to correct the erroneous function that is throwing the output significantly off.
I was wondering if this problem can be solved using RL for a desired effect.

Thanks for adding the picture. It makes much more sense to me now.


text:
Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b
Sparse linear systems are normally solved by using solvers like MINRES, Conjugate gradient, GMRES.
Efficient preconditioning, i.e., finding a matrix P such that PAx = Pb is easier to solve than the original problem, can drastically reduce the computational effort to solve for x. However, preconditioning is normally problem-specific and there is not ONE preconditioner that works well for every problem.
I thought this would be an interesting problem to apply RL since there are certain norms (e.g. condition number of matrix PA) to measure if P is a good preconditioner, but I could not find any research in this field.
Is there a specific problem why RL could not be applied?

Why would you do that?

You are right, I was wondering about a different question and went to far. I will restate my question.

Reinforcement Learning is a method for learning to perform beneficial actions in an environment. One way this is accomplished is by learning to predict useful actions as a function of the observed state of the environment. Another is by learning to predict the expected utility gain of doing an action in a particular observed state. Usually the fact that the agent executes a sequence of actions is exploited to learn more rapidly.
In contrast, the problem you describe is to solve a linear system of equations, which is to say, to learn some hidden values $x$ such that $Ax = b$ for known $A$ and $b$. Gradient decent is a natural way to solve this problem because it is easy to calculate the gradient, and, since the matrices and vectors have the same dimensionality, it is reasonable to expect that the optimization surface is smooth with a single global minima.
While RL techniques could be applied to solve this problem (I guess by predicting values of X in response to inputs b?), none of the usual features that privilege the RL approach over a standard supervised learning approach are present (there's no sequential relationship between datapoints, there's just a list of constraints essentially).


text:
How to find the category of a technical text on a surface-semantic-level
There are some predefined categories( Overview, Data Architecture, Technical Details, Applications, etc). The requirement is to classify the input text of paragraphs into their resp. category. I can't use any pre-trained word embeddings (Word2Vec, Glove) because the data entered is not in general English ( talking about dogs, environment, etc) but pure technical (How does a particular program orks, steps to download anaconda, etc). Don't have any data available on the internet to train as well. Anything that understands semantic-surface-level of a sentence will work

Did you try a simple neural network with tf-idf features?

Problem Statement

Find the category of technical text on a surface-semantic-level.
The requirement is to classify the input text of paragraphs into their respective categories.

The categories given as predefined are as follows.

Overview
Data Architecture
Technical Details
Applications
etc

Some document types that would be added in place of 'etc' might include these.

Requirements
High Level Design
System Architecture
Testing Plan
Deployment Plan
Disaster Recovery Plan

Semantic Structure of Technical Documents
It is correct that the terminology between a requirements document and data architecture will likely have much in common.  More precisely, the distributions of linguistic elements of all documents common to a given system or project are likely to contain domain commonalities.  A system that pilots a drone will have the linguistic elements "drone(s)", "hover(ing|s)", "target(ing|s)", "flight plan(s)" in similar distributions throughout its documentations.
Indicators
It is likely that these five distinguishing characteristics can be exploited in categorization.

Sentence semantics
Inserted diagrammatic and pictorial conventions
Header conventions
Linguistic elements that appear commonly in only one type of technical document
Elements and structure from copying previous documents and modifying them or the use of boiler plates and templates in specific departments

Rather than focus entirely on text recognition and abandoning the other items in the list above would be unwise.  Diagrams, such as network diagrams and UML diagrams, may be quite easy to discern using deep convolutional approaches and would clearly identify the category to which the hosting document belongs.  That is also true of test case tables.
Recognizing that the proportions between the five indicators above and sections of indicators are variables in the model upon which training is applied will produce the best results.  For instance, the final paragraph may be more telling than the rest of the body.
Also, be aware that one can couple the proportional appearance of language elements within all text documents on record with the language elements identified in the example and then in later use of the trained model.  The training is likely to progress faster and produce more accurate and reliable results if features include an indication that the linguistic element "test case(s)" appearing in the input text is significantly less prevalent in the domain of technical documentation than the linguistic element "-ing" to indicate continuous tenses of verbs.
Avoid Static Grammars
Language parsers based on fixed language rules (grammars) have not had much success compared to association based semantic mapping, and linguistics has moved away from those static models for similar reasons.  Avoid grammar based parsing.
Existing Work
For the textual categorization, the below academic publications are some of the recent work that have already gained some notoriety.

Semantic clustering and convolutional neural network for short text categorization, P Wang, J Xu, B Xu, C Liu, H Zhang, F Wang… - Proceedings of the 53rd, 2015
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification, D Tang, B Qin, T Liu - Proceedings of the 2015 conference on empirical, 2015
Learning Semantic Representations of Users and Products for Document Level Sentiment Classification*, D Tang, B Qin, T Liu - Proceedings of the 53rd Annual Meeting of the, 2015
Effective use of word order for text categorization with convolutional neural networks, R Johnson, T Zhang - arXiv preprint arXiv:1412.1058, 2014 - arxiv.org
Jumping NLP curves: A review of natural language processing research, E Cambria, B White - IEEE Computational intelligence, 2014
Recurrent Convolutional Neural Networks for Text Classification, S Lai, L Xu, K Liu, J Zhao - AAAI, 2015

Faulty Approaches
The approach in the comment of comparing the semantics of sentences will only determine one aspect of information redundancy between two sentences.
The number of comparisons are also a consideration.  For $\chi$ documents each containing $\sigma$ sentences would require $\chi \, (\chi - 1) \, \sigma \, (\sigma - 1)$ comparisons.  For $\chi = 10,000$ documents containing $\sigma = 1,000$ sentences, we have 99,890,010,000,000 sentence comparisons, the totality of which provides no particularly useful information about the category of any of the documents.
The documents must be related to a concept class, not each other.
Visualizations of the semantics aren't particularly useful unless you are looking for something and the visualization is designed to present that which is sought.
A Better Plan

Determine the number of example training documents, $t$, needed to be categorized by experts to create a sufficiently large training data set. (Consider using the PAC learning framework designed for this purpose.)
Draw that number from the full set of documents, using an appropriate random or highly pseudo-random method, herein referred to as method $\mathbb{D}$. That is the training example set.
Draw that number from the full set of documents again, using method $\mathbb{D}$. That is the test example set.
Have experts label (categorize) both example sets according to technical document type.  Use the same experts for both training and testing, and have them alternate periodically between the two sets so that their learning, fatigue, or boredom curves have a minimal effect on their categorizing of the two sets.
Profile each of the category-labeled documents in terms of the above five indicators.  Note that to use the images embedded in the document as part of the profiling, which could drastically improve system reliability, the images must be run through a separate diagram categorizing network trained separately on images to find diagrams characteristic of one of the technical document types.  That may sound like much work, but consider that categorizing drawing types is a well developed science and the labeling is only of the diagram types that are representative of particular document types.
Train an appropriately designed artificial network to categorize the documents, using the profiling results as example inputs and the labels from the categorizing from the experts.
Test using the test set
Based on the results of the test, decide whether to use the current training or re-execute a previous step using information gained from the first training and train again
Run the trained network on the full document set
Pull a sample from the result using method $\mathbb{D}$ to validate effective completion of the run

I have tried semantic similarity between 2 sentences using Tensorflow. It gives a heat chart on the basis of how semantically similar 2 sentences are. I dont know however how to leverage that.. For eg: the sentence "Company X mainly acts as a broker between the customers and insurance providers." and the category "Overview" hardly have anything in common.
2.Inserted diagrammatic and pictorial conventions cant be used because, it is just a text document which needs to be categorized
3.I dont understand when you say "Header conventions"
Is there any way that we can use Clustering here?


text:
Sentence classification and named identity detection with automatic retraining
I am learning AI and trying out my first real-life AI application.
What I am trying to do is taking as an input various sentences, and then classifying the sentences into one of X number of categories based on keywords, and 'action' in the sentence.
The keywords are, for example, Merger, Acquisition, Award, product launch, etc. so in essence, I am trying to detect if the sentence in question talks about a merger between two organizations, or an acquisition by an organization, a person or an organization winning an award, or launching of a new product, etc.
To do this, I have made custom models based on the basic NLTK package model, for each keyword, and trying to improve the classification by dynamically tagging/updating the models with related keywords, synonyms, etc to improve the detection capability. Also, given a set of sentences, I am presenting the user with the detected categorization and asking whether it is correct or wrong, and if wrong, what is the correct categorization, and also identify the entities (company names, person names, product names, etc).
So the object is to first classify the sentence into a category, and additionally, detect the named entities in the sentence, based on the category.
The idea is, to be able to automatically re-train the models based on this feedback to improve its performance over time and to be able to retrain with as little manual intervention as possible. For the sake of this project, we can assume that user feedback would be accurate.
The problem I am facing is that NLK is allowing fixed-length entities while training, so for example a two-word award is being detected as two awards.
What should be my approach to solve this problem? Is there a better NLU (even a commercial one) that can address this problem? It seems to me that this would be a common AI problem, and I am missing something basic. I would love you guys to have any input on this.


text:
Can alpha-beta pruning be used for applications apart from games?
Can alpha-beta pruning/ minimax be used for systems apart from games? Like for selecting the right customer for a product, etc. (the typical data science problems)? I have seen people do it, but can't understand how. Can someone help me understand that?
Can I do something like if - find two criteria on which customers can buy product depends on like gender and age. Find the probability for all the customers depending on age and gender if they can buy it.
like if there are 3 customers - there probability to buy a product on the basis of their age and gender is - Customer 1 - (20%, 30%), Customer 2 - (30%, 60%), Customer 3 - (40%, 20%). here the x and y represents - (probability based on age, probability based on gender ). Probability is probability to buy the product.
For minimax, will it be correct if one player(max) tries to select the customer on basis of gender and other player(min) on basis of age. so, one can be max and one can be min.
Dont know if this correct or not, but just a idea.

Thinking about this more, the answer is in fact yes, but not for the application you mention.
You cannot use alpha-beta pruning to learn a model to predict customer outcomes, because it is only useful for domains where you are concerned about an adversary. In finding a customer model, there is no reason to worry about someone coming in and forcing you to make bad decisions about the optimization of the model. Consequentially, there is no reason to use minimax search, and thus, to use alpha-beta pruning.
There are applications other than (video) games where you could use these techniques though. For example, there are security games. In these "games" we want to use AI to find a strategy to protect an airport. It is reasonable to try and design our model under the assumption that someone else wants to break it. You could use Alpha-Beta pruning here (although in practice, more sophisticated algorithms are used).

So, just confirming. It wouldn't be wrong trying to solve a problem like finding right customer for a product using minimax/alpha-beta pruning the way I explained it?

@kritka It would be a little unusual, but there are ways to do it.

@kritka although the way you described sounds like decision tree induction, which is something else all together.

Just curious, what would be the ways?

Yes, I was thinking of comparing algorithms like decision tree induction, heuristic search, minimax search for the same problem l described.


text:
How to construct a model to predict the value of a time series $y_t$ that depends from other time series $\bar{X}_t$?
I would like to know what are the standard approach to construct a model to predict the value of a time series $y_t$ that depends from other time series $\bar{X}_t$. I use to see around that for this kind of task there is a large amount of models but all autoregressive in a way.
I'm thinking about, for example VAR, SARIMAX, RNN, LSTM.
I'm looking for a model, or at least an approach, where there is no my lagged target variable as predictors. Does anyone has some references ?

A time series forecast model based upon lagged variables of that time series is commonly called auto-correlation. A time series forecast model based upon other series is called cross-correlation in the time-series literature. A general forecast that uses any number of lagged time series (including the lagged series itself) is called vector auto regression.
Any of the other ML and AI approaches can use non-linear methods to achieve the modelling and draw from an infinite amount of factors other than lagged time series. In finance, it's common to use factor based investing, for example (like fundamental value data). In Machine Learning, we are interested in finding those features (feature selection) that best model the time series outcome.


text:
Why the partial derivative is $0$ when $F_{ij}^l < 0$?. Math behind style transfer
I am currently in the process of reading and understanding the process of style transfer. I came across this equation in the research paper which went like -

For context, here is the paragraph -

Generally each layer in the network defines a non-linear
filter bank whose complexity increases with the position of
the layer in the network. Hence a given input image is
encoded in each layer of the Convolutional Neural Network
by the filter responses to that image. A layer with $N_l$ distinct filters has $N$ feature maps each of size $M$ , where $M_l$
is the height times the width of the feature map. So the re-
sponses in a layer l can be stored in a matrix $Fl ∈ R^{N_l×M_l}$
where F l is the activation of the ith filter at position j in ij
layer l.
To visualise the image information that is encoded at
different layers of the hierarchy one can perform gradient descent on a white noise image to find another image that matches the feature responses of the original image (Fig 1, content reconstructions). Let $\vec p$ and $\vec x$ be the original image and the image that is generated, and $P^l$ and $F^l$ their respective feature representation in layer l. We then define the squared-error loss between the two feature representations
$\mathcal{L_{content}(\vec p, \vec x, l)} = {1\over 2} \Sigma_{i,j} \big(F_{ij}^l - P_{ij}^l \big)$. The derivative of this loss with respect to the activations in layer $l$ [the equation above $(2)$].

I just want to know why the partial derivative is $0$ when $F_{ij}^l < 0$.

Hello. Please, put your specific question in the title. "Math behind style transfer" is very vague/general and not a question. Thank you!

K I actually tried to sum up my entire Q in that one line - wasnt able to in the beginning XD. I'll do it soon

$F_l$ is the activation of the filter. They state in the paper that they base their method on VGG-Network, which uses ReLU as its activation function. In fact, VGG uses it in all of its hidden layers. ReLU is defined as
$$f(x) = max(0,x)$$
Since ReLU is 0 for all x's below 0, the equation above holds; When x is non-positive, all terms in the loss function are constants with respect to $F_{ij}^l$.


text:
Deepmind Spriteworld run_demo.py not found
I'm trying to run the Deepmind Spriteworld demo described on the project's GitHub page, but I'm not finding run_demo.py in the distribution and the closest sounding file, demo_ui.py doesn't launch a UI  when run (tried both on Linux and Windows).
How should the Deepmind Spriteworld demo UI be launched?

I cannot get it to run either, even when cloning and installing direct from the repo (when run_demo.py is available)

I posted the question to the project's issue page, where the recommendation was to manually download and run run_demo.py. That worked for me.


text:
CSP Formulation of an algebraic problem

Is anyone able to explain how to do this? I'm not looking for the complete answer, I would settle for a "how to for dummies" explanation of how this is supposed to be solved.
I understand constraints, but in the first example it would seem to me that the second half of the first partial assignment $x_2=-1$ is a violation of the constraint 2, that says $x_1 > x_2$... when down below it says both $x_1$, and $x_2$, are $-1$


text:
How to pad sequences during training for an encoder decoder model
I've got an encoder-decoder model for character level English language spelling correction, it is pretty basic stuff with a two LSTM encoder and another LSTM decoder.
However, up until now, I have been pre-padding the input sequences, like below:
abc  -> -abc
defg -> defg
ad   -> --ad

And next I have been splitting the data into several groups with the same output length, e.g.
train_data = {'15': [...], '16': [...], ...}

where the key is the length of the output data and I have been training the model once for each length in a loop.
However, there has to be a better way to do this, such as padding after the EOS character etc. But if this is the case, how would I change the loss function so that this padding isn't counted into the loss?


text:
Can you train Transformers sequentially?
I’m currently trying to train a BART, which is a denoising Transformer created by Facebook researchers. Here’s my Transformer code
import math
import torch
from torch import nn
from Constants import *

class Transformer(nn.Module):
    def __init__(self, input_dim: int, output_dim: int, d_model: int = 200, num_head: int = 8, num_e_layer: int = 6,
                 num_d_layer: int = 6, ff_dim: int = 1024, drop_out: float = 0.1):
        '''
        Args:
            input_dim: Size of the vocab of the input
            output_dim: Size of the vocab for output
            num_head: Number of heads in mutliheaded attention models
            num_e_layer: Number of sub-encoder layers
            num_d_layer: Number of sub-decoder layers
            ff_dim: Dimension of feedforward network in mulihead models
            d_model: The dimension to embed input and output features into
            drop_out: The drop out percentage
        '''
        super(Transformer, self).__init__()
        self.d_model = d_model
        self.transformer = nn.Transformer(d_model, num_head, num_e_layer, num_d_layer, ff_dim, drop_out,
                                          activation='gelu')
        self.decoder_embedder = nn.Embedding(output_dim, d_model)
        self.encoder_embedder = nn.Embedding(input_dim, d_model)
        self.fc1 = nn.Linear(d_model, output_dim)
        self.softmax = nn.Softmax(dim=2)
        self.positional_encoder = PositionalEncoding(d_model, drop_out)
        self.to(DEVICE)

    def forward(self, src: torch.Tensor, trg: torch.Tensor, src_mask: torch.Tensor = None,
                trg_mask: torch.Tensor = None):
        embedded_src = self.positional_encoder(self.encoder_embedder(src) * math.sqrt(self.d_model))
        embedded_trg = self.positional_encoder(self.decoder_embedder(trg) * math.sqrt(self.d_model))
        output = self.transformer.forward(embedded_src, embedded_trg, src_mask, trg_mask)
        return self.softmax(self.fc1(output))

class PositionalEncoding(nn.Module):
    def __init__(self, d_model, dropout=0.1, max_len=5000):
        super(PositionalEncoding, self).__init__()
        self.dropout = nn.Dropout(p=dropout)
        pe = torch.zeros(max_len, d_model)
        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
        div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
        pe[:, 0::2] = torch.sin(position * div_term)
        pe[:, 1::2] = torch.cos(position * div_term)
        pe = pe.unsqueeze(0).transpose(0, 1)
        self.register_buffer('pe', pe)

and here’s my training code
def train(x: list):
    optimizer.zero_grad()
    loss = 0.
    batch_sz = len(x)
    max_len = len(max(x, key=len)) + 1  # +1 for EOS xor SOS
    noise_x = noise(x)
    src_x = list(map(lambda s: [SOS] + [char for char in s] + [PAD] * ((max_len - len(s)) - 1), noise_x))
    trg_x = list(map(lambda s: [char for char in s] + [EOS] + [PAD] * ((max_len - len(s)) - 1), x))
    src = indexTensor(src_x, max_len, IN_CHARS).to(DEVICE)
    trg = targetsTensor(trg_x, max_len, OUT_CHARS).to(DEVICE)
    names = [''] * batch_sz

    for i in range(src.shape[0]):
        probs = transformer(src, trg[:i + 1])
        loss += criterion(probs, trg[i])

    loss.backward()
    optimizer.step()

    return names, loss.item()

As you can see in the train code. I am training it "sequentially" by inputting the first letter of the data then computing the loss with the output then inputting the first and second character and doing the same thing, so on and so forth.
This doesn’t seem to be training properly though as the denoising is totally off. I thought maybe there’s something wrong with my code or you can’t train Transformers this way.
I'm taking first name data then noising it then training the Transformer to denoise it, but the output to the Transformers doesn't look remotely like the denoised version or even the noised version of the name. I built a denoising autoencoder using LSTMs and it did way better, but I feel like BART should be way out performing LSTMs cause it's supposedly state of the art NLP neural network model.

I think you should be more specific when you say "My Transformer is not training well at all".

The loss which is cross entropy is going all over the place, it's not decreasing at all.

What exactly do you mean by "sequentially"? Do you mean "online" (i.e. you will train it a little bit more as more data arrives) or you will train it with "sequences", or maybe something else?

I assume you're referring to this paper, which basically "reuses" the same architecture as the original "Attention is all you need" paper for machine translation; the difference being that source and target are noisy and original sentences.
I can't figure the for loop logic inside your train function; but reading your comment, you don't have to traverse the target sequentially. Instead, we rely on masks which exclude paddings, and most importantly help with implementing teacher forcing for the decoder stack. Check this post and how it creates the three types of masks needed (src, target, no_peek).
Having said that, you will need a for loop when serving your model (inference), because you will have to decode autoregressively. That is, you'll have to decode a token first, and feed it back to the decoder to generate the next token (until you reach EOS). Check "Testing the model" section in this article.


text:
How to alpha-beta pruning to expectiminimax

I have this problem above and I'm trying to think of how to apply alpha-beta pruning to the above. The algorithm states that on you're opponents turn the (expecti turn) you just return the lowest value, but does that mean you apply the probabilities to those values? So for the far left you'd get 2 as the largest value then multiply that by 0.5, but then that set's $\\beta$ in the expecti node to $0.5*2=1$ and when it goes into the branch to the right it's comparing values without the probabilities applied to it when updating $\beta$.

I don't understand the current title "How to alpha-beta pruning to expectiminimax" because a "verb" is missing. Please, edit your post to fix this.

I know I'm a bit late, but here is a response from the book "Artifical Intelligence, A Modern Approach" from Russel and Norvig: "It may have occured to you that something like alpha-beta pruning could be applied to game trees with chance nodes. It turns out that you can. The analysis for MIN and MAX nodes is unchanged, but we can also prune chance nodes, using a bit of ingenuity. At first sight, it might seem impossible to find an upper bound on the value of a chance node before looking at all its children, because the value of a chance node is the average of it's children's value, and in order to compute the average of a set of numbers, we must look at all the numbers. But if we put bounds on the possible values of the utility function, then we can arrive at bounds for the average without looking at every number. For example, say that all utility values are between -2 and +2; then the value of leaf nodes is bounded, and in turn we can place an upper bound on the value of a chance node without looking at all its children".

This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review

As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

The internet doesnt seem to have easily accessible resources on this topic. In my newbie opinion, it should not be possible to prune any nodes. Expectiminimax takes the weighted average of the children, so it would need to consider all leaf values in order to do so. There is no node which can be ignored, as it will an operand in a sum. Now lets say the left subtree was bigger, in the right subtree you evaluated the first min node and it is smaller: well since the second min node can contribute to the sum making it bigger, it cannot be ignored. Now lets say the first min node on the right subtree was bigger, well since the second term adds or reduces(reduces in this case) weight it still has to be considered to get the final value.
Since all the nodes are checked and added together with appropriate weights under chance nodes, I highly doubt there is any advantage of alpha-beta pruning here since it would take the same number of steps as regular expectiminimax. Perhaps if the tree was larger and the chance nodes were central, more nodes could have been pruned out.

This is dead wrong. If we know the maximum and minimum value a node can have, we can estimate the max/min expected value of a chance node and prune there.

@thecoolwinter I am talking about the question at hand, not in general. If you have an answer, please provide it. Except you cant because this question cannot be solved by pruning

@thecoolwinter I am talking about the question at hand, not in general. If you have an answer, please provide it. Except you cant because this question cannot be solved by pruning

alpha-beta-prunning algorithm is using for improve performance and reject the options, which not consist the condition - or it's possible to set some factors of probability to take into consideration or not.
This algorithm work on tree structure - and if there are a lot of levels (10-20) - It allows you to eliminate paths - which will logically not be used - saving memory and computing resources.
In this particular case - for finding the minimum value it works like this:
First branch:

Go to B
Go to D - and there is 2 and 3 - so return the min 2
Go From B to E - and choose 5 - the minimal value in B points is actually 3 - so there isn't need for checking the next - cause everything below E, will be higher than D (3)

Second branch:

Go to
Go to F - and check 0 and 1
If C have 1 - than not necessary to go to G - cause 1 is the smaller.

The given sequence is a simplification - and in the case of this algorithm there are also layers - min & max

Source:
https://www.javatpoint.com/ai-alpha-beta-pruning#:~:text=Key%20points%20about%20alpha%2Dbeta,values%20to%20the%20child%20nodes.

Implementation, however, is a more complex matter - unless we do copy and paste :-)
https://gist.github.com/exallium/1446104/5109388cfc21578f555dcac0ba54da680326af7b

This answer describes the application of alpha-beta pruning to minimax, not to expectiminimax.

Hi, Whats the difference?

Alpha-beta pruning for minimax doesn't specify what to do with chance nodes.


text:
Why does this formula $\sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$ approximate the variance?
How does:
$$\text{Var}(y) \approx \sigma^2 + \frac{1}{T}\sum_{t=1}^Tf^{\hat{W_t}}(x)^Tf^{\hat{W_t}}(x_t)-E(y)^TE(y)$$
approximate variance?
I'm currently reading What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision, and the authors wrote the above formula for the approximate estimation for the variance. I'm confused how the above is an approximation for $\frac{\sum(y-\bar{y})^2}{N-1}$. So, in the above equation, they're using a Bayesian Neural Network to quantify uncertainty. $\sigma$ is the predictive variance (kind of confused how they get this). $x$ is the input and $y$ is the label for the classification. $f^{\hat{W_t}}(\cdot)$ output a mean to a Gaussian distribution, with $\sigma$ being the SD for that distribution and $T$ is a predefined number of samples because the gradient is evaluated using Monte Carlo sampling.

@DuttaA sadly they don't give dimensions. I think the reason being you can adjust the model size to fit whatever you need, so there's no strict requirement in terms of dimensionality. If you like you can just think of them as vectors I think that's reasonable.

dot product maybe?

it's actually a little confusing I'm looking at the paper right now. They denote $T$ as MC samples, but then they also use the same style of $T$ over $f()$ and $E(Y)$, I assume it's transpose, but maybe they actually are putting real values to the power of $T$? That doesn't really make sense though.

@user8714896 You can try contact the authors.

If this is from equation (4), you've got a typo and the transpose in the sum term should be on the other element (so it is an inner product, and the dimensions are at least plausible).

@htl You can and should propose an edit to the post if you think there are typos there.

Why did you delete this post? I was trying to write an answer, although that answer could not be satisfactory. In any case, I don't know when I will finish writing it.

@nbro go ahead and put an answer. I posted on stats stack exchange to see if stats people had any intuition about this, that's why I deleted it, but I'm eagerly waiting to see what you write up.


text:
Computation of initial adjoint for NODE
I'm reading the paper Neural Ordinary Differential Equations and I have a simple question about adjoint method. When we train NODE, it uses a blackbox ODESolver to compute gradients through model parameters, hidden states, and time. It uses another quantity $\mathbf{a}(t) = \partial L / \partial \mathbf{z}(t)$ called adjoint, which also satisfies another ODE. As I understand, the authors build a single ODE that computes all the gradients $\partial L / \partial \mathbf{z}(t_{0})$ and $\partial L / \partial \theta$ by solving that single ODE. However, I can't understand how do we know the value $\partial L / \partial \mathbf{z}(t_1)$ which corresponds to the initial condition for the ODE corresponds to the adjoint. I'm using this tutorial as a reference, and it defines custom forward and backward methods for solving ODE. However, for the backward computation (especially ODEAdjoint class in the tutorial) we need to pass $\partial L / \partial \mathbf{z}$ for backpropagation, and this enables us to compute $\partial L / \partial \mathbf{z}(t_i)$ from $\partial L / \partial \mathbf{z}(t_{i+1})$, but we still need to know the adjoint value $\partial L / \partial \mathbf{z}(t_N)$. I do not understand well about how pytorch's autograd package works, and this seems to be a barrier to understand this. Could anyone explain how it operates, and where $\partial L / \partial \mathbf{z}(t_1)$ (or $\partial L / \partial \mathbf{z}(t_N)$ if this is more comfortable) comes from? Thanks in advance.

Here's my guess for the initial adjoint from simple example. Let $d\mathbf{z}/dt = Az$ be a 2-dim linear ODE with given $A \in \mathbb{R}^{2\times 2}$. If we use Euler's method as a ODE solver, then the estimate for $z(t_1)$ is explicitly given as $$\hat{\mathbf{z}}(t_1) = \mathrm{ODESolve}(\mathbf{z}(t_0), f, t_0, t_1, \theta))= \left(I + \frac{t_1 - t_0}{N}A\right)^{N} \mathbf{z}(t_0) $$ where $N$ is the number of steps for Euler's method (so that $h = (t_1 - t_0) /N$ is the step size). If we use MSE loss for training, then the loss will be
$$
L(\mathbf{z}(t_1)) = \Bigl|\Bigl| \mathbf{z}_1 - \left(I + \frac{t_1 - t_0}{N}A\right)^N\mathbf{z}(t_0)\Bigr|\Bigr|_2^2
$$
where $\mathbf{z}_1$ is the true value at time $t_1$, which is $\mathbf{z}_1 = e^{A(t_1 - t_0)}\mathbf{z}(t_0)$. Since adjoint $\mathbf{a}(t) = \partial L / \partial \mathbf{z}(t)$ satisfies $$\frac{d\mathbf{a}(t)}{dt} = -\mathbf{a}(t)^{T} \frac{\partial f(\mathbf{z}(t), t, \theta)}{\partial \mathbf{z}} = \mathbf{0},$$
$\mathbf{a}(t)$ is constant and we get $\mathbf{a}(t_0) = \mathbf{a}(t_1)$. So we do not need to use augmented ODE for computing $\mathbf{a}(t)$. However, I still don't know what $\mathbf{a}(t_1) = \partial L / \partial \mathbf{z}(t_1)$ should be.  If my understanding is correct, since $L = ||\mathbf{z}_1 - \mathbf{z}(t_1)||^{2}_{2}$, it seems that the answer might be
$$
\frac{\partial L}{\partial \mathbf{z}(t_1)} = 2(\mathbf{z}(t_1) - \mathbf{z}_1).
$$
However, this doesn't seem to be true: if it is, and if we have multiple datapoints at $t_1, t_2, \dots, t_N$, then the loss is
$$
L = \frac{1}{N} \sum_{i=1}^{N}||\mathbf{z}_i  -\mathbf{z}(t_i)||_{2}^{2}
$$
and we may have
$$
\frac{\partial L}{\partial \mathbf{z}(t_i)} = \frac{2}{N} (\mathbf{z}(t_i) - \mathbf{z}_i),
$$
which means that we don't need to solve ODE associated to $\mathbf{a}(t)$.

First, a forward pass is done to obtain predictions of $z$, at every $t$. Then the adjoint state is run backward in time for every $t$. Which gives the learning impulse. So an initial run is done to obtain values of the dynamical system at all time points and the last of these is the initial point for the backward pass.
This picture depicts it.

To be more specific with regards to the question in your initial paragraph the input of $t_0$ and $t_1$ into $ODE\_Solve$ are time points at which the network is evaluated. In this specific case, the first point is $t_0$ which should always be the case and $t_1$ is the last point. But you can also insert more time points for example $N$ time points than you input the sequence $t_0, t_1, t_2, ..., t_N$ into $ODE\_Solve$. In that case, $t_N$ is the last time point.
This picture also shows a lot. But in this picture, a RNN is used to generate $z_{t_0}$. The RNN encoder is given the time points in reversed order: $(t_N, t_{N-1}, t_{N-2}, ..., t_0)$.

This is probably not enough to satisfy your question, hopefully, I can get more specific with respect to the math later.


text:
Why do small datasets require more samples, while big datasets require fewer samples in negative sampling?
In the deep learning specialization course by Andrew Ng, in the video Sequence Models (minute 4:13), he says that in negative sampling we have to choose a sample of words from the corpus to train rather than choosing the whole corpus. But he said that, for smaller datasets, we need a bigger number of samples, for example, 5-20, and, for larger datasets, we need a smaller sample, for example, 2-5. By sample, I am referring to the number of words along with the target word we have taken to train the model.
Why do small datasets require more samples, while big datasets require fewer samples?

I think your title is a bit misleading. The way I read it was that you need to have more samples in small datasets, while you are referring to the negative samples in negative sampling (not dataset samples). Also your tags seem a bit irrelevant, especially the one about CNNs. I suggest modifying your question a bit so that there isn't any confusion and you can get a proper answer.

@Djib2011 The current title is the result of my edit, because the previous one was not very descriptive. If you have an idea for a better title, please, edit the question or propose an edit.

He likely found this to a be a best practice to avoid over fitting, with a small data set if you only use small and easy to learn (less words -> less degrees of freedom) sequences then you open your model to the risk of over fitting that data set where as on a large data set that has alot more total information you can train on small sequences without being at risk of over fitting because although the smaller sequences will be easier to learn the variance of sequences will be much higher.


text:
How to learn using DDPG in python solely using a timeseries datasets
I have a lengthy timeseries datasets which contains several variables (from sensors etc) to be classified as actions or states. Providing they are successfully done, I want to learn a control policy using DDPG.
But I have no knowledge of the environment.
How can I learn my policy off-line only by using these datasets without having any model of the environment? After learning off-line first, then the policy can then be used to learn and control online later in a certain real-world environment.
First, I know that experience buffer can be used to store the datasets. How should you set the buffer size in this case?
From what I understand, DDPG needs lots of data to be used for learning.
Should I build an environment model using the specified datasets? Or I don't really need this step?
All of these will be implemented in Python and maybe with the help of another tools if needed. There are some implementation of DDPG available so it is not the main problem, but this implementation must be tweaked to solve my proposed problem. Normally the implemented DDPG in Python requires a Gym-environment as an input so I must change it to satisfy my needs as I don't need Gym for my use case. And these implementations in Python are somehow on-line codes so you need to interact directly with the environment model for the algorithm to be working.
Can someone help me tackle this  problem or give me some advice regarding this? I can help giving more details if needed. Thank you.
Regards


text:
Understanding Batch Normalization for CNNs
I am trying to understand how batch normalization (BN) works in CNNs. Suppose I have a feature map tensor $T$ of shape $(N, C, H, W)$
where $N$ is the mini-batch size,
$C$ is the number of channels, and
$H,W$ is the spatial dimension of the tensor.
Then it seems there could a few ways of going about this:
Method 1: $T_{n,c,x,y} :=  \gamma*\frac {T_{c,x,y} - \mu_{x,y}} {\sqrt{\sigma^2_{x,y} + \epsilon}} + \beta$ where $\mu_{x,y} = \frac{1}{NC}\sum_{n, c} T_{n,c,x,y}$ is the mean for all channels $c$ for each batch element $n$ at spatial location $x,y$ over the minibatch, and
$\sigma^2_{x,y} = \frac{1}{NC} \sum_{n, c} (T_{n, c,x,y}-\mu_{c})^2$ is the variance of the minibatch for all channels $c$ at spatial location $x,y$.
Method 2: $T_{n,c,x,y} :=  \gamma*\frac {T_{c,x,y} - \mu_{c,x,y}} {\sqrt{\sigma^2_{c,x,y} + \epsilon}} + \beta$ where $\mu_{c,x,y} = \frac{1}{N}\sum_{n} T_{n,c,x,y}$ is the mean for a specific channels $c$ for each batch element $n$ at spatial location $x,y$ over the minibatch, and
$\sigma^2_{c,x,y} = \frac{1}{N} \sum_{n} (T_{n, c,x,y}-\mu_{c})^2$ is the variance of the minibatch for a channel $c$ at spatial location $x,y$.
Method 3: For each channel $c$ we compute the mean/variance over the entire spatial values for $x,y$ and apply the formula as
$T_{n, c,x,y} := \gamma*\frac {T_{n, c,x,y} - \mu_{c}} {\sqrt{\sigma^2_{c} + \epsilon}} + \beta$, where now $\mu_c = \frac{1}{NHW} \sum_{n,x,y} T_{n,c,x,y}$ and $\sigma^2{_c} = \frac{1}{NHW} \sum_{n,x,y} (T_{n,c,x,y}-\mu_c)^2 $
In practice which of these methods is used (if any) are correct for?
The original paper on batch normalization , https://arxiv.org/pdf/1502.03167.pdf , states on page 5 section 3.2, last paragraph, left side of the page:

For convolutional layers, we additionally want the normalization to
obey the convolutional property – so that different elements of the
same feature map, at different locations, are normalized in the same
way. To achieve this, we jointly normalize all the activations in a
minibatch, over all locations. In Alg. 1, we let $\mathcal{B}$ be the set of all
values in a feature map across both the elements of a mini-batch and
spatial locations – so for a mini-batch of size $m$ and feature maps of
size $p \times q$, we use the effective mini-batch of size $m^\prime = \vert \mathcal{B} \vert = m \cdot pq$. We learn a pair of parameters $\gamma^{(k)}$ and $\beta^{(k)}$ per feature map,
rather than per activation. Alg. 2 is modified similarly, so that
during inference the BN transform applies the same linear
transformation to each activation in a given feature map.

I'm not sure what the authors mean by "per feature map", does this mean per channel?

Are you proposing the three methods or are you pulling them out of a document?  If you are pulling them from somewhere else please cite them. Also, you want the variance and not the standard deviation in the denominator.

@BrianO'Donnell edited.


text:
Best way to train Neural Network for Voice Commands?
I want to build a Voice Assistance using Tensorflow, Like Google Assistance, So that I can give commands like:
Open Camera
Send Message
Play Music
ETC

I know I can use pre-trained model for Voice Recognition but this is not my problem. If I know correctly, What Neural Network does it learn from your input and output and creates a best algorithms.
So I want to know Is it possible to somehow train my network for my commands so that I don't have to HARD-CODED them because it is hard to remember so many command?
Is Google also Hard-Coded the commands for Google Voice Assistance?
Sorry, If I'm unable to explain this to you :P

What Neural Network does it learn from your input and output and creates a best algorithms.

This is inaccurate. A neural network is a function approximator, so it approximates an unknown function. However, it does so in many cases by learning from your input and output(as in the case of supervised learning). The function your approximating would here for example be the function that maps a certain soundwave input to "open camera", etc.
The problem with supervised learning is that you need massive datasets to accurately approximate the target function - luckily for Google they have such datasets and resources available.

Is Google also Hard-Coded the commands for Google Voice Assistance?

No, because that would not generalize well over multiple languages/accents/background noise, etc. It is therefor that neural networks as function approximators have had such a large impact over the last years - they generalize well.
If you remember when the first "voice assistance" products came out, they rarely functioned - this was often due to them being "hard coded", with only static measures for noise-suppression.

Thanks Krrl, I know they uses ML for supports multiple languages / accents etc, but I want to know about their functionality not Speech Recognition.
Like If I said "Open WhatsApp" then obliviously ML convert that in Text which google uses that specific word like Open  & WhatsApp then hey Call some function to open whatsApp. I want to know is That Function for opening whatsApp is Hard-Coded or just their ML automatically learn how to open a specific App. Sorry if you think this is wired question :P

I dont think it is a weird question, but I also am not sure if I understand what you are asking correctly. I would think that their ML algorithm would have to use the app's specific API for the action of opening the app. I also think that they have different methods for the two different tasks - perhaps one method to extract the words, and one method to "understand" the extracted words(semantically analyze them), but I am not sure of their specific architecture here, sorry.


text:
Confusion Matrix Measures vs Accuracy level in Neural Network Model
I'm working on a classification machine learning problem with two classes: high and low, which are derived from another numerical column x. Previously, if x>100, the sample is considered high, otherwise, it is considered low. I used a 1D CNN model.
I wanted to test if changing the threshold would impact on the model performance. So I increased the threshold to 1000(ie. if x>1000, the sample is considered high, otherwise, it is considered low.
).
What I did:

Increased the threshold from 100 to 1000, thus the number of samples that fall into the low category increased and that of the high category decreased. The data thus become imbalanced with the following ratio:

Ratio of low : high

(with 100 threshold)       43% : 57%

(with 1000 threshold)       8% : 92%

Noticing that the imbalanced dataset might lead to imbalanced training data, I applied undersampling to the training data to make sure that there are equal number of low and high samples.
At 100 threshold, the model achieved an accuracy level of around 80%. At 1000 threshold, the model achieved an accuracy level of around 69%, which is significantly lower than that of the 100 threshold.
I then saved each model and generated a prediction using each model. I evaluated both sets of results with the Precision, Recall and F1 Score.

                       Precision: 0.38
(with 100 threshold)   Recall: 0.86
                       F1: 0.527

                       Precision: 0.8307692307692308
(with 1000 threshold)  Recall: 0.6206896551724138
                       F1: 0.7105263157894737

Based on accuracy level, it seems 100 threshold achieved a better performance than that of 1000 threshold. However, according to F1 Score, 1000 threshold seems to achieve better score 0.71 (as compared to 0.527 of 100 threshold).
My question are:

Is it possible for a model to have lower accuracy level but higher F1 Score? How can this make sense in light of confusion-matrix measures and model accuracy level?
Which measures should I use to determine using which threshold is better?

Edit:
Since the entire code is too long, I will show some snippets of it below.
To answer @MASTER OF CODE's question, I believe the test data is balanced and tested in the validation stage at the end of each epoch as I called model.fit(x_train,y_train, validation) in Keras which used the built-in API for training and evaluation(please cmiiw).
For Undersampling:
I used np.random.choice() to randomly select an equal amount of majority samples as the minority group, then concatenated the minority and majority samples into one dataframe named under_sample, before fitting a 1D CNN model on the dataframe, see below:
np.random.seed(42)
############ minority_class_len
minority_class_len = len(df[df['Label'] ==1])

############ majority_class_indices
majority_class_indices = df[df['Label'] ==0].index

############ random_majority_indices
random_majority_indices = np.random.choice(majority_class_indices,
                                           minority_class_len,
                                           replace = False)

############ minority_class_indices
minority_class_indices = df[df['Label'] ==1].index

############ concatenate positive and negative sample indices
under_sample_indices = np.concatenate([minority_class_indices, random_majority_indices])

############ select samples by indices
under_sample = df.loc[under_sample_indices]
```

Is Your testing set also balanced? Also could You show some code to recreate this behaviour?

Hi @MASTEROFCODE, thanks for your reply, please see question edit. Unfortunately I'm unable to upload the entire code but I added some snippets, hopefully it makes it a bit clear.

1. Yes, accuracy can be lower than the F score for a set of predictions and targets. You can easily test it by generating a fixed array of predictions and a varying array of labels (or vice versa). You'll get a graph like in the image below.
Notice that you could obtain a horizontally mirrored version of the same graph by inverting the labels (or predictions).
To understand why you need to keep in mid that the F score is a skewed metric,because in its calculation we ignore an entire class, i.e. the true negatives.
$$F1= \frac{2 TP}{2TP + FP + FN} $$
So if I have an unbalanced dataset with only 10 instances of class A and 90 of class B I'll get an accuracy of 90% by predicting only B, with an f score of 0%, but if I invert the classes (i.e. predict only A) I'll get an accuracy of 10% and an F score of 18%.
2. The safer answer is always: check all metrics. Every metric tells you a different story about the model performance, so it's always a good practice to not rely only on one. But it might depends also on your specific use case. Why do you need to classify between low and high (whatever low and high refer to)? Do you care about a specific class? Then the recall is the most interesting metric to keep an eye on. In general, your problem is similar to probability thresholding when calculating the ROC-AUC score so you might want to check it out and see if it's applicable also to dataset splitting (a bit impractical if a single training session of your model is long though), if it's applicable, it's surely more reliable than the F score.


text:
Does a differential evolution algorithm mutate its population during a generation?
I'm implementing a differential evolution algorithm and when it comes to evolving a population, the page I am referencing is vague on how the new population is generated.
https://en.wikipedia.org/wiki/Differential_evolution#Algorithm
The algorithm looks like the population is mutated during evolution.
# x, a, b, c are agents in the population
# subscriptable to find position in specific dimension
# pop is filled with Agent objects
for i, x in enumerate(pop):
  [a, b, c] = random.sample(pop[:i]+pop[i+1:], 3)
  ri = random.randint(0, len(x))
  new_position = []
  for j in range(len(x)):
    if random.uniform(0, 1) < CR or j == ri:
      new_pos.append(a[j] + (F * (b[j] - c[j])))
    else:
      new_pos.append(x[j])
  # Agent() class constructor for agent, takes position as arg
  new_agent = Agent(new_pos)
  if fitness(new_agent) <= fitness(x):
    pop[i] = new_agent # replace x with new_agent

But I wonder if instead it means a new population is made and then populated iteratively:
new_pop = []
for i, x in enumerate(pop):
  [a, b, c] = random.sample(pop[:i]+pop[i+1:], 3)
  ri = random.randint(0, len(x))
  new_position = []
  for j in range(len(x)):
    if random.uniform(0, 1) < CR or j == ri:
      new_pos.append(a[j] + (F * (b[j] - c[j])))
    else:
      new_pos.append(x[j])
  new_agent = Agent(new_pos)
  if fitness(new_agent) <= fitness(x):
    new_pop.append(new_agent)
  else:
    new_pop.append(x)
pop = new_pop

Note new_pop is made, and filled with agents as the for loop continues.
The first allows previously evolved agents to be used again in the same generation; in other words, the population is changed during the evolution. The second doesn't allow updated agents to be re-used, and only at the end is the original population changed.
Which is it?

It may be a good idea to provide the sources of these pseudocodes.

Quoting the original paper:

For each target vector $x_{i,G}$ ,a mutant vector is generated according to
$$ v_{i,G+1} = x_{r_1,G} + F\left(x_{r_2,G} + x_{r_3,G}\right)$$

And later

To decide whether or not it should become a member of generation $G + 1$, the trial vector $v_{i,G+1}$ is compared to the target vector $x_{i,G}$ using the greedy criterion.

I'd say it is pretty unambiguous that author's intent was to split evolving agents' populations into "generations" indexed by $G$. A new generation $G+1$ is created by applying the DE algorithm to the previous generation $G$. No agents are changed within the generation $G$.
So, it looks like the answer to your question is "no" - the population of the current generation $G$ is not mutated. Your second approach is the correct one.

Some metaheuristics, like particle swarm optimization and bat algorithm, mutate their populations during the current generation; however, some like genetic algorithm doesn't. Thanks for clarifying, it seemed like it's closer in metaphor to the ones that do but it's nice having a declarative answer.


text:
How to use NN to generate a model which produces given distributions?
For a non-Markovian random walk, each step can go up or down. And for the $n-th$ step, its step size $s(n)$ may depend on the path of walk, and the probability for going up or down may also depend on the path of walk.
Q: Assume that the distribution of this random walk approach a given distribution, how can we use NN to find such a random walk which can generate such expected distribution ?


text:
Do we have anything like accuracy and loss in RNN models?
I have a paper about trading which has been implemented with RNN on Tensorflow. We have about 2 years of data from trading. Here are some samples :
Date, Open, High, Low, Last, Close, Total Trade Quantity, Turnover (Lacs)
2004-08-25 , 1198.7, 1198.7, 979.0, 985.0, 987.95, 17116372.0, 172587.61
2004-08-26 , 992.0, 997.0, 975.3, 976.85, 979.0, 5055400.0, 49828.65
I need to predict the the future of trading (for example, the latest 10 days ). So, how can I make sure that my model is working correctly. Do we have any "accuracy" or "loss" like what we have in Deep Learning?

RNN's stand for Recurrent Neural Networks which is, in fact, Deep Learning.
There has to be a loss since you're dealing with supervised learning and the typical loss metrics used are the same as you would see in feedforward networks (usually binary cross-entropy), the main difference being loss would be calculated between the true label at a particular time stamp $(t)$ and the prediction made from the subset of the network until time-stamp $(t-1)$. This leads the loss to act on all timestamps.
Accuracy metrics also would be used in the same way such as Mean Square Error or L1. For more details you can go through this link.
Hope this was helpful!

Yes, it was really helpful and I have another question relating to RL.
If I want to reach the maximum total return (profit), according to what we have in Reinforcement Learning and regarding this kind of data set (Stock or Trading), can we do that? If yes, how can we make sure that the return value is the best total maximum return .

Sorry to say I can't say much when it comes to RL. I think it'd be best if you were to ask this in another question. It would make it visible to people who can answer this satisfactorily


text:
What's the value of making the RL agent's output stochastic opposed to deterministic?
I have a question about a reinforcement learning problem.
I'm training an agent to add or delete pixels in a [12 x 12] 2D space (going to be 3D in the future).
Its action space consists of two discrete outputs: x[0-12] and y[0-12].
What would be the value of instead outputting a (continuous) probabilistic output representation, like the [12 x 12] space with each pixel as a probability, and sampling from it. E.g. a softmax function applied to 144 (12*12) output nodes.
My environment is deterministic itself: taking action  in state  always results in the same next state ′.
I understand that this may be more difficult to train since the output space becomes continuous instead of discrete, and therefore bigger, but does stochastic/probabilistic output have any benefits over 1 discrete output?
Thanks!

My understanding of your question is, you have 2 designs:

A deterministic policy that outputs 2 scalar for x and y respectively.
A value function that outputs the probability of each pixel in the 2D grid.

If you choose the max of softmax on (2.), you'll get the same deterministic policy as (1.), assuming there are some tie-breaking designs. So I don't think choosing the max of softmax is a great policy.
So, the question becomes "deterministic policy vs. value function". Since the value function have more information, we can have special designs on exploration.
In (1.), if you use $\epsilon$-greedy on the discrete output, It'll take a random move with $\epsilon$ probability. When exploring (instead of exploiting), all actions except the best action will have the same probability.
However, in (2.), the probability can act like a value function (the value of $Q(s, a) \forall a$ in a certain state $s$), and you can use something like upper confidence bound (UCB) instead of $\epsilon$-greedy. If you are using neural networks to output this probability distribution, you can add an entropy term in the loss function like in A3C to encourage exploration.

That makes sense. Only, If you use softmax on (2.), and do a random sample on that distribution, you do get a non-deterministic output right? Or am I incorrect?

@SumakuTension Oh yes, I meant choosing the max value of the softmaxed output isn't a good policy. Performing a random sample on the distribution does make it non-deterministic.

I've edited the answer, thanks for the correction!


text:
Ideas on a network that can translate image differences into motor commands?
I'd like to design a network that gets two images (an image under construction, and an ideal image), and has to come up with an action vector for a simple motor command which would augment the image under construction to resemble the ideal image more. So basically, it translates image differences into motor commands to make them more similar?
I'm dealing with a 3D virtual environment, so the images are snapshots of objects and motor commands are simple alterations to the 3D shape.
Probably the network needs two pre-trained CNNs with the same weights that extract image features, then output and concatenate those into a dense layer (or two), which converges into action-space. Training should probably happen via reinforcement learning
Additionally, in the end it needs recurrence, since there are multiple motor actions it needs to do in a row to get closer to the intended result.
Would there be any serious difficulties with this? Or are there any approaches to achieve the intended result? or any similar examples?
Thanks in advance

Is there an obvious/detectable endpoint for a sequence of actions (e.g getting within some limit of ideal), or an enforced one (such as max number of steps) - or both, or is this issue open in your design?

This is still an open issue in my design. Probably starting more basic, with an enforced one of a few steps to see how it performs


text:
TensorFlow fit() and GradientTape - number of epochs are different
if I define the architecture of a neural network using only dense fully connected layers and train them such that there are two models which are trained using model.fit() and GradientTape. Both the methods of training use the same model architecture.
The randomly initialized weights are shared between the two models and all other parameters such as optimizer, loss function and metrics are also the same.
Dimensions of training and testing sets are:
X_train = (960, 4), y_train = (960,), X_test = (412, 4) & y_test = (412,)
import pandas as pd, numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import accuracy_score, precision_score, recall_score, confusion_matrix

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
import tensorflow_model_optimization as tfmot
from tensorflow_model_optimization.sparsity import keras as sparsity

def create_nn():
    """
    Function to create a
    Neural Network
    """
    model = Sequential()

    model.add(
        Dense(
            units = 4, activation = 'relu',
            kernel_initializer = tf.keras.initializers.GlorotNormal(),
            input_shape = (4,)
        )
    )

    model.add(
        Dense(
            units = 3, activation = 'relu',
            kernel_initializer = tf.keras.initializers.GlorotNormal()
        )
    )

    model.add(
        Dense(
            units = 1, activation = 'sigmoid'
        )
    )

    """
    # Compile the defined NN model above-
    model.compile(
        loss = 'binary_crossentropy',  # loss = 'categorical_crossentropy'
        optimizer = tf.keras.optimizers.Adam(lr = 0.001),
        metrics=['accuracy']
    )
    """

    return model

# Instantiate a model- model = create_nn()

# Save weights for fair comparison- model.save_weights("Random_Weights.h5", overwrite=True)

# Create datasets to be used for GradientTape-
# Use tf.data to batch and shuffle the dataset train_ds = tf.data.Dataset.from_tensor_slices(
    (X_train, y_train)).shuffle(100).batch(32)

test_ds = tf.data.Dataset.from_tensor_slices(
    (X_test, y_test)).shuffle(100).batch(32)

# Define early stopping- callback = tf.keras.callbacks.EarlyStopping(
    monitor='val_loss', patience=3,
    min_delta = 0.001, mode = 'min' )

# Train defined model- history_orig = model.fit(
    x = X_train, y = y_train,
    batch_size = 32, epochs = 500,
    validation_data = (X_test, y_test),
    callbacks = [callback],
    verbose = 1 )

# Instantiate a model- model_gt = create_nn()

# Restore random weights as used by the previous model for fair comparison- model_gt.load_weights("Random_Weights.h5")

# Choose an optimizer and loss function for training- loss_fn = tf.keras.losses.BinaryCrossentropy() optimizer = tf.keras.optimizers.Adam(lr = 0.001)

# Select metrics to measure the error & accuracy of model.
# These metrics accumulate the values over epochs and then
# print the overall result- train_loss = tf.keras.metrics.Mean(name = 'train_loss') train_accuracy = tf.keras.metrics.BinaryAccuracy(name = 'train_accuracy')

test_loss = tf.keras.metrics.Mean(name = 'test_loss') test_accuracy = tf.keras.metrics.BinaryAccuracy(name = 'train_accuracy')

# Use tf.GradientTape to train the model-

@tf.function def train_step(data, labels):
    """
    Function to perform one step of Gradient
    Descent optimization
    """

    with tf.GradientTape() as tape:
        predictions = model_gt(data)
        loss = loss_fn(labels, predictions)

    gradients = tape.gradient(loss, model_gt.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model_gt.trainable_variables))

    train_loss(loss)
    train_accuracy(labels, predictions)

@tf.function def test_step(data, labels):
    """
    Function to test model performance
    on testing dataset
    """

    predictions = model_gt(data)
    t_loss = loss_fn(labels, predictions)

    test_loss(t_loss)
    test_accuracy(labels, predictions)

EPOCHS = 100

# User input- minimum_delta = 0.001 patience = 3

patience_val = np.zeros(patience)

# Dictionary to hold scalar metrics- history = {}

history['accuracy'] = np.zeros(EPOCHS) history['val_accuracy'] = np.zeros(EPOCHS) history['loss'] = np.zeros(EPOCHS) history['val_loss'] = np.zeros(EPOCHS)

for epoch in range(EPOCHS):
    # Reset the metrics at the start of the next epoch
    train_loss.reset_states()
    train_accuracy.reset_states()
    test_loss.reset_states()
    test_accuracy.reset_states()

    for x, y in train_ds:
        train_step(x, y)

    for x_t, y_t in test_ds:
        test_step(x_t, y_t)

    template = 'Epoch {0}, Loss: {1:.4f}, Accuracy: {2:.4f}, Test Loss: {3:.4f}, Test Accuracy: {4:4f}'

    history['accuracy'][epoch] = train_accuracy.result()
    history['loss'][epoch] = train_loss.result()
    history['val_loss'][epoch] = test_loss.result()
    history['val_accuracy'][epoch] = test_accuracy.result()

    print(template.format(epoch + 1,
                          train_loss.result(), train_accuracy.result()*100,
                          test_loss.result(), test_accuracy.result()*100))

    if epoch > 2:
        # Computes absolute differences between 3 consecutive loss values-
        differences = np.abs(np.diff(history['val_loss'][epoch - 3:epoch], n = 1))

        # Checks whether the absolute differences is greater than 'minimum_delta'-
        check =  differences > minimum_delta

        # print('differences: {0}'.format(differences))

        # Count unique element with it's counts-
        # elem, count = np.unique(check, return_counts=True)
        # print('\nelem = {0}, count = {1}'.format(elem, count))

        if np.all(check == False):
        # if elem.all() == False and count == 2:
            print("\n\nEarlyStopping Evoked! Stopping training\n\n")
            break

In "model.fit()" method, it takes around 82 epochs, while GradientTape method takes 52 epochs.
Why is there this discrepancy in the number of epochs?
Thanks!

Hi. Please, fix the indentation of your code, provide the required imports to execute the code and make sure the code is minimal. In any case, this question is better suited for Stack Overflow or Data Science SE.

@nbro fixed indentations and added imports


text:
How can I find the similar non-zero connections between different levels of sparsity of the same network?
I am pruning a neural network (CNN and Dense) and for different sparsity levels, I have different sub-networks. Say for sparsity levels of 20%, 40%, 60% and 80%, I have 4 different sub-networks.
Now, I want to find the non-zero connections that they have in common. Any idea how to visualize this or compute this?
I am using Python 3.7 and TensorFlow 2.0.
After the convergence of a neural network following the random weight initialization, some weights/connections increase (magnitude), while other weights decrease. You can then prune the smallest magnitude weights. I want to compare the remaining weights for say two networks having the same level of sparsity of say 50%. The idea is to have an idea of which weights were pruned away and which weights/connections remain.

The paper Visualizing the Hidden Activity of Artificial Neural Networks (by Rauber et al.) could be useful. I haven't yet read it. If I read it, I may provide an answer to this question later.

edited question to add more information


text:
How to properly use batch normalization during inference
I am trying to manually implement calculations of the image classification process using pre-trained weights from the MobilenetV2 network. I know how to apply filter weights to the channels, but not sure what to do with the coefficients from the batch normalization (BN) layers. The model uses BN after each convolution before ReLu6. As explained in many sources, BN has a lot of benefits during model training. The original Mobilenetv2 paper does say that they used BN during training, but nothing about using it during testing. The pre-trained MobilenetV2 model comes with BN layers which contain weights 4 x n_channels (I assume gamma, beta, mean, and std for each input featuremap in the BN layer). The following questions is:

How do I apply the four coefficients to a featuremap during inference? (This article explains it, but I still don't get it - aren't those imported coefficients already pre-calculated, so the operation on a featuremap is reduced to a multiply-add?)

The original paper on BN in section 3.1 says:

... Since the means and variances are fixed during inference,
  the normalization is simply a linear transform applied to
  each activation. It may further be composed with the scaling by γ and shift by β, to yield a single linear transform
  that replaces BN(x)...

Does this mean that during inference I would use only gamma and beta coefficients to "scale and shift" each pixel of a corresponding feature map? That is, something like:
for ch
  for row
    for col
      out_feature[row][col][ch] = in_feature[row][col][ch] * BN[gamma][ch] + BN[beta][ch]

Could anyone, please, confirm and explain if this is correct and re-iterate what exactly is expected from the BN layer output in terms of value ranges (before ReLu6)?


text:
How do I poison an SVM with manifold regularization?
I'm working on Adversarial Machine Learning, and have read multiple papers on this topic, some of them are mentioned as follows:

Poisoning Attacks on SVMs: https://arxiv.org/pdf/1206.6389.pdf
Adversarial Label Flips on Support Vector Machines

However, I am not able to find any literature on data poisoning for SVMs using Manifold regularization. Is there anyone who has knowledge about that?

I found the following Youtube video: Poisoning Attacks against Support Vector Machines. Is it close to what you are looking for?

I think, one could generate a data set, with a random variable in each component of the data vector, add this data to the training data set, and then shuffle the combined data set.

Can you please clarify how this answers the question?


text:
How can I assign agents to tasks based on time and affinity?
I am working on an assignment problem.
Consider $K$ agents $A_1, \dots A_K$ and $N$ tasks $T_1, \dots T_N$. Each task has a certain time $t(T_i)$ to be completed and each agent has a matching (or affinity) value associated with each task $M_{A_j}(T_i), \forall i, j$. The goal is to assign agents to tasks, such that the matching value is maximized and the overall time to complete the tasks is minimized. Moreover, an agent can be assigned to multiple tasks. However, an agent cannot start a new task before finishing the previous one.I want to solve it with GA+MOA*algorithm What would be an admissible heuristic function?

See also https://en.wikipedia.org/wiki/Interval_scheduling, but this is not exactly your problem too, I am afraid.

No this not same with my problem,tasks in interval have fixed execution start and finish time