text:
Why is it possible to eliminate this branch with alpha-beta pruning?
Can someone explain to me why it is possible to eliminate the rest of the middle branch in this image for alpha-beta pruning? I am confused because it seems the only information you know is that Helen would pick at least a 2 at the top (considering if we iterate from left to right in DFS), and Stavros would absolutely not pick anything above 7. This leaves 5 possible numbers that the rest of the branch could take on that Helen could potentially end up picking, but can't because we've eliminated those possibilities via pruning.

Meanwhile, you may want to have a look at these notes http://web.cs.ucla.edu/~rosen/161/notes/alphabeta.html. In my opinion, they provide a good explanation of the $\alpha-\beta$ pruning algorithm. I may provide an answer later, if nobody does it.

You can't prune the nodes that are cross out if we search from left-to-right in the tree using alpha-beta pruning. To do this analysis we can pretend the right branch of the tree doesn't exist. (Branch C from the root.)
In the left branch (A) of the root Helen will get 2 or more.
In the middle branch (B) from the root after going down the left, Stavros will get 7 or less.
Now, we can ask what happens if we put different values for the crossed out branches. If putting different values doesn't change the value at the root, then pruning is correct.
Suppose that after the 7 on branch A at the second level we make the value of branch B to be -10. In this case the value of the second branch will be -10, and Helen will prefer the first branch at the root.
Suppose that after the 7 on branch A at the second level we make the value of branch B to be 100. In this case the value of the second branch will be 7, and Helen will prefer this branch.
Thus, the value of the crossed off nodes matters, and they cannot be pruned.
You may find this tool useful for exploring alpha-beta on binary trees:
https://movingai.com/ab/


text:
What is the difference between using a backbone architecture and transfer learning?
I'm super new to deep learning and computer vision, so this question may sound dumb.
In this link (https://github.com/GeorgeSeif/Semantic-Segmentation-Suite), there are pre-trained models (e.g., ResNet101) called front-end models. And they are used for feature extraction. I found these models are called backbone models/architectures generally. And the link says some of the main models (e.g. DeepLabV3 or PSPNet) rely on pre-trained ResNet.
Also, transfer learning is to take a model trained on a large dataset and transfer its knowledge to a smaller dataset, right?

Do the main models that rely on pre-trained ResNet do transfer learning basically (like from ResNet to the main model)?
If I use a pre-trained network, like ResNet101, as the backbone architecture of the main model(like U-Net or SegNet) for image segmentation, is it considered as transfer learning?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How should I design this LSTM network to perform stock prediction?
I'm trying to develop a stock predictor.
I'm using LSTM but I am unsure about the structure of the Neural Network. For example, I'm assuming that the Neural Network is a many-to-one since we have many inputs (i.e Open, Close etc) and one output (stock price).
By misunderstanding is coming with how to construct the nodes. For example, what input goes into the "Cell" (or node)? I.e does say 60 timestep mean 60 days of 'Open Price' are fed into the Neural Network at t and then 60 days of 'Close' into t + 1 until we use all input to produce an output?
If someone could explain the process of how LSTM are used with stock predictions that would be appreciated.

I can't say for all cases, but I can certainly give you an example of how an LSTM could be used to predict stock prices.
An LSTM is temporal, meaning you can feed in one input, get an output and the network will remember this interaction (if it's important), which will effect the outcome of future predictions. As such, it is reasonable to feed in all the data for the day (Open, Close etc), obtain an output for the predicted stock price of the following day (or a prediction for all the inputs you feed in, the output can be as large or small as you want), then repeat for as long as you want, each time only feeding in the data for that particular day as the network will remember previous days.
In your example, 60 timesteps would mean doing a forward pass of the LSTM 60 times, for 60 days. Each of these forward passes will produce an output, which you can compare to the next actual stock price for verification, until you reach the current day, where the prediction is for the future.

@Reccessice I was thinking feed in 0-59 days first, get an output for day 60, then feed in days 1-60 get output day 61 and hope it remembers certain patterns. This is for training the model. So the result of day 0-59 input will be passed onto the next loop of day 1-60. Would this be correct?

@user33121 If you trained your data on time periods of 60 days, then yes this is definitely the way to go. Just as a warning, don't extend the LSTM past the maximum temporal depth of training and expect coherent results

What is the depth? Do you mean as in over-training the data?

@user33121 No by temporal depth I mean the number of times you feed input into the LSTM. So if you feed in 60 days in training, then do weight updates and repeat, don't then feed in 80 days in testing and expect reasonable outputs past day 60 as the LSTM was never trained to go that far

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Can we train the model to detect real users with only positive labels?
We have hundreds of thousands of customers records, and we need to take the benefits of our data to train a model that will recognize fake entries or unrealistic ones for our platform, where customers are asked to enter their names, phone number and zip code.
So, our attributes are name, phone number, zip code and IP address to train the model with. We have only data associated with real users. Can we train a model provided with only positive labels (as we do not have a negative dataset to train the model with)?

Regardless of the practicality of using an AI system to solve this problem, I would doubt that this is something that should be done. Human names are 'weird', and different cultures have different conventions. By excluding anything that doesn't conform to the majority, you are discriminating against those that are different. You cannot guarantee that names are 'true' anyway, as you can easily make up a name that looks like a real one.

Thanks, even if we already have customers names for the past 4 years?

How do you define 'bad' or 'unrealistic' names? Do you not expect any new customers? You could simply check the first names against all your existing first names to flag those up which look peculiar if that's what you want. But I would strongly advise against doing this fully automated.

Hi simo, it may be possible to answer this question in a theoretical sense, and show you how you might create a system that allowed 99.9% of your customers to use the system plus block 90%+ of attempts to naively trick the system. That is interesting from the AI stand point, and would be fine and on topic question here. Whether you should try and implement such a thing on a commercial system is another matter. My web development background would agree with Oliver's summary here - although it may depend on your purpose for making names conform.

So, could you be clear, are you asking for AI theory on detecting "realistic" names? We can answer that with suitable theory from NLP/ML. Or are you asking for advice for implementing/integrating this witha real-world system? The answer there will look more like Oliver's comment, unless we dig more into why you want such a detector

@NeilSlater, my problem of detecting real names and avoid bad names like for example spamming or bad words or inconsistent combination attributes. Its a problem we need to solve for a real-world system, as we already have real customers data for previous years, kindly see my commits below for available attributes.

@OliverMason We always expect new customers, we have multiple attributes for each customer, that is customer name, phone number and address, so we can include those attributes in the learning phase, we can also add other attributes like IP address or zip code.
Advice is highly appreciated.

@simo: So you are more focused on solving this as a real problem with your system? I think you need to stop worrying so much about names, as they are poor data to base decisions on. Please [edit] your question to be about the problem you want to solve, include what data you have available, and what actions you need your system to be able to perform. You may also want to ask about your root question in https://security.stackexchange.com/ explaining what kinds bad data you are receiving, the damage it does to your service, and advice on filtering and/or blocking it

Also, could you explain why you only have valid names or valid accounts to work with? Presumably you have a real, detected problem here and actualy have examples of non-valid data that has prompted you to try and solve this problem. I would also presume that you have more than one example of an unwanted account/order/message as otherwise there would not be a real problem to solve for your company's product, if no-one was misusing it?

@NeilSlaterwe Yes, its a real problem with our system, currently we reply on a trusted third party service but I found this an opportunity to apply what I learned in AI on a real problem, of course, my solution will not be deployed before comparing its output with the trusted third party service. We don't have non-valid data unfortunately, as we relied on a third party service, and our team was not aware of possibility of non-valid data advantages, however, we can start storing invalid data from now on..

@NeilSlater I have edited the question to reflect attributes we have.

i don't see why AI/ML is needed: you can find a list of zip codes online, you can check it with the ip address for validation, but this check won't always match even in proper cases; phone numbers can be checked for length and area code but not much else; you can perhaps train a model that infers how likely a given name is real, but it's probably not going to have an acceptable fail rate (perhaps something like 5-10%?) considering how mission critical signing up is. perhaps the better question is why you should bother with this in the first place, junk data / spam is normal for the internet :(

i am very suspicious of the fact that people actually provided AI/ML solutions to this question when the very premise of the question should be called into question

I do not see what is wrong with the premise of the question - it is akin to spam detection. Imagine they are sending snail mail catalogs to people in their database and do not wish to spend the money on Mickey Mouse at 123 Main Street. Even suppose they think the people worth having as customers all have first names in the top 100 American first names. This is evil?

The problem which you have is a classification problem. You assume a class "good users" and a distinct class "bad users". You want to train an AI to tell the two apart, but all your examples are "good users". Any reasonable AI will draw the logical conclusion from those examples: all users are good users. That's a 100% match for the training data.

This isn't necessarily classification, could also be anomaly detection. Of course, having an outlier is not always bad. If you're a small bank with relatively poor customers, a billionaire is an outlier you don't want to throw out.

Pragmatically you could use the discriminatory from a GAN for outlier detection.
Ideally you'd start collecting fakes now and do a normal model on both good and bad cases.
In the absence of that you can create a GAN to create realistic looking fakes on only real cases and then take the discriminator from that GAN to flag real-life cases for manual checks.
Please for a real life case always include these real life checks which also helps collecting cases for improving the model.


text:
How I can predict the next number in a sequence with a neural network?
I've been dabbling with machine learning and neural networks (namely, resnet50) for a few months now, mostly doing image recognition. I am currently trying to make a program that, given a string of numbers as input, can predict the next number in this sequence. For example, the input could be 1, 2, 3, 4 and the output should be 5.
I read something that said this could be done with a multilayer perceptron neural net, but that didn't elaborate much.
Any ideas, or links to tutorials/code?

Sequence prediction is very often approached with recurrent neural networks (RNNs). A specific kind of RNN, LSTMs, would be a good choice.

OEIS reports that there are 12,856 integer sequences beginning with 1,2,3,4 so one must wonder why 5 is the valid answer. If the only rule at work is "add 1 to the previous value," then a neural network does not seem at all necessary to solve the problem.

5 is the answer given by the shortest linear prediction from previous values. x(n)=2x(n-1) -x(n-2).

An idea: predict linearly your output values from 1 to N-1 previous samples and choose the minimum length predictor that gets zero error on your data. With your numbers, 2x(n-1)-x(n-2) predicts exactly 3 from 1 2, 4 from 2 3 and 5 from 3 4.


text:
Is it feasible to use GAN for high-quality image synthesis other than human faces?
The famous Nvidia paper Progressive Growing of GANs for Improved Quality, Stability, and Variation, the GAN can generate hyperrealistic human faces. But, in the very same paper, images of other categories are rather disappointing and there hasn't seemed to be any improvements since then. Why is it the case? Is it because they didn't have enough training data for other categories? Or is it due to some fundamental limitation of GAN?
I have come across a paper talking about the limitations of GAN: Seeing What a GAN Cannot Generate.
Anybody using GAN for image synthesis other than human faces? Any success stories?

Generative Adversarial Networks, basically boil down to a combination of a generic Generator and a Discriminator trying to beat each other, so that the generator tries to generate much better images (usually from noise) and discriminator becomes much better at classification. So, no it is not just suited for only synthesis high quality human face synthesis but any image type.
In fact, not only can it be used for any high quality image synthesis, it can work on non-image data types as well (such as text, etc). It all depends on the type of neural network you are using for the discriminator and generator at the end of the day.
http://openaccess.thecvf.com/content_iccv_2017/html/Osokin_GANs_for_Biological_ICCV_2017_paper.html
The above refers to a paper synthesizing cell image through GANs, as I haven't personally used GANs on a practical level.
General blog explaining GANs:
https://machinelearningmastery.com/impressive-applications-of-generative-adversarial-networks/
Human faces are much more frequently tackled, for many reasons, usually that human faces are highly symmetric and have a lot of different features, usually more than other types of images, with the added difficulty of being that we as actual humans are usually good at recognizing faces - making a neural net to fool ourselves, makes it a challenging area of research.
Hope it helped! Do let me know if I'm wrong somewhere.

I'd challenge your assertion somewhat that the generated images of other categories are of much worse quality than the faces!
Take the bikes on transparent / solid backgrounds they look great!
Where the images fail a bit is with the more complex pictures which have a lot of elements where element bleed (covers bleeding into the floor, etc.) occurs. This is simply a result of the complexity of the image and the training data base.
As an example I have developed a GAN that generates "Vaporwave"-like imagery like this:

Now my results were generally poor because unlike faces my training set was highly diverse in terms of arrangements, elements, etc. If you look at the generated bed images in your example paper not only did the GAN have to learn and generate beds but also the highly complex backgrounds which did differ severely between training images whereas in the face example the image was zoomed in on the faces and obscuring the background.
If you use human faces in a normal background setting (e.g. with the scenery around them visible) your GAN will perform equally good or bad because there is so much more complexity to learn.
You can find my experiences with non-faces GAN on Kaggle but understand the bad results are mainly due to a very small training set and the fact that these images are very different (besides the color gradient which the GAN pics up very fast).
https://www.kaggle.com/fnguyen/vaporprogan


text:
Why does reinforcement learning using a non-linear function approximator diverge when using strongly correlated data as input?
While reading the DQN paper, I found that randomly selecting and learning samples reduced divergence in RL using a non-linear function approximator (e.g a neural network).
So, why does Reinforcement Learning using a non-linear function approximator diverge when using strongly correlated data as input?

Read chapter 11 of this book. This is only a draft, if you can find the full book even better. Also, I think similar questions were answered already so try searching a bit through the website.

Maybe this is a duplicate of Why doesn't Q-learning converge when using function approximation?.

It is not so much the problem of using Reinforcement Learning to train the neural networks, it is the assumptions made about the data given to standard Neural Networks. They are not capable of handling strongly correlated data which is one of the motivations for introducing Recurrent Neural Networks, as they can handle this correlated data well.


text:
Why does the discrepancy measure involve a supremum over the hypothesis space?
I am referring specifically to the disc defined by Kuznetsov and Mohri in https://arxiv.org/pdf/1803.05814.pdf

This is a kind of worst case path dependent generalization error. But what is the intuitive way of seeing why a worst case is needed? I am probably missing something or reading something incorrectly.

The formula $G=\mathbb{E}\left[ f(Z_{T+1}) \mid \mathbf{Z}_1^T\right] - \sum_{t=1}^Tq_t \mathbb{E}\left[ f(Z_t) \mid \mathbf{Z}_1^{t-1} \right]$ actually represents a set, for all possible values of $f$. Therefore, $\text{disc}(\mathbf{q}) = \operatorname{sup}_{f \in \mathcal{F}} \left( \mathbb{E}\left[ f(Z_{T+1}) \mid \mathbf{Z}_1^T\right] - \sum_{t=1}^Tq_t \mathbb{E}\left[ f(Z_t) \mid \mathbf{Z}_1^{t-1} \right] \right)$ is the element not necessarily in that set $G$ that is greater than all elements in that set $G$, but is the smaller than any other element that is greater than any element in that set $G$. In other words, $\text{disc}(\mathbf{q})$ represents an upper bound on the discrepancy $G$, but it is the smallest possible upper bound. See also this answer for more details about the supremum and the relationship between the supremum and upper bounds.
So, unless I am wrong, this is not necessarily a worst-case analysis (but I am not even sure how one would define the worst-case analysis in this context), but we are looking for the smallest upper bound. In the context of algorithms, the worst-case analysis is with respect to the input (i.e. the worst-case input), but you can have upper or lower bounds for worst and best-case scenarios. See, for example, this answer that illustrates this. Why do we want an upper bound? Because we can be certain that we won't have a generalization error worse than it. You can't do this with e.g. a lower bound.

I am referring more to the choice of sup as opposed to any other norm. I think in this case, there is something not immediately obvious where the sup norm is driving the  $q$ weights and for these $q$ weights you then optimize over the standard objective. The sup norm is a particularly choice of norm here and maybe is just chosen as it is worst case, or maybe it is the only possible choice for the bounds that follow to be amenable.

@mathtick The supremum is not a norm. I still don't get why you're talking about worst-case. Here we are talking about an upper bound (not worst-case).

Yes it is. What I am asking is a deeper question about the topics specific to the paper.
http://mathworld.wolfram.com/SupremumNorm.html

@mathtick Ok, I was not aware of this supremum norm, but, in your case, where is this norm? The supremum norm involves an absolute value or a norm (inside the set), but you have only the supremum there and the difference between two expectations. What's your deeper question? And what "norm" or other operators would use other than the supremum?

There are usually various inequalities related different kinds of norms. For example often one is working in $L_p$ space where the $p$ norm is indicates some choice of norm. $L_1$, $L_2$, and $L_{\infty}$ are ones you probably know. See https://en.wikipedia.org/wiki/Lp_space#The_p-norm_in_countably_infinite_dimensions_and_%E2%84%93_p_spaces

And reading further after I posted, I think mostly the sup is used as it is the only way to extend the inequalities encountered further on without additional assumptions on the distributions. Not 100% on that but that is the hunch.

@mathtick You still haven't answered why the sup in your equation is used as a norm. In all examples of the supremum norm that I've seen include an absolute value or norm inside the expression, but your equation doesn't include that. That's why I am not sure that your expression represents a supremum norm. Yes, I know that there are different norms (but I mainly used or saw them in the context of linear algebra, etc.).

What does "used as a norm" mean? These can be interpreted as norms, the triangle inequality is used later in the paper etc. I just typically call these things norms without worrying about whether all properties of a norm are used.


text:
Can GraphRNN be used with very large graphs?
In the GraphRNN paper, the authors only implement the algorithm up to a graph size of 2k nodes. Would this still work on much larger graphs (on the order of $10^7$)? Or would the computation just become too substantial?


text:
How does batch size affect model size?
I'm suffering from a significant brain fart while trying to get my head around how does batch size affect overall model size e.g for CNNs. Does it serve as an additional dimension for all the weight tensors?
Considering:

VGG16 model
batch_size of 16
image size of 224x224x3
conv_1 being the initial 1x1 convolution with stride 1 and 3:64 channels mapping

The input will be a tensor of [16, 224, 224, 3] shape. Will the output of convolution layer be [16, 224, 224, 64] and therefore - will all the weights have additional 'batch size' dimension and thus - impose a linear increase of model size with respect to the batch size?

You probably mean not "model size" winch is not affected but amount of working memory, which is model size + layers output + temporary variables. It depend on framework/implementation


text:
Will AI always depend on models and thus approximations?
In section 3 of the paper The Limits of Correctness (1985) Brian Cantwell Smith writes

When you design and build a computer system, you first formulate a model of the problem you want it to solve, and then construct the computer program in its terms.

He then writes

computers have a special dependence on these models: you write an explicit description of the model down inside the computer, in the form of a set
  of rules or what are called representations - essentially linguistic formulae encoding, in the terms of the model, the facts and data
  thought to be relevant to the system's behavior. It is with respect to these representations that computer systems work. In fact, that's really what computers are (and how they differ from other machines): they run by manipulating representations, and representations are always formulated
  in terms of models. This can all be summarized in a slogan: no computation without representation.

And then he says

Models have to ignore things exactly because they view the world at a level of abstraction

He then writes in section 7

The systems that land airplanes are hybrids - combinations of computers and people - exactly because the unforeseeable happens, and because what
  happens is in part the result of human action, requiring human interpretation

As quoted above, computers depend on models, which are abstractions (i.e. they ignore a lot of details), which are written inside the computer. Therefore, the true world cannot really be encoded into an algorithm, but only an abstraction and thus simplification of the world can.
So, will AI always depend on models and thus approximations? Can it get rid of or overcome this limitation?

We can have pretty good interpretations that are not perfect, but close to it. These are able to be used to form close to perfect results.

@Lustwelpintje Are human brain processes, like creativity, intuition or imagination,  computable processes? If not, I think AI will be limited by the abstractions of the models.

@alamata I guess this is more or less open for debate at this point. I mean whether all that brain does is some sort of computation and communication between neurons. Saying that, it certainly does some sort of computation AFAIK.

AI is internally limited by model and externally limited by the environment.
Humans are externally limited by the environment but not necessarily internally limited by a computable model (as AI is).
So, humans may possess certain skills (e.g. creativity) that an AI may never possess. I had previously asked a related question Are human brain processes, like creativity, intuition or imagination, computable processes?.
Which research work supports my claims?
Brian Cantwell Smith says that there is no computation without representation (a model).
In the article The Brain Is Not Computable, Miguel Nicolelis, a top neuroscientist at Duke University, also says

The brain is not computable and no engineering can reproduce it
You can't predict whether the stock market will go up or down because you can’t compute it.
You could have all the computer chips ever in the world and you won't create a consciousness.
That's because its most important features are the result of unpredictable, nonlinear interactions among billions of cells

There's a neuroscience theory, known as predictive coding, which roughly states that the (human) brain is constantly generating and updating a model of the world.

The brain is constantly confronted with a wealth of sensory information that must be processed efficiently to facilitate appropriate reactions. One way of optimizing this processing effort is to predict incoming sensory information based on previous experience so that expected information is processed efficiently and resources can be allocated to novel or surprising information. Theoretical and computational studies led to the formulation of the predictive coding framework (Friston 2005, Hawkins and Blakeslee 2004, Mumford 1992, Rao and Ballard 1999). Predictive coding states that the brain continually generates models of the world based on context and information from memory to predict sensory input. In terms of brain processing, a predictive model is created in higher cortical areas and communicated through feedback connections to lower sensory areas. In contrast, feedforward connections process and project an error signal, i.e. the mismatch between the predicted information and the actual sensory input (Rao & Ballard, 1999). The predictive model is constantly updated according to this error signal.

This theory should not be surprising or unintuitive, given that every person possesses a slightly different perspective (or model) of the world, which is based on her (or his) personal experiences. Of course, this is just a theory, which may not be the most precise one that describes our brain, but this theory is already being validated by a number of brain imaging studies investigating predictive feedback and the processing of prediction errors.
Therefore, artificial intelligence may not be the only entity that is based on or will be limited by a model of the world. To answer your question more directly, yes, the AI will always be limited by its model and environment (e.g. hardware), in a similar way that flatlanders are limited by their 2-dimensional nature and world, but this doesn't necessarily mean we will not be able to create useful (and even sophisticated or human-like) AI systems.


text:
Can recovering a reward function using IRL lead to better policies compared to reward shaping?
I am working on a research project about the different reward functions being used in the RL domain. I have read up on Inverse Reinforcement Learning (IRL) and Reward Shaping (RS). I would like to clarify some doubts that I have with the 2 concepts.
In the case of IRL, the goal is to find a reward function based on the policy that experts take. I have read that recovering the reward function that experts were trying to optimize, and then finding an optimal policy from those expert demonstrations has a possibility of resulting in a better policy (e.g. apprenticeship learning). Why does it lead to a better policy?

You're asking two distinct questions here. I suggest that you split this post into two.

RL does not "specify immediate rewards". It often calculates a value (state value or action value), which then can be used to select an action. The value is the expected return, which is calculated from rewards. Although this may seem like nit-picking, it is important to get the terminology solid in a question, otherwise you will get answers that simply correct your terminology when you want other information. If you undertand the difference between return and reward (and I think you do from other questions), please fix this in the question

Note that IRL, in comparison, really does calculate an immediate reward function. So the difference between IRL doing that, and the value functions of RL is probably part of your question.

Hi @NeilSlater thank you for the comment, I will edit the question accordingly.

Actually I think there is no definite answer to this question, based on what I have gathered i will try to answer my own question.

Inverse Reinforcement Learning (IRL) is a technique that attempts to recover the reward function that the expert is implicitly maximising based on expert demonstrations. When solving reinforcement learning problems, the agent maximises a reward function specified by the designer, and in the process of reward maximisation, accomplishes some task that it had set out to do. However, reward functions for certain tasks are sometimes difficult to specify by hand. For example the task of driving takes into consideration many different factors such as the distance of the car in front of him, the road conditions and whether or not the person needs to get to his destination quickly. A reward function can be hand specified based on these features. However, when there exists trade offs between these different features, it is difficult to know how to specify the different desiderata of these tradeoffs.
Instead of specifying the trade offs manually, it would be easier to recover a reward function from expert demonstrations using IRL. Such a reward function can lead to better generalisations to unseen states as long as the features of driving do not change.
In the case where reward shaping fails to learn a task (such as driving), it would be better to have someone demonstrate a task and learn a reward function from these demonstrations. Solving the MDP with the learnt reward function will thus yield a policy that should resemble the demonstrated behaviour. The reward function learnt should also generalise to unseen states and the agent acting in unseen states should be able to perform actions that an expert would take when he is placed in the same conditions, assuming that the unseen states come from the same distribution as the training states.
While Reward Shaping might be able to perform the same task as well, IRL might be able to do better, based on some performance metric that will differ from problem to problem.


text:
Is this neural network architecture appropriate for CIFAR-10?
I have a CNN architecture for CIFAR-10 dataset which is as follows:

Convolutions:           64, 64, pool
Fully Connected Layers: 256, 256, 10
Batch size:             60
Optimizer:              Adam(2e-4)
Loss:                   Categorical Cross-Entropy

When I train this model, training and testing accuracy along with loss has a very jittery behavior and does not converge properly.
Is the defined architecture correct? Should I have a max-pooling layer after every convolution layer?

I know this is an old question, but could you please edit it to include the result of plot_model(your_model_object)? Moreover, you should provide the plots that show the behavior of the accuracy and loss. You should explain for how long you have been training.

To be honest, your model is not very clear. But basically after the convolution, you need to add non-linear layers. Otherwise, there is no point of Neural Networks.
You can add a Relu layer for sure.

I am using TensorFlow 2.0 Conv2D layer and I am using ReLU activation function


text:
What is the difference between exhaustive nearest neighbor search and k-nearest neighbour search?
I have two lists of feature vectors calculated from pre-trained CNN for image retrieval task:
Query: FV_Q and Reference FV_R.
>>> FV_R.shape
(3450, 128)

>>> FV_Q.shape
(3450, 128)

I am a little confused between the concept of exhaustive nearest neighbor search and k-nearest neighbor search.
In python, I use from sklearn.neighbors import KDTree to extract top k = 5 similar images from the reference database, given the query image!
Can somebody explain if there might be any similarities/differences between these two concepts?
Am I making a mistake somewhere in my feature vector comparison?

The exhaustive nearest neighbor search performs an exhaustive search of the nearest neighbor (i.e. the closest image or matches, depending on your application). Here an exhaustive search means that you will compare your query image with any other image, or, in the case of feature matching, you will compare every feature of the query image with every other feature of the target/reference image.
The $k$-nearest neighbor search finds the $k$ closest images. You typically do that by first building a KD-tree with your query (or target) image, so that to speed up the search for the $k$-nearest neighbors of your query image.
(Btw, in case you are interested in the concept of feature matching, in OpenCV, the class that you can use to perform an exhaustive search for the matches between two images is the BFMatcher (which stands for Brute Force Matcher), while the class to perform the search with a KD-tree for the k-nearest neighbhours is FlannBasedMatcher).

authors: 孛尔只今爱育黎八力哒哒哒阿达


text:
Is there a mathematical theory behind why MLP can classify handwritten digits?
I'm trying to really understand how multi-layer perceptrons work. I want to prove mathematically that MLP's can classify handwritten digits. The only thing I really have is that each perceptron can operate exactly like a logical operand, which obviously can classify things, and, with backpropagation and linear classification, it's obvious that, if a certain pattern exists, it'll activate the correct gates in order to classify correctly, but that is not a mathematical proof.

This is probably a special case of the universal approximation theorem. Here is a Wikipedia page about the theorem.

The approximation theorem says you can approximate anything. But this is kind of meaningless in so far as you can do KNN and get an arbitrary approximation of your training data also.
Proving CNN correctly extract features is, I don't think possible. Or if it is, something involving VC theory is probably the best you can do.

The statement "The approximation theorem says you can approximate anything." is false. Which approximation are you talking about? The approximation theorems are usually about approximating continuous functions, which is far from being anything.

@nbro OTOH we are only talking about approximation. Obviously continuous functions cannot be discontinuous, but they can approximate many discontinuous functions — probably something like any function with a finite number of discontinuities. That seems to be in agreement with this, for example.


text:
How are newer weight initialization techniques better than zero or random initialization?
How do newer weight initialization techniques (He, Xavier, etc) improve results over zero or random initialization of weights in a neural network? Is there any mathematical evidence behind this?

There are several ways to answer this question. First of all, there are several mathematical arguments on why using some kind of initialization is better. Consider reading, for example, Xavier et al.. Moreover, there are several numerical experiments showing the importance of initialization.
The motivation for Xavier initialization in Neural Networks is to initialize the weights of the network so that the neuron activation functions are not starting out in saturated or dead regions. In other words, we want to initialize the weights with random values that are not "too small" and not "too large". Thus the purpose is to fix the variance of the input data to each neuron to 1, because this reduces the variance of getting stuck in saturated areas, since the data is in general normalized this is equivalent to fixing the variance of each weight to $1/n$, where n is the number of input weights to the given neuron.


text:
How exactly does self-play work, and how does it relate to MCTS?
I am working towards using RL to create an AI for a two-player, hidden-information, a turn-based board game. I have just finished David Silver's RL course and Denny Britz's coding exercises, and so am relatively familiar with MC control, SARSA, Q-learning, etc. However, the course was focused on single-player, perfect-information games, and I haven't managed to find any examples similar to the type of game I have, and would like advice on how to proceed.
I am still unsure how self-play works, and how it relates to MCTS. For example, I don't know if this involves using the latest agent to play both sides, or playing an agent against older versions, or training multiple opposing agents simultaneously. Are there good examples (or repositories) for learning self-play and MCTS for two-player games?


text:
Is the LSTM component a neuron or a layer?
Given the standard illustrative feed-forward neural net model, with the dots as neurons and the lines as neuron-to-neuron connection, what part is the (unfold) LSTM cell (see picture)? Is it a neuron (a dot) or a layer?

The diagram you show works at least partially for describing both individual neurons and layers of those neurons.
However, the "incoming" data lines on the left represent all inputs under consideration, typically a vector of all inputs to the cell. That includes all data from current time steps (from input layer or earlier LSTM or time-distributed layers) - the line coming up into the LSTM cell - and the full output and cell state vectors from the whole LSTM layer on previous time step - the horizontal lines inside the cell starting on the left. Technically the top left input line could be read as either be a single neuron cell state value, or the full vector of cell state from the previous time step, depending on whether you were viewing the diagram as describing a single neuron in the layer, or the whole layer respectively.
If you visualise this cell connecting to itself over time steps, then the data lines both in and out must be whole layer vectors. The diagram is then best thought of as representing a whole LSTM layer, which is composed of various sub-layers which get combined, such as the forget gate layer (the leftmost yellow box).
Each yellow box in the diagram can be implemented very similar to a single layer of a simple feed forward NN, with its own weights and biases. So the forget gate can be implemented as
$$\mathbf{f}_t = \sigma(\mathbf{W}_f
[\mathbf{x}_t , \mathbf{y}_{t-1}] + \mathbf{b}_f)$$
where $\mathbf{W}_f$ is the weight matrix of the forget gate, $\mathbf{b}_f$ is the bias vector, $\mathbf{x}_t$ is layer input on current time step, $\mathbf{y}_{t-1}$ is layer output from previous time step, and the comma $,$ is showing concatenation of those two (column) vectors into one large column vector.
If the input $\mathbf{x}$ is an $n$-dimensional vector, and output $\mathbf{y}$ is an $m$-dimensional vector, then:

$\mathbf{c}$, the cell state (top line), and all interim layer outputs are $m$-dimensional vectors
$\mathbf{W}_f$, the forget gate weights, and the three other internal weights matrices are $m \times (n+m)$ matrices
$\mathbf{b}_f$, the forget gate bias, and the three other internal bias vectors are $m$-dimensional vectors

The other gates are near identical (using their own weights and biases, plus the third cell candidate value uses tanh instead of logistic sigmoid), and have their own weight matrices etc. The whole thing is constructed a lot like four separate feed-forward layers, each receiving identical input, that are then combined using element-wise operations to perform the functions of the LSTM cell. There is nothing stopping you implementing each yellow box as a more sophisticated and deeper NN in its own right, but that seems relatively rare in practice, and it is more common to stack LSTM layers to achieve that kind of depth.

Your explanations seem to be consistent with my knowledge. I think it would even be more helpful if you also provide the dimensionality of the vectors and matrices, so that the OP can understand that the LSTM layer can be conceptually divided into LSTM units (or blocks). However, I know that this can easily become cumbersome and tedious. Moroever, it's also worth emphasing that this view is useful because libraries usually implement LSTM layers and not LSTM units. See, for example, tf.keras.layers.LSTM.

@nbro: I added concrete dimensions for matrices and vectors. I am not sure if adding an equivalent break down to the "neuron" unit level would be worth it, other than to note that this does exist conceptually, which I think is already implied.

@NeilSlater, nice answer, I've just up-voted you, but just for the sake of completeness can you add as well the equations for input and output gates as well?

@JVGD: The input and output gates use  the exact same equation, just different variable names as they have dedicated weights and biases. E.g. $\mathbf{o}_t = \sigma(\mathbf{W}_o
[\mathbf{x}t , \mathbf{y}{t-1}] + \mathbf{b}_o)$ - instead of repeating myself I will make it clear just how similar the gates are.


text:
How does crossover work in a genetic algorithm?
If I had the weights of a certain number of "parents" that I wanted to crossbreed, and I used whatever method to pick out the "best parents" (I used a roulette wheel option, if that's any relevant), would I be doing this correctly?
For example, suppose I have picked the following two parents.
\begin{align}
P_1 &= [0.5, -0.02, 0.4, 0.1, -0.9] \\
P_2 &= [0.42, 0.55, 0.18, -0.3, 0.12]
\end{align}
When I'm iterating through each index (or gene) of the parents, I am selecting a weight from one parent only. I called this rate the "cross-rate", which in my case is $0.2$ (i.e. with $20$% chance, I will switch to choosing the other parents' weight).
So, using our example above, this is what would happen:
\begin{align}
P_1 &=[\mathbf{0.5}, \mathbf{-0.02}, 0.4, 0.1, \mathbf{-0.9}] \\
P_2 &= [0.42, 0.55, \mathbf{0.18}, \mathbf{-0.3}, 0.12]
\end{align}
So the child would be
$$C = [0.5, -0.02, 0.18, -0.3, -0.9]$$
I would choose $0.5$ from $P_1$, but for every time I choose a weight from $P_1$, there's a 20% chance that I actually choose the corresponding gene from $P_2$. But, for the first weight, I end up not landing on that 20% chance. So I move onto the second weight, $-0.02$. This time, we hit the 20% chance, so now we swap over. Our next weight is now from $P_2$, which is $0.18$. And so on, until we hit another 20% chance.
We keep doing this until we hit the end of the indexes ($P_1$ and $P_2$ have the same number of indexes, of course).
Is this the correct way to form a child from 2 parents? Is this the correct "crossbreeding" method when it comes to genetic algorithms?

As far as I know, there isn't a "specified correct way". The whole idea is that you want the population to converge and increase the sample rate in that more optimal looking place. What works best all depends upon your fitness landscape.
You could also crossover by doing something like
crossover_point = random_number_size_genome
child[:] = parent_a[:crossover_point] + parent_b[crossover_point:]

Or have multiple points of crossover or more exotic types of crossover.
As far as I know, the impact on your algorithm because of the different crossover algorithms shouldn't be that different.
Fitness function and how you select the fittest has a way bigger impact.
You have understood the crossover step correctly, looking at your example.

What does "in that more optimal looking place" mean? Please, clarify your answer.

The position algorithm has chosen to converge to using crossover that time step. What is hopefully a more optimal place than the initial one.


text:
Are there reinforcement learning algorithms not based on Markov decision processes?
Are all RL algorithms based on the MDP? If not, could you give examples of some which aren't? I've looked elsewhere, but I haven't seen it explicitly said.

Here's a very interesting answer to the question What are some alternative learning paradigms besides Markov decision processes?. I had also asked a similar question two years ago here.

Thank you for your response; that Quora post is pretty interesting.


text:
Generation of realistic real-valued sequences using Wasserstein GAN fails
My goal is to generate artificial sequences of real-valued data (e.g. time series) with GANs. Starting simple I tried to generate realistic sine-waves using a Wasserstein GAN. But even on this simple task it fails to generate any useful samples.
This is my model:
Generator
model = Sequential()
model.add(LSTM(20, input_shape=(50, 1)))
model.add(Dense(40, activation='linear'))
model.add(Reshape((40, 1)))

Critic
model = Sequential()
model.add(Conv1D(64, kernel_size=5, input_shape=(40, 1), strides=1))
model.add(MaxPooling1D(3, strides=2))
model.add(LeakyReLU(alpha=0.2))
model.add(Conv1D(64, kernel_size=5, strides=1))
model.add(MaxPooling1D(3, strides=2))
model.add(LeakyReLU(alpha=0.2))
model.add(Flatten())
model.add(Dense(1))

Is this model capable of learning such a task or should I use a different model architecture?


text:
Does the bag-of-visual-words method improve the classification accuracy?
I'm a beginner in computer vision. I want to know which option among the following two can get better accuracy of image classification.

SIFT features + SVM
Bag-of-visual-words features + SVM

Here's a reference: https://www.mathworks.com/help/vision/ug/image-classification-with-bag-of-visual-words.html.

Basically, deep learning is the current method for object classification. Pretty much all classic areas are dead.

Bag-of-visual words (BOVW) was classicly used in computer vision before the introduction of neural networks or some more advanced classical techniques, such us VLAD or Fisher Vectors. In any case, it is a good technique to use, but it is not the state-of-the-art today, and I won't recommend you use it for a real-life project.

Thanks for your advice! As I understand it, the BoW first apply knn to create a histogram vector and then apply svm to classify. I'm not sure such a technique is called "vector quantization" or "two stage classification". And I'm curious about that if such a technique improves the accuracy of classification.


text:
How to estimate the capacity of a neural network?
Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?

VC dimension
A rigorous measure of the capacity of a neural network is the VC dimension, which is intuitively a number or bound that quantifies the difficulty of learning from data.
The sample complexity, which is the number of training instances that the model (or learner) must be exposed to in order to be reasonably certain of the accurateness of the predictions made given some data, is proportional to this number.
The paper VC Dimension of Neural Networks (1998) by Eduardo D. Sontag provides a good introduction to the VC dimension of neural networks (even though these concepts are quite abstract and you may need to read them several times to fully grasp them). The information in this answer is highly based on that paper.
Shattering and VC dimension
In section 2, Concepts and VC Dimension, he describes the basic concepts behind the VC dimension (not only for neural networks), such as the concept of shattering (i.e. what does it mean for a set of sets to shatter another set?), which is a well-known concept in computational learning theory and is used to define the VC dimension (see definition 2), so you definitely need to get familiar with this concept to understand the VC dimension and, therefore, the capacity of a neural network (calculated with the VC dimension).
VC dimension of functions and neural networks
He then provides an equivalent definition of the VC dimension but for functions (equation 6). Given that neural networks represent functions, then we can also define the VC dimension of a neural network. A specific combination of weights of neural networks represents a specific function, for which the VC dimension can be defined. To be more precise, a parametrized function (and a neural network) can be denoted as
$$
\beta : \mathbb{W} \times \mathbb{U} \rightarrow \mathbb{R}
$$
where $\mathbb{W} = \mathbb{R}^p$ and $p$ is the number of weights (or parameters) of the neural network, $\mathbb{U}$ is the input space and $\mathbb{R}$ the output space. So, in this case, $\beta$ can also represent a neural network, with a certain parameter space $\mathbb{W}$, an input space $\mathbb{U}$ and an output space $\mathbb{R}$.
The vector $\mathbf{w} = (w_1, \dots, w_p) \in \mathbb{W}$ represents a specific combination of weights of the neural network, so it represents a specific function. The set of all functions for each choice of this weight vector can be denoted as
$$
\mathcal{F}_{\beta} = \{ \beta(\mathbf{w}, \cdot) \mid \mathbf{w} \in \mathbb{W} \}
$$
The VC dimension (VCD) of $\beta$ can then be defined as
$$
\text{VCD}(\beta) := \text{VCD}(\mathcal{F}_{\beta})
$$
Therefore, the VC dimension is a measure of the capacity of a neural network with a certain architecture. Moreover, the VC dimension is equivalently defined for a certain set of functions associated with a neural network.
How to calculate the VC dimension?
To calculate the actual VC dimension of a neural network, it takes a little bit of more creativity. Therefore, I will just report the VC dimension of some neural networks. For more details, you should fully read the cited paper (more than once) and other papers and books too (especially, the ones described in this answer, which provide an introduction to CLT concepts).
VC dimension of a perceptron
The VC dimension of a perceptron is $m + 1$, where $m$ is the number of inputs. Given that a perceptron represents a linear and affine function, the VC dimension of the perceptron is also equal to the number of parameters. However, note that, even though the VC dimension of the perceptron is linear in the number of parameters and inputs, it doesn't mean the perceptron can learn any function. In fact, perceptrons can only represent linear functions. See section 3.1 of VC Dimension of Neural Networks for more details.
VC dimension of a single hidden layer neural network
Let $n$ be the number of hidden units, then the VC dimension of a single hidden layer neural network is less than or equal to $n+1$. See section 3.2 of VC Dimension of Neural Networks for more details.
VC dimension of multi-layer neural networks with binary activations
The VC dimension of multi-layer neural networks (MLPs) with binary activations and $p$ weights (or parameters) is $\mathcal{O}(p \log p)$. See theorem 4 (and related sections) of the paper VC Dimension of Neural Networks for more details.
VC dimension of MLPs with real-valued activations
The VC dimension of MLPs with real-valued activations is no longer bounded by $\mathcal{O}(p \log p)$ and can be exponential in the number of parameters. See section 5.3 of VC Dimension of Neural Networks.
VC dimension of MLPs with linear activations
The VC dimension of MLPs with linear activations is $\mathcal{O}(p^2)$.  See theorem 5 of the paper VC Dimension of Neural Networks.
Notes
The VC dimension is often expressed as a bound (e.g. with big-O notation), which may not be strict.
In any case, the VC dimension is useful because it provides some guarantees. For example, if you use the VC dimension to describe an upper bound on the number of samples required to learn a certain task, then you have a precise mathematical formula that guarantees that you will not need more samples than those expressed by the bound in order to achieve a small generalization error, but, in practice, you may need fewer samples than those expressed by the bound (because these bounds may not be strict or the VC dimension may also not be strict).
Further reading
There is a more recent paper (published in 2017 in MLR) that proves new and tighter upper and lower bounds on the VC dimension of deep neural networks with the ReLU activation function: Nearly-tight VC-dimension bounds for piecewise linear neural networks. So, you probably should read this paper first.
The paper On Characterizing the Capacity of Neural Networks using Algebraic Topology may also be useful and interesting. See also section 6, Algebraic Techniques, of the paper I have been citing: VC Dimension of Neural Networks.
The capacity of a neural network is clearly related to the number of functions it can represent, so it is strictly related to the universal approximation theorems for neural networks. See Where can I find the proof of the universal approximation theorem?.

the value of n+1 for a single hidden layer neural network holds only if input weights and biases are constant. The model then becomes an affine function of output weights which is equal to n+1 (including bias term c0) ~ VC-dim = n+1

Most methods for measuring the complexity of neural networks are fairly crude. One common measure of complexity is VC dimension, a discussion which can be found here and here. For example, neural networks have a VC dimension that is too large to give a strong upper bound on the number of training samples needed for a model (the upper bound provided by VC analysis is much higher than what we have observed neural networks to be able to generalize from).
Another common measure of capacity is the number of parameters. We see in the paper "Understanding deep learning requires rethinking generalization", published at ICLR with over 1400+ citations, that networks with more parameters than data often have the capacity to memorize the data. The paper provides compelling evidence that traditional approaches to generalization provided by statistical learning theory (VC dimension, Rademacher complexity) are unable to fully explain the apparent capacity of neural networks. In general, neural networks seem to have a large capacity, given the apparent good performance on certain tasks.
Beyond these ideas, the universal approximation theorem tells us that the set of neural networks can approximate any continuous function arbitrarily well, which strongly suggests that any neural network has a large capacity.


text:
When is max pooling exactly applied in convolutional neural networks?
When using convolutional networks on images with multiple channels, do we max pool after we sum the feature map from each channel, or do we max pool each feature map separately and then sum?
What's the intuition behind this, or is there a difference between the two?

The pooling operation is applied to the output of the convolution layer. More precisely, it is applied separately for each of the input channels (or slices). So, if the pooling layer receives an input volume of $H_i \times W_i \times D$, then it will produce an output volume $H_o \times W_o \times D$, so the depth of the output volume is equal to the depth of the input volume. There is no sum involved in the pooling operation. For example, in the case of max pooling, you will choose the maximum number of a certain 2D window of values. You do this for each of the input channels.
In the article (which is part of some course notes) Convolutional Neural Networks: Architectures, Convolution / Pooling Layers, Andrej Karpathy says

Pooling layer downsamples the volume spatially, independently in each depth slice of the input volume.

The following diagram (a screenshot of one of the figures from the mentioned article) should provide some intuition behind the pooling operation.

The pooling operation in a CNN is applied independently to each layer and the resulting feature maps are disjoint. This is the very reason that in most schematics depicting a certain CNN architecture, we obtain three output maps from an input image (corresponding to the convolutions and pooling operations performed on the RGB channels separately).

Each channel in an image captures a set of information that might not be demonstrated by the other channels owing to the color receptivity. Hence, pooling performed independently is intuitive, in the sense that a group of pixels in the Red Channel may not provide similar features as the same set of pixels in a Blue or Green channel. Thus comparison of pixels is restricted within a channel.
The following image provides a visualization of the RGB channels converted to greyscale. Note the brightness of the Red Color (comparing with the color-image) in the three different channels for a better intuition:

[Image Source]
Finally, to feed the multi-dimensional outputs of the Convolutional Layers to a Fully Connected layer, the feature maps are stacked along the depth dimension and flattened to form a vector. This ensures that the Dense layer learns to classify images (using a Softmax activation in the final Dense layer) based on the non-linear combinations of these high-level features.


text:
How can I generate unique random patterns (similar to the ones in Nutella jars)?
How can I generate unique patterns, as they did for these Nutella jars? See, for example, the video Algorithm designs seven million different jars of Nutella.

You are probably looking at something like randomly generated constructive grammars. Not sure if that is considered an AI technique. Nowadays you might also choose to use a GAN (which is definitely considered a modern AI technique), but to do so with abstract shapes and patterns like in the picture you would proabably generate a training set using some randomly generated constructive grammars . . .

Some excerpts from     Nutella 'Hired' an Algorithm to Design New Jars. And It Was a Sell-Out Success:
The "algorithm" is called HP Mosaic and is included free in HP SmartStream Designer for HP printers.
More about how the algorithm works here: https://www.linkedin.com/pulse/hp-mosaic-20-steven-chow
HP Mosaic takes the vector PDF file as input (also known as a Seed file), and generates a large number of variations on the file by transforming it — scaling, transposition, and rotation — randomly.


text:
How does Monte Carlo have high variance?
I was going through David Silver's lecture on reinforcement learning (lecture 4). At 51:22 he says that Monte Carlo (MC) methods have high variance and zero bias. I understand the zero bias part. It is because it is using the true value of value function for estimation. However, I don't understand the high variance part. Can someone enlighten me?

When using terms like "high" for high variance, this is in comparison to other methods, mainly in comparison to TD learning, which bootstraps between single time steps.
It is worth spelling out what the variance applies to and where it comes from: Namely the Monte Carlo return $G_t$ distribution, which can be calculated as follows:
$$G_t = \sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1}$$
I am using a common convention here by the way that capital letters such as $G_t$ stand for random distributions of variables and lower case letters such as $g_t$ stand for examples/samples from those distributions. It is not always strictly used, especially in pseudo-code, but is important to make clear for this answer.
So a value function, such as action value function $q_{\pi}(s,a)$ can be written:
$$q_{\pi}(s,a) = \mathbb{E}_{\pi} [ \sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1} | S_t=s, A_t=a] = \mathbb{E}_{\pi} [ G_t | S_t=s, A_t=a]$$
i.e. it is fairly clear that the mean of Monte Carlo return $\mathbb{E}[G_t]$ from any specific starting point is the same as the value function from the same starting point, by definition (see Sutton & Barto or David Silver's definitions) so samples from it - taking specific measures $g_t$ - are unbiased estimates of it.
The variance $\text{Var}(G_t)$ might be written
$$\text{Var}(G_t) = \text{Var}(\sum_{k=0}^{T-t-1}\gamma^k R_{t+k+1}) \le \sum_{k=0}^{T-t-1}\gamma^k \text{Var}(R_{t+k+1})$$
The second step is technically equal only in a "worst case" scenario since to add variances directly, we need the variables to be independently sampled, whilst $R_{t}$ and $R_{t+1}$ could be correlated, and often are. So, technically we should account for covariance here between distribitions on different time steps. The maths for that is fiddly, and I won't show it here.
What you can say intuitively about the inequality, is unless $R_{t}$ and $R_{t+1}$ are perfectly correlated throughout each trajectory, then variance of $G_t$ will increase depending on the number of timesteps and possible rewards in the return, by some fraction between 0 and 1 of the total variance of all time steps. You can either add zero variance on a timestep, or some fraction of $\text{Var}(R_{t+n})$ depending on whether the relationship between [$R_t$ to $R_{t+n-1}$] and $R_{t+n}$ is fully deterministic ($+0$) or completely independent ($+\gamma^{n-1}\text{Var}(R_{t+n})$)*
The amount of covariance involved relies on details of the policy, state transition and reward functions. In general, the policy during RL will include some stochastic behaviour -such as $\epsilon$-greedy that will add to the variance on each time step. The state transition and reward functions might add yet more. In any case:
$$\text{Var}(G_t) \ge \text{Var}(R_{t+1})$$
and in practice, when $t+1 \neq T$, the inequality can be large, although highly dependent on the details of the MDP. For off-policy Monte Carlo with basic importance sampling in fact the variance can be unbounded.
This variance might be considered high, compared to the variance of typical single-step TD returns:
$$G_{t:t+1} = R_{t+1} + \gamma q_{\pi}(S_t, A_t)$$
The variance of which is limited to that inherent in a single time step (one reward function, one state transition and one policy decision). The problem of course being that
$$\mathbb{E}[G_{t:t+1}] \neq \mathbb{E}[G_{t}]$$
. . . unless $q_{\pi}(s, a)$ is perfect. Which is why we can say that TD returns are biased (whilst $q_{\pi}(s, a)$ is not converged, the expectation of the TD target does not equal the true value), but Monte Carlo returns are higher variance (when number of timesteps to end of trajectory is more than one, the effects of random factors in policy, state transition and reward function accumulate).

* You could explicitly construct an environment with covariance between time steps that can reduce overall variance of a MDP's retrun, e.g. the state includes reward so far, and future rewards are manipulated so that the return fits to a certain distribution. This would be a special case where variance of the return from some starting points could be lower than the variance from the first step, but this is an unusual set up and likely to be impossible for an agent to learn optimal control.

You say "it is fairly clear that the mean of Monte Carlo return...is the same as the value function from the same starting point", but I don't think this is "clear", in the sense that, unless you know the definition of the state-action value function, then this is not clear. You also say "What you can say intuitively about the inequality...", but this is not intuitive at all (at least to me, in the way you describe it). I guess you're saying this because you suppose that the variance of a bigger sum becomes larger.

You also then say "by some fraction between 0 and 1 of the total variance of all time steps", but I am not sure what you're talking about. Furthermore, you use the notation $\sigma^2(X)$ to denote the variance. Maybe the notation $\text{Var}(X)$ would be more intuitive. You also use both $\mathbb{E}$ and $\mu$, which makes things more confusing, if you are not familiar with all the notation (not my case). You may also attempt to expand the definition of $\text{Var}(X)$ in terms of the expectation and see what comes out of it (I haven't tried it).

@nbro: The variance of a sum of independent variables is equal to the sum of the variances of the variables. In an RL trajectory, the variables are not independent, but they are often at least partially independent.

@nbro I am changing the notation as suggested. OP is already happy with unbiased nature of Monte Carlo estimate, so I am just mentioning that in passing. You are right it is because $v_{\pi}(s) = \mathbb{E}{\pi}[G_t|S_t = s]$ by definition (and similar for $q{\pi}$), and I have already written answers about that so may link one

Ok, thanks for addressing this. After re-reading this answer, I think you should also use either the expression "random variable" or "probability distribution" and not "variable distribution" (which I've never heard of, unless it really refers to a distribution that varies, which can be the case, but I think you didn't mean that).


text:
How can we design the mutation and crossover operations when the order of the genes in the chromosomes matters?
Consider an optimization problem that involves a set of tasks $T = \{1,2,3,4,5\}$, where the goal is to find a certain order of these tasks.
I would like to solve this problem with a genetic algorithm, where each chromosome $C = [i, j, k, l, m]$ corresponds to a specific order of these five tasks, so each gene in $C$ corresponds to a task in $T$.
So, for example, $C = [1,3,5,4,2]$ and $C' = [1,5,4,2,3]$ would be two chromosomes that correspond to two different orders of the tasks.
In this case, how could we design the mutation and cross-over operations so that these constraints are maintained during evolution?
The genetic algorithm should produce the three best chromosomes or order of tasks.

I have edited this post to leave the question that I think was your original question. Please, make sure that this is really the case. If not, feel free to edit your post again and clarify what your question really is.

If I understood correctly, your problem is about finding the optimal way to execute a series of tasks in order to maximize the results, using Genetic Algorithms.
In few words, you're trying to solve the salesman problem.

If I am correct, you're looking for Crossover and Mutation algorithms that allow you to work with ordered sets of elements. For these scenarios you usually go for the classic PMX
(Partially Mapped Crossover) and Interchange Mutation. But, there are plenty of other crossover algorithms you can use OX1, OX2 (both variants of the Order Based Crossover), Shuffle Crossover, Ring Crossover, etc. Let's start from the mutation, that is easier.
For simplicity I'll represent the ordered genome like an array of integers: int[] genome = {1, 2, 3, 4, 5};
Interchange mutation
The concept is pretty basic: to mutate an ordered genome you just swap two elements. Easy.

    public int[] InterchangeMutation(int[] genome)
    {
        int i1 = random.Next(0, genome.Length);
        int i2 = random.Next(0, genome.Length);

        var copy = genome.ToArray(); //just making a copy here
        copy[i1] = genome[i2];
        copy[i2] = genome[i1];

        return copy;
    }

PMX Variation Crossover
This is a bit more complicated as we have to take repetitions into account. From experience, I like to use this variation of the Partially Mapped Crossover. It is way easier to implement than the original one (you can find the paper online) but it will cost some more computational complexity. Longer the genome, higher the price you will pay.

Start by selecting two parents to use for the crossover.
From the first parent (P1) select a random section that will be passed over.
For the remaining values:
3A. If they are not in the copied section, take them from P2
3B. If they are in the copied section, take them from P1
3C. End up filling the gaps with the missing values in the order they are in P1

 public int[] PMX2Crossover(int[] P1, int[] P2)
    {
        //Initializing child genome
        int[] child = new int[P1.Length];
        for (int i = 0; i < P2.Length; i++) child[i] = -1;

        //Step1: getting random section to copy over
        int i1 = random.Next(0, P1.Length);
        int i2 = random.Next(0, P1.Length);

        //Step 2: Copying over section from P1
        for (int i = Math.Min(i1, i2); i < Math.Max(i1, i2); i++) child[i] = P1[i];

        //Step 3A: Copying values from P2
        for (int i = 0; i < P2.Length; i++) if (child[i] ==-1 && !child.Contains(P2[i])) child[i] = P2[i];

        //Step 3B: Copying values from P1
        for (int i = 0; i < P2.Length; i++) if (child[i] == -1 && !child.Contains(P1[i])) child[i] = P1[i];

        //Step 3C: Copying remaining values from P1
        int emptyGene = child.IndexOfFirst(-1);
        while (emptyGene != -1)
        {
            child[emptyGene] = FirstMissingGene(P1, child);
            emptyGene = child.IndexOfFirst(-1);
        }

        return child;
    }

    private int FirstMissingGene(int[] parent, int[] child)
    {
        foreach (var gene in parent) if (!child.Contains(gene)) return gene;
        return -1; // should never get here
    }

You can lower down the complexity of the crossover to O(n) (from O(n*n)) simply using a hashmap that keeps track of the genes already added to child.
To get the first child call PMX2Crossover(P1, P2); and for the second just swap the parents PMX2Crossover(P2, P1);
Hope this helps you.
Source: I have been a bachelor professor of AI for a period.

I edited the original post to clarify it even further. The current version of the question is an attempt to be the question to your answer, which seems valuable (that's why I tried to save this post by reformulating the question, in the first place). Please, make sure that the current version of the question is really the question that you're addressing in this answer.

You could use np.random.choice to shuffle the arrays. You could use a distance metric to find new arrays that are mutants of the the current good set.


text:
Which AI methods are most appropriate for login face recognition?
I want to make a face authentication application. I need to approve the face during the login based on whether the registered face and the login face match.
Which are the possible appropriate AI methods or technologies for this task?

you shouldn't do this unless its just for fun. a simple photograph of the person wold suffice to break the login scheme, and hence it is insecure. the iPhone, which has Face ID, also works on the dimension of depth, as in it can recognize the contours of your face. this make it much harder to crack.

One of the methods which is quite fast and easy to implement.
You can do Principal Component Analysis (PCA) based face recognition. You can go through this paper for the theory behind it. For an example implementation you can see this blog post.
The process, roughly, is as following:
If you have a grayscale image of size $(20,20)$, then this image can be flattened to a vector of size $400$. If you have $5$ images of size $(20,20)$ then your data matrix will be of size $(5,400)$.
We wish to find the principal components of the distribution of faces, or the eigenvectors of the covariance matrix of the data matrix, which will be of size $(5,5)$. Each eigenvector (out of total $5$), of size $(5,1)$, accounts for some amount of variation among the images, and they are ordered by the amount (the eigenvalues corresponding to the eigenvectors) they account for. These eigenvectors can be thought of as a set of features that characterize the variation between the images. Recognition is done by projecting a new image (flattened to size $(400,1)$)into the subspace spanned by the eigenvectors of the covariance matrix and then classifying the face by comparing its coordinates in the face space with the coordinates of the known individuals.
You can see an implementation in this git repo.

why are we using grayscale images?

@BivinVinod Using grayscale image only to make an easy example.


text:
Can sequence-to-sequence models be used to convert source code from one programming language to another?
Sequence-to-sequence models have achieved good performance in natural language translation. Could these models also be applied to convert source code written in one programming language to source code written in another language? Could they also be used to convert JSON to XML? Has this been done?
There are plenty of models that generate source code (which looks like real source code) using RNNs, although the generated source code doesn't make logical sense. I haven't been able to find any models or examples that take valid existing code and convert it into different valid code.

I don't think common techniques would work very well for this. The reason being (currently) most models perform poorly at perfect results. By that I mean, they are prone to very small mistakes. Like in translating french to english, if done well, you might find a word containing one extra vowel, a small mistake any human would have no problem with. However, as you know, this would not pass in a programming language.


text:
Reinforcement learning without trajectories
Does it make sense to use Reinforcement Learning methods in an environment that does not have trajectories?
I have a lot of states and actions in my environment. However, there are no trajectories.
If the agent takes action $a$ in the state $s$, it reaches $s'$. In $s'$, it takes action $a'$ and reaches state $s''$. However, if it takes the reverse order of actions $a'$ and then $a$, it would reach the same state $s''$.
How reinforcement learning methods handle with this?

There is still a "trajectory" in your example, and in fact it also fits to a simple grid  world where $a$ is "go north", $a'$ is "go east" and $s''$ is the postion North East of  $s$. But that doesn't seem to match your concerns? I would guess that you mean that choice of $a$ has no impact on the transition $s \rightarrow s'$? Does $a$ influence anything else - what does the reward depend upon? Is the state progression deterministic in any useful way i.e. $s \rightarrow s'$ has a probability distribution that is not simply equiprobable random sampling from all possible states?

@NeilSlater Thanks for your answer.
I will make it more clear.
I have a set of actions that affect my environment, and I want to find them.
Executing these actions from any state affect the environment in the same way.
About the reward, I get a non-zero reward from the environment only at the end of the trajectory.
This reward is vary from state to state (most of the states are 0).

@NoamMoscovich Just edit your post to clarify the question and problem! This should also attract other people, because when you edit a post it will pop up again in the front page.

Please, I know this post is old, but try to clarify what your problem really is. As Neil said, it seems that there's a trajectory in your case.


text:
How can I predict the nutrients in dishes given the ingredients used to prepare them?
I want to know which algorithm will work most efficiently for calculating nutrients present in a food dish if I am giving the ingredients used in the food. Basically, let us assume that I want to make a health status for a person A based on the intake of food and based on it create a diet for him.

@DuttaA we are using a weighing scale which weighs the ingredients before cooking food. Based on it, I want to calculate the nutrients content. Like a particular vegetable has x amount of vitamin A but after boiling it, it drop downs to A - Z. So this A - Z needs to be updated to a specific person's data and use it to calculate how healthy he is and based on it create a diet for him.


text:
If you have a very distorted image, would affine transformations applied to images make object detection algorithms make more mistakes?
If you have a very distorted video/image, would affine transformations of the images make object detection algorithms make more mistakes compared to a normal camera?

By affine transformations, do you mean rotations and scaling (i.e. do you have in mind something more specific)? You should also explain why you're asking this question.


text:
Are there any approaches other than deep learning to deal with unexpected questions in a question answering system?
I'm working on a question answering bot as my graduation project. The main concept is having a text file with many sentences, and building a question answering bot which answers a user's question based on the text file in hand.
Until now, I used tf-idf and cosine similarity and the results are somewhat satisfactory. The main problem is, if the user was to ask a question that doesn't have a word that is in the text file, my bot can't deduce what to bring back as an answer. For example, if I have a sentence in my text file that says "I have a headache because my heart rate is low", if the user was to ask "Why do you have a headache?", my bot chooses the correct sentence, but if he asked "What's wrong with you?" my bot doesn't know what to do.
All I've seen on the web until now are deep learning methods and neural networks, such as LSTM and such. I was wondering if there are any pure NLP approaches to go with my requirements.

Maybe you could take the most of the WordNet database to find synonyms and so on: https://wordnet.princeton.edu/


text:
How to understand the concept of self-supervised learning in AI?
I am new to self-supervised learning and it all seems a little magical at the moment.
The only way I can get an intuitive understanding is to assume that, for real-world problems, features are still embedded at a per-object level.
For example, to detect cats in unseen images, my self-supervised network would still have to be composed exclusively of cats.
So, if I had 100 images of cats and 100 images of dogs, then I thought self-supervised approaches would learn the features of the images. For example, if an image is rotated 90 degrees, it learns what was in the image that was rotated 90 degrees. However, if I wanted to classify just cats using this representation, then I wouldn't be able to do so without separating out what makes a cat a cat and a dog a dog.
Is my assumption correct?

This question is very related to this one, but you're specifically asking 1. "Why SSL really works?" and 2. "What is the relationship between the downstream task and the data used to learn representations?", if I understand correctly. Can you confirm that? If those are your questions, none of the current answers below seem to really address them. They only state what SSL is, which you already know, apparently.

I don't think your interpretation is correct. Take images as example.

Supervised Learning
e.g. classification (maybe use CNN with a L2 loss function)
Assume you have many images with different labels. You wish to find a function to approximate the function $y=f(x)$ given a lot of $(\hat x, \hat y)$ sample pairs.
Unsupervised Learning
e.g. clustering (maybe use k-means)
Assume you have many images, but we don't have the labels or we just want to see if there's a way to categorize them into different categories. So we cluster the images by some  characteristic that isn't pre-defined.
Self-Supervised Learning
e.g. super resolution (maybe use CNN with a L2 loss function)
You have many high resolution images without labels, but, your target is to train a model to up sample a low resolution image. So you can have the high resolution images as target, and down size the image to be the input, and try to train the image pairs. So the target is not some manually tagged labels, but generated directly from the data.

The contents of this answer seem to be correct, but maybe you should address the part of the OP "For example, to detect cats in unseen images, my self-supervised network would still have to be composed exclusively of cats...". The OP already seemed to know what SSL is. They wanted to understand  why it really works and the relationship between the downstream task and the pre-text task (and data), if I understood correctly. Your answer doesn't address this.

Andrew Zisserman, who is a pioneer in the field of self-supervised learning, described self-supervised learning in a talk at ICML as:

Self-supervised Learning is a form of unsupervised learning where the data provides the supervision. In general, we withhold some part of the data and task the network with predicting it. The network is forced to learn what we really care about e.g. a semantic representation, in order to solve it.

Thus, self-supervised is a subset of unsupervised learning, where you generate the labels from the given data itself. There are a few patterns of research being done for self-supervised learning:
1. Reconstruction:
In this, researchers have set up pretext tasks as predicting the color image from gray-scale image (Image Colorization), predicting the high-resolution image from the low-resolution version (Image Super-resolution) and removing some part of the image and trying to predict it (Image Inpainting).
2. Common Sense Reasoning:
You could take patches of 3x3 images and shuffle the patches and ask the network to predict the correct order (Jigsaw puzzle).
Similarly, you could take the center patch and some random patch and train model to predict where the random patch is located in relation to the center patch (context prediction).
There is another approach where you randomly rotate image into {0, 90, 180, 270} degrees and ask the model to predict the rotation angle applied (Geometric Transformation Recognition).
3. Clustering:
You could cluster the images into K categories and treat those clusters as labels. Then, a model can be trained on those clusters and you get representations. You can again repeat clustering and model training for few epochs. Papers for these include: DeepCluster and Self-Labelling.
4. Contrastive Learning:
In this paradigm, augmentations of the image is taken and the task is to bring two augmentations of the same images near while making the distance between this image and some other random image far. Papers for these include: SimCLR and PIRL.

The contents of this answer seem to be correct, but maybe you should address the part of the OP "For example, to detect cats in unseen images, my self-supervised network would still have to be composed exclusively of cats...". The OP already seemed to know what SSL is. They wanted to understand  why it really works and the relationship between the downstream task and the pre-text task (and data), if I understood correctly. Your answer doesn't address this.


text:
How to perform binary classification when one class is more predominant than the other?
Assuming we have big $m \times n$ input dataset, with $m \times 1$ output vector. It's a classification problem with only two possible values: either $1$ or $0$.
Now, the problem is that almost all elements of the output vector are $0$s with a very few $1$s (i.e. it's a sparse vector), such that if the neural network would "learn" to give always 0 as output, this would produce high accuracy, while I'm also interested in learning when the 1s occurs.
I thought one possible approach could be to write a custom loss function giving more weight to the 1s, but I'm not completely sure if this would be a good solution.
What kind of strategy can be applied to detect such outliers?

I like this discussion about accuracy as a scoring rule: https://stats.stackexchange.com/q/312780/247274.

From your case, it seems like you want your algorithm to classify both 1s and 0s with high accuracy. To increase the number of 1s and get it to a comparable level as 0s, you could generate new examples of 0s by tweaking some features or adding random noise.
If you don't care about classifying 0s and only care about classifying 1s (which doesn't seem like what you want to do but putting this out there), you can create a surrogate loss function which assigns more loss weight to 1s than 0s (eg - 1000 weight for 1s and 1 weight for 0s.)

As described in this post, this problem is known as "unbalanced dataset" problem, which can have different solution approaches. If you use supervised learning, augmentation approaches
could  help. Otherwise, unsupervised approaches need some proper distance measure for outliers detection.


text:
Does TD(0) prediction require Robbins-Monro conditions to converge to the value function?
Does the learning rate parameter $\alpha$ require the Robbins-Monro conditions below for the TD(0) algorithm to converge to the true value function of a policy?
$$\sum \alpha_t =\infty \quad \text{and}\quad \sum \alpha^{2}_t <\infty$$

This question was also asked at https://stats.stackexchange.com/q/451115/82135.

The paper Convergence of Q-learning: A Simple Proof (by Francisco S. Melo) shows (theorem 1) that Q-learning, a TD(0) algorithm, converges with probability 1 to the optimal Q-function as long as the Robbins-Monro conditions, for all combinations of states and actions, are satisfied. In other words, the Robbins-Monro conditions are sufficient for Q-learning to converge to the optimal Q-function in the case of a finite MDP. The proof of theorem 1 uses another theorem from stochastic approximation (theorem 2).
You are interested in the prediction problem, that is, the problem of predicting the expected return (i.e. a value function) from a fixed policy. However, Q-learning is also a control algorithm, given that it can find the optimal policy from the corresponding learned Q-function.
See also the question Why doesn't Q-learning converge when using function approximation?.

Thanks for that, I was wondering is it true that TD(0) converges to the value function in Expected value when $\alpha$ is constant? Is there a condition that $\alpha \in (0,1)$?

@KaneM Right now, I am not aware of this result. Please, ask another question on the site because may someone else can answer your question.

The result is in Sutton's Learning to Predict by the Methods of Temporal Differences , on page 24 of this version. I think I understand the result now. It is saying that there is some range of  alphas $\alpha \in (0,t)$ such that TD(0) converges to the value function in expected value.


text:
Is a dystopian surveillance state computationally possible?
This isn't really a conspiracy theory question. More of an inquire on the global computational power and data storage logistics question.
Most recording instruments such as cameras and microphones are typically voluntary opt in devices, in that, they have to be activated before they start recording. What happens if all of these devices were permanently activated and started recording data to some distributed global data storage?
There are 400 hours of video uploaded to YouTube every minute.
Let’s do some very rough math.
I’m going to assume for the rest of this post that the average video is 1080p which is 2.5GB (or $10^9$ bytes) per hour. From that, we get about 400 hrs * 60 mins * 2.5GB/hrs * 24 hrs = 1.5 petabytes (or $10^{15}$ bytes) per day.
But YouTube videos post are voluntary, and they are far from continuous video streams.
There are about 3.5 billion smartphones in the world. If video was continuously streamed and recorded, going through the same video math above ($3.5 * 10^9 * 1.5 * 10^{15} * 24)$ = 126 yottabytes (or $10^{24}$ bytes) per day.
The IDC projects there will be 175 zettabytes (or $10^{21}$ bytes) in 2025.
Unless my math is very wrong, it would seem as though smartphone cameras alone could produce more data in one day than all of the data created in human history in 2025.
This, so far, has only been about the data recording, but, to implement a surveillance state, all recorded data would need to be processed by AI to intelligent flag data that is significant. How much processing power would be needed to filter 126 yottabytes into relevant information?
Overall, this question is motivated by the spread of dystopian surveillance media like Edward Snowden NSA whistle blowing leaks or George Orwell's sentiment of "Big Brother is Watching You".
Computationally, could we be surveilled, and to what extent? I imagine text messages surveillance would be the easiest, does the world have the computation power to surveil all text messages? How about audio? or video?

Note that your 2.5GB/hour number is typically used for TV/movie video, which is (deliberately) very dynamic by nature.  Video from most other sources is often much more compressible (e.g. most surveillance cameras spend long periods with identical video frames, or large portions of the scene not changing, cell phones are often in somebody's pocket, etc).  Also, video doesn't have to be 1080p to be useful for surveillance.  A lot of surveillance video even from 1080p-capable cameras is actually sent/stored at much lower resolution because HD is just not needed for most surveillance purposes.

How is this question strictly related to AI? Maybe make it clearer!

I think your question isn't really meaningful as asked.  You add up a bunch of numbers of how much data is produced and then ask if it could all be processed. Processed how?  You could use all the compute power of the universe to process 1 megabyte in infinite ways.  Obviously won't be doing that... The game is using available compute power efficiently (only in places and to the degree useful for some goal) on these large data sets.

You don't necessarily have to analyse it all. Just by having such data available you can achieve a lot in terms of surveillance, as long as you can retrieve relevant parts.
A few years ago there was a Radiolab podcast, "The Eye in the Sky" (there's a full transcript on the site). The basic idea is that you have a plane circling a city 24/7, and filming what goes on. If there was a crime somewhere, you retrieve the recordings after the event, and you can track back to where vehicles involved in the crime were coming from, and where they went after the crime. If nothing happens, you simply archive the data, and perhaps remove it after a month or so.
This method was used to solve a hit-and-run assassination of a police woman who was on her way to work. The gang who committed the attack were rather surprised when the police showed up at their secret hide-out a few days later, as they could see on the images where the cars involved went to later. At the time and place of the murder there were obviously no witnesses who could have done that. And this involved no computational processing at all.
The possibilities this opens up are just scary, as you can track pretty much anybody's movements without actually needing someone to follow them. Add to that street-level CCTV, and not much can happen without you being able to find out.
In this scenario there is no processing at all, but you could imaging simple processing steps, such as tracking vehicles or changes in the environment, which could be used to give clues about potentially 'interesting' events. So instead of using it 'passively' as a kind of memory, you could use that data to identify things that happened that you weren't aware of.
And this is without even any clandestine access to people's data. If you add that dimension, then you might even be able to identify crimes/etc before they even happen. Text processing can be quite fast, but is not easy to do, as presumably few people would openly communicate about things they were planning. So I guess we're still a long way away from that.
Of course there is the ethical dimension (which is mentioned in the podcast): who has access to that data, and who decides what it is used for? If you do, and you suspect your partner of being unfaithful, who/what would stop you from checking out their movements? Or check up on that politician who might have a secret affair, or a gambling issue, or who keeps being in the same locations as a well-known drug dealer. All rather scary.
While a complete analysis of all such data would be very heavy computationally, and fraught with false positives and recall problems, it might simply be enough to index it by time, location, and perhaps people involved (face recognition seems to be reasonably good, though still with a rather high error rate). This is enough already to make me feel worried about the future.

I also think worth adding to this good answer the one-trick item I was going to write: Edge processing & filtering. It is possible to reduce the amount of device-level data down a lot - orders of magnitude - by filtering only to active/interesting scenarios, e.g. there's no point videoing the inside of someone's pockets. Embedded AI assists with this by adding smarter filters, not just "is there a signal", but "is something interesting happening", which can decrease centralised bandwidth, CPU and storage requirements yet further.

And if you're writing a book, you can have the protagonists buy 100 phones and hack them to return false data. (Someone did this recently to confuse Google Maps)

@user253751 Not actually false data, he just loaded them on a cart, walked through his street, and Google Maps concluded there was a traffic jam because 100 "vehicles" were moving slowly. No hacking necessary.

@gerrit It's false data because there weren't 100 vehicles moving slowly in a traffic jam.

@user253751 The data are real, but Googles algorithms drew incorrect conclusions from real data.

You would also want to consider physical limitations. If you are even storing 126 yottabyte of data per day, then if we look at the current theoretical densest data storage medium, DNA, at 215 petabytes per gram, we get...
${(126 * 10^{24}) \over (215 * 10^{15})} = 586046511$ grams per day
586046511 g = 586046 kg = 586 Metric Tonnes just for storage.

to put that number in perspective, note that it's actually a bit less than the 695 metric tonnes of garbage produced daily by the US. (tI took numbers from : https://www.treehugger.com/environmental-policy/trash-numbers-startling-statistics-about-americans-and-their-garbage.html)

that's very sad

The answer is really very simple. If you have the dystopian power over all the mobile devices in the first place, you would not make them send all their data over to any "global data storage" just like that. Instead, you would have put a local AI on each device that filters, processes, categorizes and flags the important parts, sending only those parts plus an intelligent summary of the remaining data to a global AI. The global AI combines and synthesizes the parts that all the local AIs send to it, and may request further data from the local AIs based on what it wants to know.
Naturally, since you are a competent dystopia architect, you design each local AI to be intelligent enough to subvert any human's attempt to remove it, stop its activity, or otherwise interfere with its data collection and processing. The local AIs also continuously communicate in a distributed network with other local AIs regarding their status and any adversarial activities, so that they can quickly act to defend themselves if the need arises, and also notify the global AI of any attack. In this surveillance state, it is an easy task for the global AI to send armed agents to deal with any threat to the AI network that manages to gain any foothold in the information cyberspace.
The point is that the most durable dystopia is a defended distributed dystopia, which would make it robust and scalable.


text:
Irregular results while prediction identical object on same image
I used the pre-trained model faster_rcnn_resnet101_coco.config with my own dataset.
I have two issues

some objects were not detected, while I learned it, with a high number of steps, and test over the same learning data, but still have some missing data
sometimes, the same identical objects predicted correctly on some images and sometimes it got missed, although the objects are identical

Any help would be appreciated.


text:
How is the gradient with respect to weights derived in batch normalization?
At the bottom of page 2 of the paper L2 Regularization versus Batch and Weight Normalization, the equation for the gradient of the output with respect to the weights is given as:
$$
\triangledown y_{BN} (X; w, \gamma, \beta)  = \frac{X}{\sigma(X)}\gamma g'(z).
$$
Can someone break down into smaller steps on how the author got to that equation?

Hi and welcome to AI SE! Meanwhile, if you don't receive an answer, what you can do is try to understand the formula step by step. To do that, you first need to understand all the terms in it. For example, what is $X$? It should be the input. $g$ should be the non-linearity? What are all other terms?

Watch this: https://youtu.be/d14TUNcbn1k?t=354. If you are still unsure of anything in relation to backprop, respond here and I'd be happy to help