text:
Which AGI systems have already been implemented and tested?
I wish to compile a (somewhat) comprehensive list of AGI systems that have actually been created and tested (to whatever degrees of success) instead of those that simply advertise they are going to 'do' something about it or have patented theoretical concepts.
For the purposes of this question, we can use the following definition of AGI:

Artificial general intelligence (AGI) is the intelligence of a machine that can understand or learn any intellectual task that a human being can

As far as I know, no "true" (i.e. as intellectual and physically capable as a human) artificial general intelligent system (AGI) has been implemented or is practically useful (this is confirmed by Ben Goertzel, who is one of the leading researchers in AGI [1, 2]).
The closest to a practical AGI might be Sophia (or similar robots), which may look like an AGI, but it lacks several capabilities that we humans have and its ability to adapt to new circumstances is limited. Sophia uses OpenCog [1], which is supposed to be a software framework to develop AGI. Sophia is used for the loving AI project.
There are also theoretical frameworks for AGI, such as AIXI, which, in any case, have several flaws, such as incomputability (in the case of AIXI). There are approximations of AIXI, but these approximations can only be used to solve toy problems (such as tic-tac-toe), so they aren't really useful to solve complex real-world problems. However, it is possible that better approximations to AIXI and more theoretical frameworks for AGI will be developed that can deal with more complex problems.
It's also important to note that, despite their success, AlphaGo and AlphaStar are narrow AI systems, as they can only solve one specific problem (although the same approach can be adapted to solve very similar problems too, e.g. AlphaZero).
If you want to know more about AGI, the Scholarpedia's article Artificial General Intelligence, curated by Ben Goertzel, provides a good overview of the AGI field, including but not restricted to definitions of AGI and approaches to the development of AGI systems, such as

universal (AIXI was created based on this approach),

symbolic (Soar was created based on this approach), which is based on the physical symbol system hypothesis,

emergentist (or sub-symbolic), which is based on the idea that general intelligence is expected to emerge from sub-symbolic dynamics (where a sub-symbolic system refers e.g. to an artificial neural network),

hybrid (e.g. CLARION), which is a combination of the universal, symbolic or sub-symbolic approaches.

You probably also want to take a look at this question, which has a few answers that mention a paper and book that you can read to know more about the AGI field. You may also be interested in this related question too.


text:
A problem about the relation between 1-oracle and 2-oracle PAC model
This problem is about two-oracle variant of the PAC model. Assume that positive and negative examples are now drawn from two separate distributions $\mathcal{D}_{+}$ and $\mathcal{D}_{-} .$ For an accuracy $(1-\epsilon),$ the learning algorithm must find a hypothesis $h$ such that:
$$
\underset{x \sim \mathcal{D}_{+}}{\mathbb{P}}[h(x)=0] \leq \epsilon \text { and } \underset{x \sim \mathcal{D}_{-}}{\mathbb{P}}[h(x)=1] \leq \epsilon$$
Thus, the hypothesis must have a small error on both distributions. Let $\mathcal{C}$ be any concept class and $\mathcal{H}$ be any hypothesis space. Let $h_{0}$ and $h_{1}$ represent the identically 0 and identically 1 functions, respectively. Prove that $\mathcal{C}$ is efficiently PAC-learnable using $\mathcal{H}$ in the standard (one-oracle) PAC model if and only if it is efficiently PAC-learnable using $\mathcal{H} \cup\left\{h_{0}, h_{1}\right\}$ in this two-oracle PAC model.
However, I wonder if the problem is correct. In the official solution, when showing that 2-oracle implies 1-oracle, the author returns $h_0$ and $h_1$ when the distribution is too biased towards positive or negative examples. However, in the problem, it is required that only in 2-oracle case we can return $h_0$ and $h_1$. Therefore, in this too-biased case, it seems that there may not exist a 'good' hypothesis at all.
Is this problem wrong? Or I make some mistake somewhere?

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How can I show that the VC dimension of the set of all closed balls in $\mathbb{R}^n$ is at most $n+3$?
How can I show that the VC dimension of the set of all closed balls in $\mathbb{R}^n$ is at most $n+3$?
For this problem, I only try the case $n=2$ for 1. When $n=2$, consider 4 points $A,B,C,D$ and if one point is inside the triangle formed by the other three, then we cannot find a circle that only excludes this point. If $ABCD$ is convex assume WLOG that $\angle ABC + \angle ADC \geq 180$ then use some geometric argument to show that a circle cannot include $A,C$ and exclude $B,D$.
For the general case I’m thinking of finding $n+1$ points so that a ball should be quite ‘large‘ to include them, and that this ball can not exclude the other 2 points.  However, in high-dimensional case I do not know how to use maths language to describe what is ‘large’.
Can anyone give some ideas to this question please?

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How does non-max suppression work when one or multiple bounding boxes are predicted for the same object?
My understanding of how non-max suppression works is that it suppresses all overlapping boxes that have a Jaccard overlap smaller than a threshold (e.g. 0.5). The boxes to be considered are on a confident score (may be 0.2 or something). So, if there are boxes that have a score over 0.2 (e.g. the score is 0.3 and overlap is 0.4) the boxes won't be suppressed.
In this way, one object will be predicted by many boxes, one high score box, and many low confident score boxes, but I found that the model predicts only one box for one object. Can someone enlighten me?
I currently viewing the ssd from  https://github.com/sgrvinod/a-PyTorch-Tutorial-to-Object-Detection
Here is the code.
#Finding Jaccap Overlap and sorting scotes
class_scores, sort_ind = class_scores.sort(dim=0, descending=True)
class_decoded_locs = class_decoded_locs[sort_ind]  # (n_min_score, 4)
overlap = find_jaccard_overlap(class_decoded_locs, class_decoded_locs)
suppress = torch.zeros((n_above_min_score), dtype=torch.uint8).to(device)

for box in range(class_decoded_locs.size(0)):
# If this box is already marked for suppression
    if suppress[box] == 1:
    continue
    suppress = torch.max(suppress, overlap[box] > max_overlap)
    suppress[box] = 0

Hello welcome to AI.SE!

I might be able to help with the theory, but the coding... it is a non standard API such as Tensorflow or Pytorch (it might be custom code for what I can tell).
The key element here is that the bouding boxes are removed only if they hold a prediciton for the same class that the box that is overlapping with (but with less confidence, that is why it gets removed).
Here is an example, where we have:

Two classes $c \in [c_1, c_2] = [$ "star"$, $ "moon" $]$
Three bounding boxes

The blue bounding boxes holds prediction for the class $c_1$ so their predictions are $p(c_1)_{box1} = 0.8$ and $ p(c_1)_{box2} = 0.9$. On the other hand, the green box holds a prediction for the class $c_2$.
The three boxes are highly overlaping so the overlap between any box $x$ with any box $y$ will be above the IoU threshold: $IoU(box_x, box_y) > 0.5$. So in principle all boxes are suceptible to be removed.
However the NMS only applies for boxes predicting the same class (in the case the blue one). So the NMS algorihtm is: if the boxes are overlapping, $IoU(box_1, box_2) > 0.5$, which is true, remove all non maximal class probability boxes. Said differently, take just the box with highest $p(c_1)$ and remove the rest. So the $box_1$ with class probability $p(c_1) = 0.8$ would be removed.
So what happens with green box? Isn't it overlapping as well? Yes, but consider that the green box is not trying to predict the same object, is trying to predict another object, $c_2$, which happens to be very close to the first object, $c_1$. This way object detectors support detection of different overlapping objects.


text:
How can I match numbers with expressions?
Let's say I have the number 123.45 and the expression one hundred twenty-three and forty-five cents.
Can I develop AI to identify these two values as a match? If I can, how should I do that?

Machine learning is massive overkill. Just convert the numbers to word: https://stackoverflow.com/questions/19504350/how-to-convert-numbers-to-words-in-python


text:
Why don't the neural networks inside LSTM cells contain hidden layers?
I watched a video explaining how LSTM cells have very rudimentary feed-forward neural networks, basically a 2 layer input-output with no hidden layers.
Why don't LSTM cells have more complex neural networks before each gate, i.e. containing 1 hidden layer?
I would think that if you want more advanced gating decisions, that you would use at least 1 hidden layer to have more complex processing.

What is the video? Maybe it was considering the nodes themselves, as you can have multiple layers of LSTM nodes without problem, try it :)

The reason is likely because the benefits of increasing the number of hidden units in an LSTM outweigh (or are essentially the same as) the benefits from increasing the depth before each node. Each of these increase computation time, so you should always chose the one that has the largest benefit. This conclusion was likely come to simply through experimentation. I can not find any papers on this topic however, but I would guess it's been done as a small segment of a paper and was probably quickly shown to have minimal or no benefit

Mostly LSTM dealing with sequence of low-dimensional samples, like words (below 300 dims). For LSTM dealing with high-dimensional samples like images (much more then 10^3 dims) LSTM could be put on top of traditional feed-forward network like CNN


text:
Why is the average reward plot for my reinforcement learning agent different than the usual plots?
I'm building an RL agent using SARSA and Q-Learning for testing its capabilities.
The environment is a 10x10 grid, where it gets a reward of 1 if he reaches the goal while he takes -1 every time he takes a step out of the grid. So, it can freely move out and every time it takes a step outside of the grid it gets -1.
After tuning the main parameters

alpha_val: 0.25
discount: 0.99
episode_length: 50
eps_val: 0.5

I get the following plot for 10000 episodes (The plot is sampled every 100 episodes):

But when I look at the plots online I see usually plots like this one:

Since I'm new at RL, I'm asking some comments about my outcome or any type of suggestion if anyone of you think that I'm doing something wrong.

You need to optimize your action selection method

Well, the way to know that the agent is actually learning is by looking at its behavior while it performs the task, and by comparing against a known optimal performance.
So, does your agent reaches the goal quickly? Does it step out of the grid frequently? What is the maximum possible sum of rewards / minimum number of steps attainable? Is the agent close to that limit? From your graphic, and if I understood correctly your RL problem, the maximum average reward per step should be close to 1 (depending on the specific environment you are using), so I guess you are not so far from the optimal solution.
Also, probably if you keep training for a longer period, your agent will reach a stable solution that might or might not be optimal. If you keep training after that, your curves surely will look like the ones you found online.

Thank you for your answer. Yes, the maximum average reward per episode is 1 and yes, the agent a the end achieve a good average reward. My doubt is that it takes so much time and for more than 5000 episodes (which have length 50 each one) it reach the goal less than 40% of the time. The grid is 10x10 and only the goal gives a reward of +1. I wonder if this problem is caused by the fact that in reality the space of movement is infinite since it can move freely out of the grid taking a -1 of penalty for each step outside. The question is why it takes so many episodes to get a nice reward?

I also tried for 100k episodes by it is unstable. I noticed that it stay many time stuck just outside the grid and it stay there.  Since its actions are (Up,Down,Left,Right,Stay)

This could mostly be depending on your exploration rate. Consider this exploration threshold:
exploration_threshold = EPS_END + (EPS_START - EPS_END) * math.exp(-1. * episodes / EPS_DECAY)
sample = random.random()
if sample > exploration_threshold:
      # take action using the policy
    else:
      # take action at random

Here the EPS_END=0.9, EPS_START=0.05 are starting and ending exploration probabilities. This will force the agent explore the action space with a decaying high probability and gain experiences. After a while it will choose actions from the policy with a high probability, and as the policy converges, it yeilds the maximum expected utility. This will help learn faster and cause the decaying exponential shape of the convergence plot.


text:
If I trained a model to perform semantic segmentation on images with only one object, would it also work on images with multiple objects?
I'm working on semantic segmentation tasks in the medical space using the U-Net. Let's say that I train a U-Net model on medical images with the goal of segmenting out, say, ligaments, from a medical image. If I train that model on images that contain just a single labelled ligament, it will be able to segment out single ligaments pretty well, I assume. If I present it with an image with multiple ligaments, should it also be able to segment the multiple ligaments well too?
Based on my understanding, semantic segmentation is just pixel-wise classification. As a result, shouldn't the number of the objects in the image not be relevant since it's only looking at individual pixels? So, as long as a pixel matches that of a ligament, it should be able to segment it equally right?
Or am I misunderstanding some piece?
Basically, if I train a U-Net on images with just single ligaments, will it also be able to segment images with multiple ligaments equally as well based on my logic above?

I'm not aware of a model that could generalize from single segment to multi segment without nontrivial surgery, augmentation, and retraining, but I have had some success doing this in cases where the regions to segment are non-overlapping: After finding the first segment, edit the input image to distort or obscure the segmented region and run it through again, repeating until it stops finding new segments.

Without experimental evidence to back me up, I can not answer this with 100% confidence. However, I am fairly certain that this will cause issues depending on the model.
U-net is essentially an auto-encoder, and due to the fact that it is all just one big neural network, it is likely it will learn the easiest pattern (as all NN do), and that is to find one single instance of an object and shade that region.
Now why does this depend on the model? Well let's say you are using something slightly different, where region proposals are generated by a determinisitc algorithm we've predefined, then these regions are run through a CNN to segment them. In this case, as each region is without context of the entire image, the difference between 2 objects in an image and 1 is indistinguishable to the network (as regions may overlap), and as such, only using images with 1 object will not pose any problems (there is a name for early models like these, though it escapes me).
So assuming I am correct, what should you do? The models that use a deterministic algorithm for region proposals are slow and old, so I wouldn't suggest that. Instead, I would think that you should first do some testing, to see if it actually does cause issues. Assuming it does, a good option could be to tamper with the training data and separate segments by a few pixels to sort of "force" multiple objects into existence.
Regardless of such, I would still suggest using U-net. Fixing this issue (if it does arrise) should be relatively easy to do, so there's little to lose by using U-net and just trying the training.

In the problem of Semantic Segmentation one solves the problem of investigating, whether a particular pixel on the image belongs to a certain class. For example, on the photo taken from the street one may be interested , whether this pixel belongs to the road, traffic light, sign, people or background of no interest.
Supeficially, in this setting it doesn't matter, whether there is a single instance of an object or multiple, and the thing, which is more important is the abundance of a particular class on an image.
However, CNN adopt to a particular patterns, pertinent to the distribution, from which the data comes from. It can be the case, that a network has captured some property and tooks a pattern, which is relevant only for one instance case. Images with multiple instances of class of interest are like outliers in the data, and may be treated due to this incorrecty.
I am not sure, whether it makes sense in yout particular problem, but maybe it is worth performing a mosaic augmentation https://towardsdatascience.com/data-augmentation-in-yolov4-c16bd22b2617. So that you will have for some samples of the augmented data a multiinstance case.

I think you misunderstood the question. The question is not about the difference between semantic and instance segmentation. The question is whether the model would generalize and be able to segment images with a different number of objects than the number of objects that can be found in the training data. I think the OP is not interested in knowing whether the model would be able to differentiate the instances of the same type of object (i.e. they were not asking about instance segmentation). So, I don't think that this answer really addresses the question.

@nbro I mean that superficially due to the absence of notion of single or multiple instances of some class the task is the same. However, if in the training data there are instances with only single intstance, and in the test data with multiple, it is likely to be treated as outlier, different from the distribution for which the NN was trained. Should I delete my answer?

Your answer contains useful information, but maybe you should also add some information that more directly addresses the question. So, I wouldn't delete your answer, if you plan to include this info.


text:
Why am I getting the logarithm of the probability bigger than zero when using Neural Spline Flows?
I am using a normalizing flow (Neural Spline Flows) to approximate a probability. After some training, the average loss is around 0.5 (so the logarithm of the probability = -0.5). However, when I am trying it on some new test data, I am getting some values of the logarithm of the probability bigger than zero, which would mean that the probability for that element is bigger than one (which doesn't make sense).
Does anyone know what could cause this? Isn't the flow supposed to keep all the probabilities below 1 automatically?

I have always seen the negative odd-logs ration when doing similar work. As it is now, I don't think you are constraining the splines appropriately Link to reading


text:
What is the mathematical definition of an activation function?
What is the mathematical definition of an activation function to be used in a neural network?
So far I did not find a precise one, summarizing which criterions (e.g. monotonicity, differentiability, etc.) are required.  Any recommendations for literature about this or – even better – the definition itself?
In particular, one major point which is unclear for me is differentiability. In lots of articles, this is required for the activation function, but then, out of nowhere, ReLU (which is not differentiable) is used. I totally understand why we need to be able to take derivatives of it and I also understand why we can use ReLU in practice anyway, but how does one formalize this?

There is no strict definition of suitability of an activation function for neural networks. Instead there are a number of desirable traits, and functions that don't meet them or come close enough may perform badly in general (but those functions may still work in specific cases)
If you are using gradient descent as a training method, then differentiability is closest to being a requirement. However, even then as you noticed with ReLU, it is not an absolute requirement, provided behaviour close to discontinuities is good. For example $\frac{1}{x}$ or $log{x}$ make bad choices here due to how they behave for values near to $0$.
Wikipedia summarises some important desirable traits, plus compares popular activation functions in this table.

Nonlinear – When the activation function is non-linear, then a two-layer neural network can be proven to be a universal function approximator. The identity activation function does not satisfy this property. When multiple layers use the identity activation function, the entire network is equivalent to a single-layer model.
Range – When the range of the activation function is finite, gradient-based training methods tend to be more stable, because pattern presentations significantly affect only limited weights. When the range is infinite, training is generally more efficient because pattern presentations significantly affect most of the weights. In the latter case, smaller learning rates are typically necessary.
Continuously differentiable – This property is desirable (RELU is not continuously differentiable and has some issues with gradient-based optimization, but it is still possible) for enabling gradient-based optimization methods. The binary step activation function is not differentiable at 0, and it differentiates to 0 for all other values, so gradient-based methods can make no progress with it.
Monotonic – When the activation function is monotonic, the error surface associated with a single-layer model is guaranteed to be convex.
Smooth functions with a monotonic derivative – These have been shown to generalize better in some cases.
Approximates identity near the origin – When activation functions have this property, the neural network will learn efficiently when its weights are initialized with small random values. When the activation function does not approximate identity near the origin, special care must be used when initializing the weights.

You can see that ReLU is an outlier on more than one of these traits. The reason that it is popular is that despite these weaknesses, the things it does well - speed of operation plus helping combat the vanishing gradient problem - make it a solid choice for very deep networks. In fact the success of ReLU has inspired use of a number of similar-looking activation functions that hope to keep its benefits whilst having more of these desirable traits.
Ultmately though, almost any non-linear function can be used successfully in a neural network. The more of the desirable traits it has, the more likely you can use it in general cases with existing approaches to training.


text:
What's the difference between semi-supervised VAEs and conditional VAEs?
Can someone explain the difference? I'm assuming the difference is just that the neural nets representing the encoder and decoder are trained in a semi-supervised manner in semi-supervised VAE, which in conditional the approximation to the posterior and the posterior's distributions are conditioned on some labels? So, I'm guessing that semi-supervised VAE affects the loss evaluation, whereas, in conditional VAEs, the inference network is conditioned on another label as well?


text:
In AlphaZero, which policy is saved in the dataset, and how is the move chosen?
I've been doing some research on the principles behind AlphaZero.
Especially this cheat sheet (1) and this implementation (2) (in Connect 4) were very useful.
Yet, I still have two important questions:

How is the policy network updated? In (2), board positions are saved in a dataset of tuples (state, policy, value). The value is derived from the result of the self-played game. However, I'm not sure which policy is saved: the number of times that each move has been played, the prior probabilities for each move (I guess not), or something else?

The cheat sheet says that (for competitive play) the move is chosen with the greatest N (=most visited). Wouldn't it be more logical to choose the move with the highest probability calculated by the policy head?

However, I’m not sure which policy is saved

The policy from the Monte Carlo tree search is stored, as we can get the policy estimate from the network later by passing the given state through the network, which is used to calculate the cross entropy loss to update the network's policy (summed with Mean squared error loss between value head's prediction and the actual value/reward).

Wouldn’t it be more logical to choose the move with the highest probability calculated by the policy head?

Depends on the number of searches you've performed, after thousands of simulations the MCTS would give better results, as it approximates the minimax tree.

To be more precise, I would say that "the visit counts" of the actions from the root noe of the Monte Carlo tree search are stored. Just "the policy" might be a bit too vague. Also, welcome to the site :)

@DennisSoemers but aren't the visit counts of each action used to calculate the policy?

Sure, but "policy" can mean a lot of things. It can mean what you intend it to mean, you're not wrong. But the strategy we use in the "Selection" phase (like PUCT) is also often called a "tree policy". In Reinforcement Learning, a policy would be a mapping from states to a probability distribution over actions. You could view the visit counts (after normalising) as being such a (discrete) probability disrtibution.. but it could also refer to the policy network being trained! Hence, there's room for confusion. Explicitly saying that the visit counts gets stored leaves less room for confusion :)


text:
Has anyone attempted to take a bunch of similar neural networks to extract general formulae about the focus area?
When a neural network learns something from a data set, we are left with a bunch of weights which represent some approximation of knowledge about the world.  Although different data sets or even different runs of the same NN might yield completely different sets of weights, the resulting equations must be mathematically similar (linear combinations, rotations, etc.).  Since we usually build NNs to model a particular concrete task (identify cats, pedestrians, tumors, etc.), it seems that we are generally satisfied to let the network continue to act as a black box.
Now, I understand that there is a push for "understandability" of NNs, other ML techniques, etc.  But this is not quite what I'm getting at.  It seems to me that given a bunch of data points recording the behavior of charged particles, one could effectively recover Maxwell's laws using a sufficiently advanced NN.  Perhaps that requires NNs which are much more sophisticated than what we have today.  But it illustrates the thing I am interested in: NNs could, in my mind, be teaching us general truths about the world if we took the time to analyze and simplify the formulae that they give us1.
For instance, there must be hundreds, if not thousands of NNs which have been trained on visual recognition tasks that end up learning many of the same sub-skills, to put it a bit anthropomorphically.  I recently read about gauge CNNs, but this goes the other way: we start with what we know and then bake it into the network.
Has anyone attempted to go the opposite way?  Either:

Take a bunch of similar NNs and analyze what they have in common to extract general formulae about the focus area2
Carefully inspect the structure of a well-trained NN to directly extract the "Maxwell's equations" which might be hiding in them?

1 Imagine if we built a NN to learn Newtonian mechanics just to compute a simple ballistic trajectory.  It could surely be done, but would also be a massive waste of resources.  We have nice, neat equations for ballistic motion, given to us by the "original neural networks", so we use those.
2 E.g., surely the set of all visual NNs have collectively discovered near-optimal algorithms for edge/line/orientation detection, etc.).  This could perhaps be done with the assistance of a meta-ML algorithm (like, clustering over NN weight matrices?).

Hi and welcome to this community! You say "Carefully inspect the structure of a well-trained NN to directly extract the "Maxwell's equations", but a neural network represents a function (or a set of functions). How do you plan to transform a function into an equation? Also, some neural networks have thousands or millions of parameters. How do you suggest inspecting the structure of such a big (trained) neural network? Furthermore, can you clarify how this is different than XAI?

My understanding of XAI is that it's like the SQL EXPLAIN PLAN for NNs.  "Justify your answer, but don't give me the full theory."  Whereas, extracting the approximated function would be like discovering the query engine itself.
How would you rediscover Newton's laws? You might say: "Ok, we've got some observational data on moving objects with known masses, locations and velocities.  Let's make the network predict accelerations or forces or momenta." (or permute the observables) Do something analogous for other "laws".
As for "how"? That's what the experts are for, right? ;)

To be more precise, XAI suggests to me that we can rationalize a particular (input, output) pair, as opposed to describing an over-arching framework which could effectively replace the NN itself as a computational shortcut (in exactly the same way as Newton's laws could replace a "ballistic NN").  As for analyzing overly large networks, we have ML tools to help us discover patterns in such things, no? If I could tell you exactly how to do it, I'd just do it myself. ;)

It's still unclear what you're looking for. You say "As for analyzing overly large networks, we have ML tools to help us discover patterns in such things, no?", so do you want to find encoded equations in the structure of a trained NN with another pattern recognition model? Or are you looking for an artificial scientist? Then, no, there is no artificial scientist (AFAIK). As I said, neural networks represent functions and not equations. The closest thing to what you're talking about that I recall is the neural programmer-interpreter (and similar stuff).

Every NN approximates some function, but only represents said function implicitly. The first step would be to develop a process which discovers an explicit, algebraic/analytical form for said function, perhaps by starting with simple candidates and iteratively enriching them until they converge on the NN's representation. Obviously, the hyperplane representing the ordinary least squares regression of the NN function would be the simplest starting point, and one could add higher-order terms until they stopped improving the fit.

However, "interesting" NNs likely represent a very complicated function which is the composition of much simpler "component" functions, and trying to infer such components may be possible with other ML techniques (like PCA, just to throw something out there...not sure how to directly apply it to this problem). Such a process is not an 'artificial scientist', because it doesn't uniquely identify a "simplest form". Humans using the tools would use their professional judgment to say: "Ah-hah! I spot an elegant formula embedded in this function!" Just as data scientists work their craft.

After several months, I came across this question again. I think that you're asking two distinct questions here: 1. "How to extract the commonalities that different neural networks may have?" and 2. "How to extract the explicit or symbolic representation of the function that a neural network represents?". These two questions are quite distinct, although somehow related. Despite that, I suggest that you edit your post to ask only one question here. Remove one of the two: the one that you don't like less. Then maybe create another post to ask the second question.

I provisionally closed this post until you focus on one of the questions. I will re-open it after you have followed my suggestion above.


text:
How can I extract the reason of the legal compensation from a court report?
I'm working on a project (court-related). At a certain point, I have to extract the reason of the legal compensation. For instance, let's take these sentences (from a court report)

Order mister X to pay EUR 5000 for compensation for unpaid wages

and

To cover damages, mister X must pay EUR 4000 to mister Y

I want to make an algorithm that is able from this sentence to extract the motive of legal compensation. For the first sentence

Order mister X to pay EUR 5000 for compensation for unpaid wages

the algorithm's output must be "compensation for unpaid wages" or "compensation unpaid wages ".
For the second sentence, the algorithm's output must be "cover damages". Output can be a string or a list of string, it doesn't matter.
As I'm not an NLP expert (but I have already worked on a project on sentiment analysis, so I know some stuff about NLP), and there are so many articles, I don't know where to start.
I'm working on French texts, but I can get away with working on English texts.

Maybe you can get away with using just regex and detect numbers followed by EUR or maybe detect n  years in jail phrase.


text:
How can a system recognize if two strings have the same or similar meaning?
How can a system recognize if two strings have the same or similar meaning?
For example, consider the following two strings

Wikipedia provides good information.
Wikipedia is a good source of information.

What methods are available to do this?

Getting the intent of the sentence is not an easy task. To get you started on what to do, have a look on word vectors. You can also download pre-trained word2vec models. They help in getting similarity of words and reasoning with words. To get the intent of a sentence, you can use LSTM.
Fun fact most NLP algorithms strip away punctuation with is sufficient for most cases, but to give a counter example.
The defendant, who looked apologetic, was found guilty.
The defendant who looked apologetic was found guilty.

They mean different things and are difficult to catch the intent even with the best algorithms.
PS: For those wondering the difference, in the second sentence it seems like there were two defendants, and it was the one who looked apologetic who was found guilty while the other walked away free.


text:
Is this a problem well suited for machine learning?
The light field of a certain scene is the set of all light rays that travel through the volume of that scene at a specific point in time. A light field camera, for example, captures and stores a subset of the light field of a scene.
I've got an unstructured subsampling of such a scene (a few billion rays, each having a direction and light intensity information).
What I wish to do now is to create an approximation of the original scene that created this light field, with the approximation consisting of 3 arbitrarily positioned (alpha-)textured 2D planes (in 3D space), where each point on the surface radiates light towards uniformly in all directions based on the pixel color at this position.
So, I guess, this is like finding regions in the volume where similarly 'colored' rays intersect, such that the planes maximize the number of intersections they can cover.
So, the available data is the few billions of rays, the desired output is the parameters(position, normal and size) of the three planes plus one RGBA texture for each.
I'm asking here about experiences and opinions: Is this problem rather well-suited for a machine learning approach or rather not?
Edit:
A classical algorithm I could think of to solve this would be to voxelize the volume and use pathtracing to add a color sample for each ray to all cells along its way, then give each cell some value based on how similar all its contained samples are and then search for planar surfaces that intersect as many high rated cells as possible.
But maybe machine learning is better suited for such a problem?

Do you have more data than just direction and intensity? To define the trajectory inside a volume you also need some reference location that the ray is projecting to or from - e.g. its intersection point with the scene/volume.

Oh, of course! I do have the point of impact of each ray (NOT the point of origin! think as if the whole scene was an image sensor. So, I know from what direction the lightrays come from and where they impact as well as the color/intensity of the source, but not how far away this source is along the ray

also some of the lightrays originate outside of the volume for which I have impact points


text:
Does summing up word vectors destroy their meaning?
For example, I have a paragraph that I want to classify in a binary manner. But because the inputs have to have a fixed length, I need to ensure that every paragraph is represented by a uniform quantity.
One thing I've done is taken every word in the paragraph, vectorized it using GloVe word2vec, and then summed up all of the vectors to create a "paragraph" vector, which I've then fed in as an input for my model. In doing so, have I destroyed any meaning the words might have possessed?
Considering these two sentences would have the same vector:

My dog bit Dave

Dave bit my dog

How do I get around this? Am I approaching this wrong?
What other way can I train my model? If I take every word and feed that into my model, how do I know how many words I should take? How do I input these words? In the form of a 2D array, where each word vector is a column?
I want to be able to train a model that can classify text accurately.
Surprisingly, I'm getting a high (>90%) for a relatively simple model like RandomForestClassifier just by using this summing up method. Any insights?

Summing up a sequence of word vector maybe used in practice sometimes. However, the operation of addition is non-reversible, meaning that once you sum up a few numbers, you cannot get the original numbers back. However summing up a sequence of word vectors may work depending on your task. You should also normalize the values, or just use average value.
For details: https://towardsdatascience.com/document-embedding-techniques-fed3e7a6a25d#ecd3
To feed data with different lengths, you can also try padding and trimming it. Set a constant L for length of paragraph and trim/pad all list of word vectors to this length. Padding adds 0 vectors to the begining of the list and trimming trims the first part of a text until it is equal to L. Even in LSTM networks padding and trimming is still used as even though you can feed as long of a text to a LSTM as you want, you still have to process the word vectors in batches, which requires them to be the same length.
Example code in python on padding/trimming vector:
def pad_trim(list_vec, L):
    //list vec: {vec1,vec2,vec3......}
    //Assuming vecN have a size of 200
    return list_vec[-L:0] if len(list_vec) > L else [[0] * 200] * (L-len(list_vec)) + list_vec

However in inference, you can ignore maximum length if you used a RNN based method, although as the network has not been trained on lengths more than L, it may perform better or worse.
Generally speaking, you should go for concatenating if possible, so you can keep all information in the sentence. However both may work just fine depending on your task.
For RNN based and CNN based model example, you should check this out:
https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f

Comments are not for extended discussion; this conversation has been moved to chat.

But because the inputs have to have a fixed length

Do they?  Why?  The go-to strategy would be to use an RNN (possibly with LSTM or GRUs, but probably not necessary) and train it to process input sequentially and output the final classification of the paragraph.  This has the advantage of being able to take into account word order and constellations, as well as processing variable size inputs.
Intuitively, I would think simply adding word vectors will include a lot of noise from commonly occurring words that don't provide much meaningful information from the classification.  I would consider Bayesian methods or dimensionality reduction methods to limit the input to the more useful input vectors.

How can I use an LSTM to input paragraphs with variable lengths to classify them?


text:
How can I develop a genetic algorithm with a constraint on the sum of alleles?
I'm working on a genetic algorithm with a constraint on the sum of the alleles, e.g. if we use regular binary coding and a chromosome is 5-bits long I'd like to constrain it so that the sum of the bits has to be 3 or less (011100 is valid but 011110 is not). Moreover, the fitness function is such that invalid chromosomes cannot be evaluated.
Any ideas on how this problem could be approached?
I've started looking into the direction of messy GAs (since those can be over-specified) but I'm not sure if there's anything there.

There are multiple ways to handle 'illegal' individuals, each one with pros and cons:

Abortive methods: The individuals that violate constraints are eliminated as soon as discovered (i.e. after crossover or mutation) and new individuals are generated in order to keep the population stable. This usually implies a slower creation of new generations, as some individuals are discarded.

Contraceptive methods: The crossover and mutation are written in such a way to make it impossible for a newly generated individual to violate any constraints. This way to act is usually more efficient, as no individuals are discarded, but it might not be possible.

Penalization function: The fitness function gives a huge penalization to the individuals that violate constraints (usually proportionally to the constraints it violates). In this way they usually do not get to reproduce and their genes are eventually lost. You'd go this way if illegal individuals are very rare and there is no possibility for them to take over the full population (causing the algorithm to fail).

By experience, try to act on the crossover algorithm first (contraceptive way), it is the best way to exclude constraints violations. If this is not possible, pick one of the other two methods, depending on how often the illegal individuals are generated (not so often -> penalization, very ofter -> abortive)  and on how easy it is to penalize individuals that violate constraints (easy -> penalization, not easy -> abortive).

A contraceptive way to handle your example is writing the crossover as following:
const crossover = (p1: boolean[], p2: boolean[]) =>
{
    // simple crossover, getting half genome from each parent
    const child: boolean[] = [...p1.slice(0,3), ...p2.slice(3,6)];

    // contraceptive part
    while (countEnabledGenes(child) > 3) child[getRandomIndex()] = false

    return child
}

const countEnabledGenes = (genome: boolean[]) => genome.filter(gene => gene).length;
const getRandomIndex = () => Math.round(Math.random() * 6);

But this is a fairly complex problem and each scenario has its own specifications.


text:
How exactly does MICE imputation combine multiple datasets into one?
I'm trying to understand Multiple Imputation with Chained Equation (MICE) imputation process (a statistical method for imputing missing data). I have read some articles and I have understood how the imputation happens, but I didn't get the "pooling" step.
After analyzing the resulting datasets with Rubin's rules, how to pool these datasets? How to get only one dataset?
In the end, do I combine all these datasets? If yes, how? Or do I compare every dataset's estimators with Rubin's estimators and choose one dataset?

I have edited your post to clarify what your question seemed to be. You were using the term "pooling", but I'm not sure this is the right term for what you were asking. Pooling is more commonly used in the context of CNNs, so it can be a misleading term. Please, make sure that the question in the title was your main specific question. If not, feel free to edit your post again to put in the title your main specific question.

MICE is a Multiple imputation method, it contains three phases:

Imputation phase: The missing data are estimated M times from a specific model to obtain M complete and potentially different data sets.
Separate analysis phase: The selected analysis is performed separately on each of m = 1, ..., M sets of imputed data to obtain estimates (central tendency and variance).
Combined analysis phase: The results obtained  are combined according to the rules established by Rubin to obtain a single final estimate.


text:
How can I do hyperparameter optimization for a CNN-LSTM neural network?
I have built a CNN-LSTM neural network with 2 inputs and 2 outputs in Keras. I trained the network with model.fit_generator() (and not model.fit()), to load just parts of the training data when needed, because the training data is too large to load at once.
After the training the model was not working. So I checked training data (before and after augmentation). The training data are correct. So I thought the reason why the model does not work must be that I have not found the optimal hyperparameters yet.
But how can I do hyperparameter optimization on a network with multiple inputs and outputs and trained with model.fit_generator()? All I can find online is hyperparameter optimization of networks with a single input and single output and trained with model.fit().

The model not working does not mean that the hyperparameters is the problem. More often than not even with wrong hyperparameters a model should work but just not work that well. What data are you using? Maybe you can try using a more simple LSTM model first before trying a CNN LSTM model.


text:
What is the best way to smoothen out a loss curve plot?
I am currently using a loss averaged over the last 100 iterations, but this leads to artifacts like the loss going down even when the current iteration has an average loss, because the loss 100 iterations ago was a large outlier.
I thought about using different interval lengths, but I wonder if an average over the last few iterations really is the right way to plot the loss.
Are there common alternatives? Maybe using decaying weights in the average? What are the best practices for visualizing the loss?

Why do you need to average the loss? Loss is suppose to decrease over time, so getting the average of loss over time is meaningless

The variance of the loss per iteration is a lot larger than the decrease of the loss between the iterations. For example I currently have a loss between 2.6 and 3.2 in the last 100 iterations with an average of 2.92. As the scatter plot is almost useless to see the trend, I visualize the average as well.

Oh. Perhaps yous re looking for this: https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average

You can use the Exponential Moving Average method. This method is used in tensorbaord as a way to smoothen a loss curve plot.

The algorithm is as follow:

However there is a small problem doing it this way. As you can see S_t is initialized with the starting value, which makes the starting curve inaccurate.

The green curve is the ideal curve for the algorithm, but the purple curve is the predicted curve. The curve is not correct on the start. To solve this, a correction factor is added in, thus making the algorithm this:

This introduces WeightedCount which decreases over time to 0.
Exponential Moving Average is also used is other areas of deep learning, the most notable being some optimization algorithms. It is used in Adam, RMSProp and other similar optimizers to smooth out the gradients to make the path to minimal loss a more direct and straightforward path.


text:
Can we use imitation learning for on-policy algorithms?
Imitation learning uses experiences of an (expert) agent to train another agent, in my understanding. If I want to use an on-policy algorithm, for example, Proximal Policy Optimization, because of it's on-policy nature we cannot use the experiences generated by another policy directly. Importance Sampling can be used to overcome this limitation, however, it is known to be highly unstable. How can imitation learning be used for such on-policy algorithms avoiding the stability issues?


text:
What are some approaches to estimate the transition and observation probabilities in POMDP?
What are some common approaches to estimate the transition or observation probabilities, when the probabilities are not exactly known?
When realizing a POMDP model, the state model needs additional information in terms of transition and observation probabilities. Often these probabilities are not known and an equal distribution is also not given. How can we proceed?

I had asked a more general but very similar question here a long time ago.


text:
When using neural networks, should I bin the continuous variables and apply some transformation before performing variable selection and modeling?
I come from a background of scorecard development using logistic regression. Steps involved there are:

binning of continuous variables into intervals (eg age can be binned into 10-15 years, 15-20 years, etc)

weight of evidence transformation

coarse classing of bins to ensure event rate has a monotonic relationship with the variable

Variable selection is made from the coarse classed transformed variables.
I was wondering if I should follow the same steps for ANN models. That is, should I bin the continuous variables and apply some transformation before performing variable selection and modeling.

My guess is continous will give better results. Although it's probably dependent on the dataset, so the only one who can answer your question is you.


text:
How can I classify instances into two categories and then into sub-categories, when the number of features is high?
I'm working with a problem where I have a lot of variables for different cases of different users. Depending on the values of the different variables of a concrete user in a concrete case, the algorithm must classify that user in that case as:

Positive
Negative

But if the user is classified as positive, it must be classified as:

Positive normal
Positive high
Positive extra-high

If a case is positive, depending on the values of a part of the parameters, we know that the probability to be, for example, positive normal is bigger or lower.
To sum up, I see the problem as a spam detector with different spam types.
May this work if I apply an algorithm like:

Random Forest
Decision Tree

Or maybe I can include the negative case as a new group and then implement a K-means algorithm? Maybe this would help to find new groups of parameters that will say that the concrete case forms part of a group for sure.
Which one will fit best with a lot of parameters?

why don't you just use 4 classes in your random forest classifier?
-high positive
-normal positive
-extra high positive
-negative

@Lustwelpintje Is  random forest the classifier that could obtain the most success cases ?

Depending on the data and complexity.. I have had some really nice results with a simple random forest. I do believe a neural network can classify more complex functions better than a random forest, but it takes more time to get it right.

@Lustwelpintje what about Pytorch, could be a good option ?

Although I have not used it yet, From what I heard Pytorch will also be fine.


text:
What is "temporal depth"?
I need some explanation about the following paragraph (page 3) from the paper A Novel Approach for Robust Multi Human Action Detection and Recognition based on 3-Dimentional Convolutional Neural Networks.

We introduce a 3D convolution neural network with the following notations: $I(x, y, d)$ as an input video with a size of $x y$ and $d$ the temporal depth

What is "temporal depth"? Is it the number of frames?

Temporal Depth is a third parameter of a time series data. For example if you have a video clip of length 25 frames and on training a model you are giving first five frames with respect to time. Your temporal depth will be 5.

authors: Исидора Ајдер


text:
How GoogleNet actually deal with reducing overfitting?
Today I was going through a tutorial of Andrew Ng about Inception network. He said that GoogLeNet's hidden layers are also good in prediction and it had somehow a regularization effect, so it reduces overfitting. I also search on this topic and tried to figure out by reading the GoogLeNet paper. But I am not satisfied.
Can anyone give me any mathematical intuition or reasoning about this in detail?

Hi. Please, add a link to the tutorial you were reading or watching.

Course: Convolutional Neural Network, Vendor: deeplearning.ai, Website: coursera.org, Course Timestamp: 2nd week (InceptionNet)

Is this the video?

The main reason of overfitting in any neural network is having too many unrestricted trainable degrees of freedom in the model. Methods similar to dropout reduce the number of neurons at each training run which effectively means having a smaller network. On the other hand in $l_1$  and $l_2$ regularization, a term added to the loss function which put a constraint on the total loss calculated at each run. So what we are trying to minimize with such regularizations is not just $L$, but $L +l_1*f(w)$ (for example).
What I understand from that paper is that the auxiliary outputs do both at the same time: the point is to build smaller version of the same inception network within itself, and use the losses obtained from those as a constraint on the final loss function. The full model described in the paper is essentially 3 separate models: the first one is a network with 3 inception modules, the second one with 6 and the final one has 9. When loss is calculated, results from the auxiliary outputs added to the total loss with a 0.3 weight. Let us write this as follows:
$L = 0.7*L_9+ 0.3*(L_6+L_3).  $
Here $L_3$ and $L_6$ represent the losses calculated at first and second outputs, and $L_9$ is the loss calculated at the final output of the network.
This is the function we wish to minimize during training. But when the evaluation is made, the auxiliary outputs are discarded, and just the final softmax layer is used. Not very dissimilar from the idea of using dropout during training but using the full model for predictions.

It might be a good idea to use latex to format your answer.


text:
To deal with infinite loops, should I do a deeper search of the best moves with the same value, in alpha-beta pruning?
I have implemented minimax with alpha-beta pruning to play checkers. As my value heuristic, I am using only the summation of material value on the board regardless of the position.
My main issue lays in actually finishing games. A search with depth 14 draws against depth 3, since the algorithm becomes stuck in a loop of moving kings back and forth in a corner. The depth 14 player has a significant material advantage with four kings and a piece against a single king, however, it moves only one piece.
I have randomly selected a move from the list of equally valued moves and this leads to more interesting games (thus preventing the loop). However, whichever player used this random tactic ended up far worse off.
I am not quite sure how to solve this problem. Should I do a deeper search of the best moves with the same value? Or is the heuristic at fault? If so, what changes would you suggest?
So far I have tried a simple genetically generated algorithm that optimizes a linear scoring function (that accounts for the position). However as the algorithm optimized, it led to only draws and the same king loop.
Any suggestions for how to stop this king loop are very welcome!

It may be that you don't have a good condition for "tie-breakers" in your terminal states(try boundary value analysis, fiddle with your if-statement conditions), so it keeps undoing its own progress. What are the evaluation- and utility functions in your implementation?

I think this issue stems from the fact you aren't taking position into account. I would think this because as the game progresses, the number of moves that will result in a piece being taken becomes less and less, especially when there's only a few pieces left and quite a bit of "chasing" must occur before a piece is taken, likely more chasing then a depth of 14 allows.
To remedy this, you could, towards the end of the game, add to the value of a state the inverse of the total distance each friendly piece has from other pieces, that way the agent will try to move towards other pieces and minimise this distance. If you find the right scale for this heuristic, the agent will prioritise moving towards enemy pieces only when it cant find any moves that result in taking a piece, helping it break out of this loop.


text:
What is the difference between game theory and machine learning?
What is the difference between game theory and machine learning?
I had gone through the papers Deep Learning for Predicting
Human Strategic Behavior, by Jason Hartford et al., and When Machine Learning Meets AI and Game
Theory, by Anurag Agrawal et al., but I am not able to understand.

These are big areas, so here is a brief description of the differences:
Game theory is concerned with studying solutions for 'games', which are basically a set of decisions leading to certain outcomes. In game theory you look at strategies to achieve the best outcome for a given participant. One classic example (which isn't really a game in the traditional sense) is the Prisoner's Dilemma: you and your friend have been arrested, and if only one of you testifies against the other, that person gets a reduced sentence, and the other one a much longer one. If you both testify against each other, you both get a medium sentence, and if you both keep quiet, you both go free. You don't know what your partner in crime does, so do you a) testify, or b) keep quiet? If you keep quiet, you might go free if your partner also keeps quiet, but if he testifies, you are in it for a long time. So it's risky to keep quiet, even though you get the better outcome. If you testify you might avoid a longer sentence, but also will not go free. What is your best choice?
Game theory is often used in economics to model behaviour, as a rational agent would try to optimise gains.
Machine Learning, on the other hand, is a way of training a statistical classifier. You feed features into an algorithm, and the algorithm then gives you a certain output, depending on the data you have trained it with. This hasn't got anything to do with game theory per se, but I guess you could use machine learning to train an algorithm to choose moves in a game situation and then compare how that matches the optimal choices according to game theory.
As I said, this is a very brief comparison. For more details I suggest you follow the links to read up on those two fields.
UPDATE: Now that the papers are accessible — game theory is indeed used as a benchmark. In the first paper, the rational agent assumption from game theory is being modeled, but without a human expert telling the algorithm what that means. So you learn (using deep learning) what it means to be rational. In the second paper the authors attempt to learn a better algorithm than tit-for-tat, and indeed use game theory as a theoretical framework for comparison/evaluation.

The other answer gives the nice famous example of the sort of problem that game theory tackles and it partially describes what machine learning is.
However, it does not emphasize that this type of game theory problem, where you have two or more agents competing with each other, also appears in the context of machine learning. More concretely, machine learning can also be applied in the context of a multi-agent system, where you have multiple learning agents that compete with each other in an environment. Typical examples of these problems are two-player board games, like chess, go, or tic-tac-toe, which can be solved with machine learning, and, in particular, reinforcement learning (a specific type of machine learning): for instance, you can learn afterstate value functions to play tic-tac-toe.
There's a subarea of RL that tackles these problems with multiple agents, known as multi-agent reinforcement learning (MARL). One simple mathematical framework that generalises MDPs to multiple agents is the Markov games (aka stochastic games), which can be used to model games like rock-paper-scissors or tic-tac-toe. We could also model a multi-agent system as a single-agent system, where the other agents are incorporated into the environment. If you are interested in MARL, you could read, for example, the paper A comprehensive survey of multiagent reinforcement learning (2008) by Lucian Busoniu et al.
So, I think there are several connections between game theory and machine learning, and even other subareas of AI, such as game AI (e.g. the minimax algorithm is often taught in AI programs as an example of an adversarial search algorithm; read this to know more about the difference between search and learning) and evolutionary algorithms (in fact, there's also a related subfield of game theory known as evolutionary game theory).


text:
Can a trained object detection model deal with variations of the input?
Suppose an object detection algorithm is good at detecting objects and people when an object and person is close to a camera and upright. If the person walks farther away from the camera and is "upside-down" from the perspective of the camera (e.g. a fisheye camera), should the algorithm still be good at detecting people and objects in this position?

Not necessarily. Supposing your data is from the distribution of possible images containing an upright person close to the camera. Something like a neural network would perform poorly on the new data since it comes from a distribution other than on what it was trained.
You could try augmenting the dataset to try to get some synthetic "far away upside down people" but there are no guarantees here.
I'll look for the source but A. Ng cite's an experiment where a team trained a neural network on a large dataset of vehicle images. They did not realize that the cars were all facing the same direction and their model performed very poorly on images that were very similar with the primary difference being a horizontal flip.


text:
Are there ensemble methods for regression?
I have heard of ensemble methods, such as XGBoost, for binary or categorical machine learning models. However, does this exist for regression? If so, how are the weights for each model in the process of predictions determined?
I am looking to do this manually, as I was planning on training two different models using separate frameworks (YoloV3 aka Darknet and Tensorflow for bounding box regression). Is there a way I can establish a weight for each model in the overall prediction for these boxes?
Or is this a bad idea?

There are plenty of ensemble methods for regression, e.g. Bagged Trees, Random Forests, Gradient Boosting like XGBoost and AdaBoost. These are all applicable for classification and regression. However, your example towards the end of your question does not sound like that is actually what you are looking for. But rather you would like to combine two totally different models. Can you please clarify?

? This means that there are not promising versions of this algorithm fro regression until 2012.
After your question, I have found one of the survey research paper which is done or ensemple methods for regression. This table also extracted from this paper.
Read this paper, it will help you a lot more
This one is latest paper published on object detection with an ensemble approach

There's similar boosting classes in XGBoost for regression. You can implement their built-in classes for your problem, rather than implementing from scratch. You can read more about it from their website.
You can also take a look at catboost, which implements a different approach.


text:
What is the correct dimension of mu/logvar and z in the VAE?
I'm having a problem to understand the needed dimensions of a VAE, especially for mu, logvar and z layer.
Let's say I have an input of 512x512, 1 color channel (CT images), batch size 32.  Then my encoder/decoder looks like the following:
self.encoder = nn.Sequential(
            nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),  # 32x512x512
            nn.ReLU(True),
            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x256x256
            nn.ReLU(True),
            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x128x128
            nn.ReLU(True),
            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x64x64
            nn.ReLU(True),
            nn.Conv2d(32, 32, kernel_size=3, stride=2, padding=1),  # 32x32x32
            nn.ReLU(True))

self.decoder = nn.Sequential(
            nn.ConvTranspose2d(32, 32, kernel_size=4, stride=2, padding=1),
            nn.ReLU(True),
            nn.ConvTranspose2d(32, 32, kernel_size=4, stride=2, padding=1),
            nn.ReLU(True),
            nn.ConvTranspose2d(32, 32, kernel_size=4, stride=2, padding=1),
            nn.ReLU(True),
            nn.ConvTranspose2d(32, 32, kernel_size=3, stride=1, padding=1),
            nn.ReLU(True),
            nn.ConvTranspose2d(32, 1, kernel_size=4, stride=2, padding=1),
            nn.Sigmoid())

What is the correct dimension of mu/logvar and z? latent_dim = 1000, filter_depth=32.
I'm not sure if the input of the linear layer mu/logvar is right or not.
mu = nn.Linear(self.filter_depth * 32 * 32, self.latent_dim)
logvar = nn.Linear(self.filter_depth * 32 * 32, self.latent_dim)
z = nn.Linear(self.latent_dim, self.filter_depth * 32 * 32)


text:
How can I recognise possibly overlapping line segments in 2D?
I am given a 2-dimensional picture (black & white, white background) and it is assumed that there are some 'sticks' (basically 'thick lines' with different widths and lengths) that are (mostly) overlapping with one another.
I want to somehow recognize these sticks (where they are and how big they are).
Is there any approach you would recommend or, even better, anything that already exists? I am working with MatLab, but a general (theoretical) approach would be also fine! I am open to machine-learning, but I'd prefer classical algorithms here.

Have you looked at  Hough transforms? https://www.mathworks.com/help/images/ref/houghlines.html


text:
How should I handle different input sizes in graph convolution networks?
I'm a student beginning to study deep learning, and would like to practice with a simple project using a Graph Convolutional Network.
However, I'm not quite sure how to handle different input sizes of graphs for the GCN. How would I do this?
Is zero-padding the only way to solve this problem? While zero-padding is applicable to CNNs, I'm not sure if it is for a GCN.

Hello and welcome. Can you provide some details about the problem you're trying to solve? I might be able to point you to some relevant literature.

Sorry to late reply. I hope to use GCN to study protein structure and its relevant property. As you know different protein has different length of sequence, hence different size of graph.


text:
Is logistic regression used for unconstrained or constrained optimisation problems?
Is logistic regression used for unconstrained or constrained optimization problems, and why?

Hi Miko. You were asking two separate questions here, so I removed your second question about SVMs. Here you can ask about logistic regression and, in another post, you can ask about SVM.


text:
In Faster R-CNN, how can I get the predicted bounding box given the neural network's output?
The RPN loss in Faster RCNN paper is
$$
L({p_i}, {t_i}) = \frac{1}{N_{cls}} \sum_{i} L_{cls}(p_i,p_i^*) + \lambda \frac{1}{N_{reg}} \sum_i p_i^* L_{reg}(t_i, t_i^*)
$$
For regression problems, we have the following parametrization
$$t_x=\frac{x - x_a}{w_a}, \\ t_y=\frac{y−y_a}{h_a}, \\ t_w= \log \left( \frac{w}{w_a} \right),\\ t_h= \log \left(\frac{h}{h_a} \right)$$
and the ground-truth labels are
$$t_x^*=\frac{x^* - x_a}{w_a},\\ t_y^*=\frac{y^*−y_a}{h_a}, \\ t_w^*= \log \left( \frac{w^*}{w_a} \right), \\ t_h^*= \log \left(\frac{h^*}{h_a} \right)$$
where

$x$ and $y$ are the two coordinates of the center, $w$ the width, and $h$ the height of the predicted box.

$x$ and $y$ are the two coordinates of the center, $w$ the width, and $h$ the height of the anchor box.

$L_{reg}(t_i, t_i^*) = R(t_i − t_i^*)$, where $R$ is a robust loss function (smooth $L_1$)

These equations are unclear to me, so here are my two questions.

How can I get the predicted bounding box given the neural network's output?

What exactly is the smooth $L_1$ here? How is it defined?

If I understood well you have 2 questions.

How to get the bounding box given the network output
What Smooth L1 loss is

The answer to your first question lies in the equation (2) in the section 3.2.1 from the Faster R-CNN paper. As all anchor based object detector (Faster RCNN, YOLOv3, EfficientNets, FPN...) the regression output from the network are not the bounding box coordinates. The regression output predicts the shift of the predicted bounding box with respect to the selected anchor (all of these networks, use more than 1 anchor per location, check section 3.1.1 from the paper).
So basically what your network predict is $t_x, t_y, t_w, t_h$:

And the bounding box coordinates are $x, y, w, h$, and the anchor coordinates are $x_a, y_a, w_a, h_a$. So in order to compute $x, y, w, h$ from $t_x, t_y, t_w, t_h$, you just have to invert the equations above. However I think you could gain more intuition about it if you take your time and read the whole section 3.1 from the paper. I know sometimes is a pain, but you will grasp the high level concept.
With regard to your second question. Yes the loss is computed with the output from the network and the "coded" ground truth, meaning you compute the loss with the paramters $t$ (predicted) against $t^*$ (coded ground truth) instead for computing loss with the real coordinates of the bounding boxes (decoded output from the network). For the equation on Smooth L1 loss check this wonderful documentation.


text:
Is weight pruning applied to all layers or only to dense layers in CNNs?
I was reading about weight pruning in convolutional neural networks. Is it applied for all the layers including convolutional layers or only it is done for dense layers?

Hi and welcome to this community. Where were you reading about weight pruning?


text:
What are the current big challenges in natural language processing and understanding?
I'm doing a paper for a class on the topic of big problems that are still prevalent in AI, specifically in the area of natural language processing and understanding. From what I understand, the areas:

Text classification
Entity recognition
Translation
POS tagging

are for the most part solved or perform at a high level currently, but areas such as:

Text summarization
Conversational systems
Contextual systems (relying on the previous context that will impact current prediction)

are still relatively unsolved or are a big area of research (although this could very well change soon with the releases of big transformer models from what I've read).
For people who have experience in the field, what are areas that are still big challenges in NLP and NLU? Why are these areas (doesn't have to be ones I've listed) so tough to figure out?

According to a nice article by Sebastian Ruder https://ruder.io/4-biggest-open-problems-in-nlp/ based on answers from top NLP researchers https://docs.google.com/document/d/18NoNdArdzDLJFQGBMVMsQ-iLOowP1XXDaSVRmYN0IyM/edit

Natural language understanding
NLP for low-resource scenarios
Reasoning about large or multiple documents
Datasets, problems, and evaluation

I recommend having a look at the article. More details in the slides https://drive.google.com/file/d/15ehMIJ7wY9A7RSmyJPNmrBMuC7se0PMP/view

According to Dan Jurafsky, a researcher on NLP and NLU, the current hard problems in NLP are (see slide 6)

Questioning answering
Paraphrase
Summarisation
Dialogue

Other hard problems for which there are already some good solutions are

Sentiment analysis
Coreference resolution
Word sense disambiguation
Parsing
Machine translation
Information extraction

Ambiguity in natural language is one of the biggest challenges for NLP and NLU systems. Other challenges are (see slide 10)

non-standard words
idioms
tricky entity names
neologisms
world knowledge
segmentation issues


text:
How are weights updated in a genetic algorithm with neural network?
Suppose an AI is to play the game flappy bird. And the fitness function is how long the bird has traveled before the game ends.
Would we have multiple neural networks initialized at the beginning with random weights (as in each bird has its own network) and then we determine the neural networks that have lasted the longest for the game and then we perform a selection of weights from the "better" neural networks followed by mutation? Those will then be used as the new weights of a brand new neural network (ie the offspring from two "better" neural networks?)?
If that is the case, does that mean there is no backpropagation because there isn't a cost function?

I found my answer in a different post: How to evolve weights of a neural network in Neuroevolution?. Note that the genetic algorithm is a subcategory of the neuroevolution algorithm. Short answer, my original thoughts were correct.

There are other combinations possible with GAs and NNs, but I think that is the most usual way to see them combined with the GA searching for ideal weights and hyperparameters of the NN.

I don't think that genetic algorithms are a subcategory of neuroevolution.


text:
How can I train a neural network to detect subliminal messages?
Is there a way to train a neural network to detect subliminal messages? Where can I find the dataset on which to train the neural network?
If I have to create the dataset, how would I go about it?
United Nations has defined subliminal messages as perceiving messages without being aware of them, it is unconscious perception, or perception without awareness. Like you may be aware of a message but cannot consciously perceive that message in the form of text, etc.
There are two many types of subliminal messages, one which can be made through visual means, another which can be made through audio.
In visual means, I'm referring to these types:

Messages which are flashed for very short while on the screen.
Messages whose opacity is changed to blend with the background.
Messages whose colors are varied slightly to blend with the background.

Example of 3rd type of subliminal messages: if there is a background of red, on this can be shown a message made up of slight variation of red, as a conscious mind can't distinguish between such close shades of red, people will take the entire thing to be red block, but a subconscious mind notice the slight variation in color, register the message, because humans can see millions of colors.

You would find as many subliminal messages and normal (non-subliminal-message) images as you could, then train a classifier on those two sets.

Setting aside the dubious nature of subliminal messages. You have to clearly define what subliminal is. In all your cases presented, a manual algorithmic approach would be better. An ANN would tend to average out the subliminal messages as noise, just as your brain would. In the case of #2, #3, it's a matter of contrast, in which case computer vision methods of bringing out low contrast details to the forefront combined with a manual algorithm (perhaps an OCR pass for content not expected to have text) would be the way to go. ANN are not magic bullets, and still need to be used appropriately.

By "dubious nature" do you mean they don't work?

@noviceFedora, sort of. There is research showing some effect, but minor and not toward anything a person wasn't already feeling actionable towards. Subliminal messaging effects have been exaggerated just like the effects of hypnotizing. All research I've ever read shows the brain keeps it's hold on the individual, and any attempt to influence them beyond surface level fails. Thus, it's dubious nature, refers to it's exaggerated impact, and thus a likelihood of exaggerated worry about the issue.

Note that this is similar to your other worry on EE stackexchange about the Fry Effect and it's potential usage to attack someone with voices and other auditory hallucinations. The things you read online about microwave induced auditory hallucinations are greatly exaggerated. In reality, such a thing is hard to control anywhere near that level, from a fundamental physics perspective.

I request you to watch these two videos of Derren Brown: 1. Tricking Advertisers with subliminals: https://www.youtube.com/watch?v=43Mw-f6vIbo      and       2. A Toy's Story: How to control a nation: https://www.youtube.com/watch?v=EchfO2pjOrM              Both of these videos show how subliminals can be used to manipulate people, there are few other videos by other magicians and mentalists too, if you want I can find them, you might say they were staged but I don't think so. Would you please provide links to your claims that they minimal effect. From my experience I think they do work.

I don't know if you have seen my thread on Dream.In.Code.net or not, in one of my posts there I have explained my reasons for believing subliminals work. I think subliminals and hypnotism have to be tailored to specific individuals using details of the subject. I don't know what your background is, but if you don't have a background in human perception, neurology, psychology, it is unfounded to hold the belief that subliminals and hypnotism don't work. Maybe the people who use these technologies spread misinformation to keep people defenseless.

Unless you don't trust the establishment, I'd suggest reading into any and all whitepapers, peer reviewed research, on the subject. Yes, hypnotist and mentalist, subliminal messaging...etc, can indeed be staged, and the usefulness can be exaggerated. Don't rely on youtube videos, shows, or other things, with no evidence of controls, process, and proof to be your guide.

" but if you don't have a background in human perception, neurology, psychology, " That is an attempt at argument by authority. A claim should be able to stand on it's own, you can't brush off someone's criticism by only stating they don't have the required credentials. I have a background in electrical engineering, signals processing, and have taken a fair amount of courses on neurology, psychology, and human perception (I'm fairly big into transhumanism, so it's in my area of interest)... yet, my credentials, nor your credentials matter here. You're making the claim, the onus is on you.

Would you please provide me links to whitepapers, peer reviewed research which formed your opinion about hypnotism and subliminals having mariginal effect. If subliminals utilized personal details of a person, like their loved one's name, significant events in ones life, they might work.

The best would be if you have access to a University research portal, as most will exist behind paywalls. However, Google Scholar would be a good start for your research efforts. Just be careful to make sure the paper you read is peer reviewed, I am not sure if Google indexes pre-print papers and such.

Typing Google Scholar subliminals, showed me few links and all of them seem to confirm they work. It has to use a combination of technologies to make it work, like using binaural beats, brain synchronizing, etc.

Please post a link to a couple of them. I have a feeling you're just seeing an admission of existence and ignoring the efficacy. Keep in mind, I said exaggerated. Subliminal messaging is a thing, it's just not particularly effective.

I think this has to be reciprocal, I have given you links to support my claim, you haven't given anything. It's true I had a cursory look at the results of the Google Scholars, would you please provide me your links, then I'll share mine. As Christopher Hitchens said: "What is asserted without evidence, can be dismissed without evidence." and "Extraordinary claims require extraordinary evidence". I have satisfied my burden for proof from my links. Would you please satisfy your burden of proof with your links.

Let us continue this discussion in chat.

Maybe tomorrow same time.


text:
Is it useful to train a CycleGAN in a supervised setting?
I am very interested in the application of CycleGANs. I understand the concept of unpaired data and it makes sense to me.
But now a question comes to my mind: what if I have enough paired image data, is then a CycleGAN an over-engineering, if I use it in a "supervised" setting (input matches with the label - but still a CycleGAN)? For what kind of application could it be useful? Would it be more useful to process it using a "normal" supervised setting?
So, basically, my question is whether it is useful to train a CycleGAN in a supervised setting?

Right now im trying cyclegan dcgan implementation on this (paired pictures from danbooru dressed to nude. cycle_gan model). I ll report text result in comparison to pix2pix here asap. But maybe 5-7 days will require.


text:
How to use BERT as a multi-purpose conversational AI?
I'm looking to make an NLP model that can achieve a dual purpose. One purpose is that it can hold interesting conversations (conversational AI), and another being that it can do intent classification and even accomplish the classified task.
To accomplish this, would I need to use multimodal machine learning, where you combine the signal from two models into one? Or can it be done with a single model?
In my internet searches, I found BERT, developed by Google engineers (although apparently not a Google product), which is an NLP model trained in an unsupervised fashion on 3.3 billion words or more and seems very capable.
How can I leverage BERT to make my own conversational AI that can also carry out tasks? Is it as simple as copying the weights from BERT to your own model?
Any guidance is appreciated.

Think of BERT (or similar models) as as good starting place for understanding context.
A couple options to make BERT contextualize dialogue:

Concatenate all messages with a seperator embedding and finetune a language model like BERT

This has shown good results in this paper, but understand it has weaknesses like struggling to determine order or author comprehension or conversation disentanglement. Additionally as threads grow longer this becomes more and more expensive

Use A BERT like model as an RNN and pass along author/memory information as the hidden state

Weaknesses here is that you are putting a lot of strain on the hidden state of the model, and ideally you want to train each comment block separately so the gradient wouldnt need to unroll or traverse the network in backprop (I actually implemented a variant of this for my work, and had good results)

The 2 modes above are just baseline examples that you can alter for your own needs, and im sure there is also others if you brainstorm about it.
Note if you want generation capabilities, you would want to use GPT2 or XLNet instead of BERT for unidirectional embeddings. Hope this helped.


text:
How can I avoid displaying the velocity in the updated tracklets?
In target tracking, the dimensions of objects change, especially if they are detected using a LIDAR sensor. Also, the static objects in consecutive frames are not 100% static, their position changes a little bit, due to the point cloud segmentation algorithms (which is somehow expected).
After I associate a tracklet (which maintains the objects previous dimension) and a measurement (that has changed its dimension in the current frame) and perform a Kalman update, a small velocity is induced in the new updated tracklet, even if my object is static (I am considering the reference point of the object and tracking its center).
Is there any solution for not inducing and displaying such a velocity in the updated tracklets?


text:
How to train image segmentation task with only one class?
Is there a neural network that has architecture optimizations for segmenting only one class (object and background)? I have tried U-net but it is not providing good enough results.
I am wondering if this can be due to the fact that my dataset has different image resolutions/aspect ratios.

When you say "only one class (object and background)", in fact you are saying two classes: the object class and the background class.

To allow us help you, you should describe a few more the context: kind of images, ... . Also the expected result.

@pasabaporaqui Thanks for reply. I'm trying to train a car segmentation. I have custom dataset I've collected. I expect it to have precise boundary around the car on an image.


text:
Why does the ELBO come to a steady state and the latent space shrinks?
I'm trying to train a VAE using a graph dataset. However, my latent space shrinks epoch by epoch. Meanwhile, my ELBO plot comes to a steady state after a few epochs.
I tried to play around with parameters and I realized, by increasing the batch size or training data, this happens faster, and ELBO comes to a steady state even faster.
Is this a common problem, with a general solution?
With these signs, which part of the algorithm is more possible to cause the issue? Is it an issue from computing loss function? Does it look like the decoder is not trained well? Or it is more likely for the encoder not to have detected features that are informative enough?

Edit:
I figured out that the problem is probably caused by the loss function. My loss function is a combination of the KL term and reconstruction loss. In the github page for graph auto-encoders, it is suggested that the loss function should include normalization factors according to the number of nodes in the graph. I haven't figured it out exactly, but by adding a factor of 100 to my reconstruction loss and a factor of 0.5 to my KL loss, the algorithm is working fine. I would appreciate it if someone can expand on how this exactly is supposed to be set up.

This is an old question, but can you explain what you mean by "my latent space shrinks"? Do you mean that the Gaussian from which you sample the latent vectors shrinks? Also, what do you mean by "the ELBO comes to a steady state"? Do you mean that the ELBO doesn't further decrease and remains more or less constant after some epoch? It would have been nice if you include the plots that show these behaviours. If you still have them, I would appreciate if you edit your plot to include them and clarify these points/questions.


text:
Should I consider mean or sampled value for action selection in ppo algorithm?
When considering the policy network in PPO algorithm, we need to fit a Gaussian distribution to the neural network output (for a continuous action space problem). When I use this network to obtain action, should I sample from the fitted distribution or directly consider the mean output from the policy network? Also, what should be the standard deviation of the fitted distribution?


text:
How does a transformer leverage the GPU to be trained faster than RNNs?
How does a transformer leverage the GPU to be trained faster than RNNs?
I understand the parameter space of the transformer might be significantly larger than that of the RNN. But why does the transformer structure can leverage multiple GPUs, and why does that accelerate its training?

A recurrent network depends on the previous time steps hidden state. Therefore you can’t calculate the $t$th hidden state without first calculating the $t-1$th. This is not the case with transformers, there is no such recursive dependency — the recursive nature is capture in other ways (eg through self attention and explicitly positional encoding)

@DavidIreland That seems like a good starting point for a formal answer.

A recurrent neural network (RNN) depends on the previous hidden state from the previous time step. That is, an RNN is a function of both the data for the sequence at time $t$ and the hidden state from time $t-1$. This means that we cannot compute the $t$th hidden state without calculating the $t-1$th state, and the $t-1$th state without the $t-2$th state, and so on.
In contrast to this, a transformer is able to fully parallelise the processing of the sequence because it does not have this recursive relationship, i.e. a transformer is not a recursive function -- the recursive nature of the sequence is processed in other ways, such as through positional encoding. We can see this by the way self attention works.
If we first consider the general attention mechanism framework, then we have a query $q$ and a set of paired key-value tuples $\textbf{k}_1, ..., \textbf{k}_n$ and $\textbf{v}_1, ..., \textbf{v}_n$. In general, for each key, we will apply some attention function $\beta$ (such as a neural network) to obtain attention scores, $a_i = \beta(\textbf{q}, \textbf{k}_i)$. We then define an attention vector $\textbf{a}$ where the $i$th element is the $i$th attention score, and we take a softmax of this vector to obtain attention weights $\alpha_i$ where $\alpha_i$ is the $i$th element of $\mbox{softmax}(\textbf{a})$. The output of the attention mechanism for query $\textbf{q}$ is then the weighted sum $\sum_{i=1}^n \alpha_i \textbf{v}_i$.
Now that we have the necessary background for an attention mechanism, we can look at self attention which is the backbone of Transformer. If we have a sequence denoted by $\{\textbf{x}_1, ..., \textbf{x}_n\}$, then we can define a set of queries, keys and values to be these $\textbf{x}_i$ values. Note that previously we only had a single query, but here we will have multiple queries which is really how Transformer is able to parallelise the processing of the sequence. If we define $\textbf{Q}, \textbf{K}, \textbf{V}$ to be the matrices of the queries, keys and values (e.g. the $i$th row of $\textbf{Q}$ corresponds to the $i$th query, and similarly for the others). Self attention is as simple as performing attention over these query, key and values -- the name self comes from the fact that the queries, keys and values are all the same and represent the $i$th element of the sequence. Now, we can write the above attention mechanism as $a_{i, j} = \beta(Q, K)$ where we now have a matrix of attention scores (because we have $i$ queries and $i$ keys the matrix will be square), and we can take softmax row-wise to get the attention weights (again, this will be an $i\times i$ matrix). If we call the matrix of attention weights $\textbf{A}$ then the output of a self attention layer will be given by $\textbf{A} \textbf{V}$. As you can see, there is no recursive nature here and this is all parallelisable, e.g. it can be broken up and put onto multiple GPU's at the same time -- this would not be possible with an RNN as you would have to wait for the output of the previous layer.

please someone let me know if I have messed up the orders of my matrix multiplications.

@abcd kind of, yes. The RNN has access to information from previous time steps through the hidden state, but if you look at how this is updated (in e.g. LSTMs which use a gating mechanism) then it starts to lose more information about time $t$ at time $t+k$ as $k$ increases. This isn't wholly true in transformers, as they attend to every previous token in the context window.

I'm less sure on them but I think the same problem would happen. For really long sequences, the hidden state would start to lose information (I think).

@abcd kind of hard to answer in a comment, probably it would require a writing out of the equations used in attention (perhaps you could ask it in a question and either myself or someone else will answer it), but basically if you look at how self-attention works, you can see that they attend to every token in the sequence without any loss of information (e.g. there is no attempt to collect all prior information in a single dynamic state)

The issue with Recurrent models is that they don't parallelization during training.
Sequential models performs better with more memory but faces problem in learning long-term memory dependencies.
On the other hand Transformers take into account of self attention which boosts the speed of how fast the model can translate from one sequence to another and establishes dependencies b/w input and output and focus on relevant parts of the input sequence, which in turn eliminates recurrence and convolution unlike RNNs where sequential computation inhibits parallelization.

This answer does not provide much insight into how transformers are "more parallelizable" than RNNs. You say "unlike RNNs where sequential computation inhibits parallelization", but that's not very useful because you don't explain why transformers avoid the issue and why "sequential computation" is really the problem. You talk about many things that are irrelevant to answer the question and only give a few words to answer the question, which are unclear.


text:
Which neural network should I use to transform the pixels of a video overtime?
I want to train a network with video data and have it transform pixel values overtime on an input video. This is for an art project and does not need to be super elaborate, but the videos I want to render out of this might be big in resolution and frame count.
Which neural network would be appropriate for this task? I think I'm looking for a convolutional network (but I'm not too sure of that either). Which framework could easily allow me to do this?
Now, I'm no proper programmer, but self-learned on the go. I know some Javascript, but rather would like to learn more Python. Ideally, the easier and simpler the better though: I would be perfectly happy with something like "Uber Ludwig" (except maybe that it's from Uber).

Hi and welcome to this community. Could you please specify how you want to transform the pixels of the video? What's your ultimate goal?

regarding this project: i want to train a network with a certain type of footage so it develops some kind of abstract idea of what visually happens there over time. in a second step, i then want to apply this 'abstract idea' to other imagery (so: generate new images!), either stills or video.

that said, i spend quite some time today looking at ludwig - which seems super cool, only it does not seem to be able to output images: https://uber.github.io/ludwig/user_guide/#image-output-features-and-decoders


text:
Why do current models use multiple normalization layers?
In most current models, the normalization layer is applied after each convolution layer. Many models use the block $\text{convolution} \rightarrow \text{batch normalization} \rightarrow \text{ReLU}$ repeatedly. But why do we need multiple batch normalization layers? If we have a convolution layer that receives a normalized input, shouldn't it spit out a normalized output? Isn't it enough to place normalization layers only at the beginning of the model?

One issue is that a normalized set of initial weights may not stay normalized as learning progresses; so given that we adjust weights proportionately according to their relative values and also when working on a subset of the learning data the model may become convinced that one subset of features is important and others not, this can result in the weights becoming unbalanced again. In the learning curve we may see this as a plateau, where the learning becomes convinced that a few features are much more important than they really are and fails to find new features that can contribute even more because tiny proportionate changes did not move them far, or quickly, enough into a noticeable range.
So when we re-sample from the database to get the next batch we need to be fully open to the possibility that the previous batch learning was too heavily biased towards its own favoured set. In effect we are trading accuracy for openness to new feature combinations, and at the same time assisting generalization.


text:
How can I recognise the name of a molecule given an image of its structure?
I want to recognize the name of the chemical structure from the image of the chemical structure. For example, in the image below, it is a benzene structure, and I want to recognize that it is benzene from the image (I should be able to recognize all these structures as benzene).
How can I recognize the name of a molecule given an image of its structure?

like in a research paper we have bunch of text & figures (here chemical structures) I want to recognize those chemical structures

This could end up being much harder or much easier than classifying images of molecules when there are almost an unlimited number of categories, but my first thought was to write a piece of software that will take an image of a molecule and produce a drawing of it that could be recognized by a program that is designed to recognize drawings. (I think ChemDraw does this.)


text:
Using U-NET for image semantic segmentation
I'm getting literally crazy trying to understand how U-NET works. Maybe it is very easy, but I'm stuck (and I have a terrible headache). So, I need your help.
I'm going to segment MRI to find white matter hyperintensities. I have a dataset with MRI brain images, and another dataset with the WMH. For each one of the brain images, I have one black image with white dots on it in the WMH dataset. These white dots represent where is a WMH on its corresponding brain image.
This is an image from the MRI brain images:

And this is the corresponding WMH image from the WMH dataset:

How can I use the other images in network validation?
I suppose there will be a loss function and this network is supervised learning.

first hit when googling "unet example"  https://github.com/zhixuhao/unet  will give you an idea how to use unet.  you will need to be very familiar with your data, how to deal with the images and masks just to get it right. in my experience, unets work pretty well out of the box.  you may run into questions like "how big of an image to feed in", "memory problems", etc. you'll need to take those one at a time.

Have you solved this problem? It's not clear exactly what you're asking. Are you asking how to augment your training data?

@nbro I think I was asking about validation data parameter used in model.fit method. Six months later, I have understood what I have to do (I think). Maybe the question is about how to use validation data and how to use the method predict to validate training.