text:
How is neural architecture search performed?
I have come across something that IBM offers called neural architecture search. You feed it a data set and it outputs an initial neural architecture that you can train.
How is neural architecture search (NAS) performed? Do they use heuristics, or is this meta machine learning?
If you have any papers on NAS, I would appreciate if you can provide a link to them.

You could say that NAS fits into the domain of Meta Learning or Meta Machine learning.
I've pulled the NAS papers from my notes, this is a collection of papers/lectures that I personally found very interesting. It's sorted in rough chronological descending order, and *** means influential / must read.
Quoc V. Le and Barret Zoph are to good authors on the topic.

The Evolved Transformer
Exploring Randomly Wired Neural Networks for Image Recognition
GRAPH HYPERNETWORKS FOR NEURAL ARCHITECTURE SEARCH
Backprop Evolution
Progressive Neural Architecture Search
*** DARTS: Differentiable Architecture Search
*** Efficient Neural Architecture Search via Parameter Sharing - ENAS
*** Progressive Neural Architecture Search
AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
Automatic Machine Learning - Prof. Frank Hutter
Google Brain - Neural Architecture Search - Quoc Le
*** Regularized Evolution for Image Classifier Architecture Search
Autostacker: A Compositional Evolutionary Learning System
Generating Neural Networks with Neural Networks
Finding Competitive Network Architectures Within a Day Using UCT
Neuroevolution: A different kind of deep learning
Evolving Deep Neural Networks
Pieter Abbeel: Deep Learning-to-Learn Robotic Control
*** SMASH: One-Shot Model Architecture Search through HyperNetworks

Here are two review articles:

Elsken, Metzen, Hutter: Neural Architecture Search: A Survey (2019), Journal of Machine Learning Research 20, 1-21
He, Zhao, Chu: AutoML: A Survey of the State-of-the-Art (2019)

The question is not about AutoML, but only NAS, so I suggest you remove your second suggestion.

@nbro 1. about half of the paper is about NAS. 2. if there were tons of reviews one could get finicky, but losing one out of two?

I don't get what you mean by "but losing one out of two" in this context.

Neural architecture search (NAS) is a method of automating the design (that is, the choice of the values of the hyper-parameters) of artificial neural networks. There are different approaches to search the space of neural network architectures. For example, you can use reinforcement learning or evolutionary (or genetic) algorithms.
Check out the paper Neural Architecture Search with Reinforcement Learning (2017), by Barret Zoph and Quoc V. Le, where the authors train, using reinforcement learning (specifically, REINFORCE), a recurrent neural network (the "controller") to generate (convolutional and recurrent) neural network architectures, so that to maximise the expected accuracy of the generated architectures on a validation dataset. They achieve some good results using this approach.
See also Efficient Neural Architecture Search via Parameter Sharing (2018), by Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le and Jeff Dean (which thus includes some of the authors of NAS), which is similar to NAS, but more efficient, hence the acronym ENAS (efficient NAS).


text:
When using hashing in tile coding, why are memory requirements reduced and there is only a little loss of performance?
In the book "Reinforcement Learning: An Introduction" (2018) Sutton and Barto explain, on page 221, a form of tile coding using hashing, to reduce memory consumption.
I have two questions about that:

How can this approach reduce memory consumption? Doesn't it just depend on the number of tiles (you have to store one weight for each tile)?

They state that there is only a "little loss of performance". In my understanding, the sense of tile coding (and coarse coding) is, that near-by states have many tiles in common and far-away states have only few tilings in common. With tilings "randomly spread throughout the state space" this isn't the case. How does this not influence performance?

you can look at the implementation here

I already saw the implementation and understand the concept, but not why it works!

@ F.M.F.  did you find an answer to this posted question?

@user3489173 In my understanding, the memory consumption is reduced, by assigning weights to tiles lazily and therefore only storing weights for parts of the state-action space that actually are reached.

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
How define a reward function for a humanoid agent whose goal is to stand up from the ground?
I'm trying to teach a humanoid agent how to stand up after falling. The episode starts with the agent lying on the floor with its back touching the ground, and its goal is to stand up in the shortest amount of time.
But I'm having trouble in regards to reward shaping. I've tried multiple different reward functions, but they all end up the same way: the agent quickly learns to sit (i.e. lifting its torso), but then gets stuck on this local optimum forever.
Any ideas or advice on how to best design a good reward function for this scenario?
A few reward functions I've tried so far:

current_height / goal_height
current_height / goal_height - 1
current_height / goal_height - reward_prev_timestep
(current_height / goal_height)^N (tried multiple different values of N)
...

In section 6.2.2 of this paper they provide some reward functions so you can try with these.

See https://gymnasium.farama.org/environments/mujoco/humanoid_standup/#rewards for a description of how to get a humanoid ragdoll to stand up!

Note that I have got this far so far myself...  https://www.youtube.com/watch?v=ISZ_xgybazQ  (it reliably rolls over onto it's front)

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
How do I plot a matrix of ratings?
I have a .csv file called ratings.csv with the following structure:
userID, movieID, rating
3,      12,      5
2,      7,       6

The rating scale goes from 0 to 5 stars. I want to be able to plot the sparsity of the matrix like it's done in the following picture:

As you can see, ratings scale goes from 0 to 5 on the right. It is a very well thought plot.
I have Matlab, Python, R etc. Could you come up with something and help me? I’ve tried hard but I cannot find the way to do it.

You're looking for a heatmap. Check out e.g. https://stackoverflow.com/q/33282368/3924118 (if you like Python more than the others). See also this documentation.

I did it!
A = importdata('u.data');
user_id = A(:, 1);
movie_id = A(:, 2);
rating = A(:, 3);

% Build matrix R and w (weights matrix)
R = zeros(943, 1682);
w = zeros(943, 1682);
for i=1:100000
    R(user_id(i), movie_id(i)) = rating(i);
    w(user_id(i), movie_id(i)) = 1;
end

m = HeatMap(R)
ax = hm.plot; % 'ax' will be a handle to a standard MATLAB axes.
colorbar('Peer', ax); % Turn the colorbar on
caxis(ax, [0 5]); % Adjust the color limits

Output:


text:
Can $\Phi$ measure of Integrated Information Theory serve as reward for reinforcement learning system?
Can $\Phi$ measure (computed rigorously or approximately) of Integrated Information Theory serve as reward for self-evolving/learning reinforcement learning system and hence we let this system to become/evolve into conscious system?

This is an old question, but can you please edit your post to explain what you mean by "$\Phi$ measure of IIT". Provide more details about this "$\Phi$ measure" and why you think it could server as the "reward signal" of an RL agent, as you seem to be suggesting.


text:
What is the proof that the branch and bound algorithm always finds optimal path in a graph?
I've been studying Branch and Bound's graph algorithm and I hear it always finds the optimal path because it uses previously found solutions to find others
However, I haven't been able to find a proof of why it finds the optimal path. (In fact, most sites kind of do a bad job generalizing the algorithm itself.)
What is the proof that this algorithm always finds the optimal path in the case of a graph with 1 or more goal nodes?

Comments are not for extended discussion; this conversation has been moved to chat.

Branch and Bound is similar to an exhaustive search, except it incorporates a method for computing lower bounds on branches. If the lower bound on a given branch is greater than the upper bound on the problem (i.e. the current best solution encountered), that branch can be discarded since it will never produce an optimal solution.
Hence, since you explore all options except those you know will produce values less optimal than your current best value, you are guaranteed to encounter the global optimum.
Note this is a generic algorithm, and you will need to reference a specific implementation if you want proof of why it satisfies these criteria.

My attempt was proof by contradiction. We can assume B&B found a sub-optimal path, but that would create a contradiction, because the only way B&B would miss an optimal path is if it skipped it completely (this part I don't know how to prove) or the related part of the search space was pruned.


text:
What is the advantage of using a VAE over a deterministic auto-encoder?
What is the advantage of using a VAE over a deterministic auto-encoder?
For example, assuming we have just 2 labels, a deterministic auto-encoder will always map a given image to the same latent vector. However, one expects that after the training, the 2 classes will form separate clusters in the encoder space.
In the case of the VAE, an image is mapped to an encoding vector probabilistically. However, one still ends up with 2 separate clusters. Now, if one passes a new image (at the test time), in both cases the network should be able to place that new image in one of the 2 clusters.
How are these 2 clusters created using the VAE better than the ones from the deterministic case?

"However, one expects that after the training, the 2 classes will form separate clusters in the encoder space.", what do you mean by "the 2 classes"?

It seems that you think that we want to perform classification with VAEs or that images that we pass to the encoder fall into more than one category. The other answer already points out that VAEs are not typically used for classification but for generation tasks, so let me try to answer the main question.
The variational auto-encoder (VAE) and the (deterministic) auto-encoder both have an encoder and a decoder and they both convert the inputs to a latent representation, but their inner workings are different: a VAE is a generative statistical model, while the AE can be viewed just as a data compressor (and decompressor).
In an AE, given an input $\mathbf{x}$ (e.g. an image), the encoder produces one latent vector $\mathbf{z_x}$, which can be decoded into $\mathbf{\hat{x}}$ (another image which should be similar or related to $\mathbf{x}$). Compactly, this can be presented as $\mathbf{\hat{x}}=f(\mathbf{z_x}=g(\mathbf{x}))$, where $g$ is the encoder and $f$ is the decoder. This operation is deterministic: so, given the same $\mathbf{x}$, the same $\mathbf{z_x}$ and $\mathbf{\hat{x}}$ are produced.
In a VAE, given an input $\mathbf{x} \in X$ (e.g. an image), more than one latent vector, $\mathbf{z_{x}}^i \in Z$, can be produced, because the encoder attempts to learn the probability distribution $q_\phi(z \mid x)$, which can be e.g. $\mathcal{N}(\mu, \sigma)$, which we can sample from, where $\mu, \sigma = g_\theta(\mathbf{x})$. In practice, $g_\theta$ is a neural network with weights $\phi$. We can sample latent vectors $\mathbf{z_{x}}^i$ from $\mathcal{N}(\mu, \sigma)$, which should be "good" representations of a given $\mathbf{x}$.
Why is it useful to learn $q_\phi(z \mid x)$? There are many uses cases. For example, given multiple corrupted/noisy versions of an image, you can reconstruct the original uncorrupted image. However, note that you can use the AE also for denoising. Here you have a TensorFlow example that illustrates this. The difference is is that, again, given the same noisy image, the model will always produce the same reconstructed image. You can also use the VAE for drug design [1]. See also this post.

thanks, i thought those are same. Are they also implemented differently?

@user8426627 They have a similar implementation because both have an encoder and a decoder, but the encoder, in the case of the VAE, produces a mean and variance, rather than just a latent vector.

that sounds like input is interpreted differently (input preparation is not part of nn) but the network itself has same architecture

@user8426627 It is not the input, it is more the output. The mean and the variance will be the output of some NN. However, both in the case of AE and VAE, at least conceptually, there is an encoder and a decoder. In this way, they are similar.


text:
A* is similar to Dijkstra with reduced cost
According to this Wikipedia article

If the heuristic $h$ satisfies the additional condition $h(x) \leq d(x, y) + h(y)$ for every edge $(x, y)$ of the graph (where $d$ denotes the length of that edge), then $h$ is called monotone, or consistent. In such a case, $A^*$ can be implemented more efficiently — roughly speaking, no node needs to be processed more than once (see closed set below) — and $A^*$ is equivalent to running Dijkstra's algorithm with the reduced cost $d'(x, y) = d(x, y) + h(y) − h(x).$

Can someone intuitively explain why the reduced cost is of this form ?

Djikstra is actually a special case of A*.
Think of Djikstra intuitively as A* with the heuristic cost being always zero.

What you are doing when calculating $d'(x,y)$:

$d(x,y)$: calculating the original edge distance from $x$ to $y$
$h(y)$: plus the heuristic from $y$ to the goal
$h(x)$: minus the heuristic from $x$ to the goal

So, using this recalculation of the original edge-values ($1.$) in Dijkstra's algorithm you are inherently accounting for the heuristic component of A* by incorporating it ($2.$) into the value of the edge traversed, and discarding ($3.$) the accumulated previous heuristic values of previous nodes in the path.
The additional condition $h(x) ≤ d(x, y) + h(y)$ ensures the new edge-values are strictly positive.


text:
How are the attention weights normalised in the transformer?
In the Transformer (adopted in BERT), we normalize the attention weights (dot product of keys and queries) using a softmax in the Scaled Dot-Product mechanism. It is unclear to me whether this normalization is performed on each row of the weight matrix or on the entire matrix. In the TensorFlow tutorial, it is performed on each row (axis=-1), and in the official TensorFlow code, it is performed on the entire matrix (axis=None). The paper doesn't give many details.
To me, both methods can make sense, but they have a strong impact. If on each row, then each value will have a roughly similar norm, because the sum of its weights is 1. If on the entire matrix, some values might be "extinguished" because all of its weights can be very close to zero.

I might have been confused by the meaning of "axis=None" and "axis-1" which both actually select the last dimension. So the softmax is performed on the rows in both implementations.


text:
How do big companies, like Facebook, model individuals and their interaction?
As a layman in AI, I want to get an idea of how big data players, like Facebook, model individuals (of which they have so many data).
There are two scenarios I can imagine:

Neural networks build clusters of individuals by pure and "unconscious" big data analysis (not knowing, trying to understand and naming the clusters and "feature neurons" on intermediate levels of the network) with the only aim to predict some decisions of the members of these clusters with highest possible accuracy.

Letting humans analyze the clusters and neurons (trying to understand what they mean) they give names to them and possibly add human-defined "fields" (like "is an honest person") if these were not found automatically, and whose values are then calculated from big data.

The second case would result in a specific psychological model of individuals with lots of "human-understandable" dimensions.
In case there is such a model, I would be very interested to know as much about it as possible.
What can be said about this:

Is there most probably such a model (that is kept as a secret e.g. by Facebook)?

Has somehow tried to guess how it may look like?

Are there leaked parts of the model?

My aim is to know and understand by which categories Facebook (as an example) classifies its users.

I don't have a source as this is just a guess, but I would imagine they may model users using a graph. You could look into Graph Neural Networks, which are really interesting and a lot of the data sets they use in papers are social networks. Also recommender systems can be modelled as a special kind of graph where you have two different types of nodes -- a type for users and a type for items.


text:
How should I design the LSTM architecture for multivariate time series forecasting problems?
There is plenty of literature describing LSTMs in a lot of detail and how to use them for multi-variate or uni-variate forecasting problems. What I couldn't find though, is any papers or discussions describing time series forecasting where we have correlated forecast data.
An example is best to describe what I mean. Say I wanted to predict number of people at a beach for the next 24 hours and I want to predict this in hourly granularity. This quantity of people would clearly depend on the past quantity of people at the beach as well as the weather. Now I can make an LSTM architecture of some sort to predict these future quantities based upon what happened in the past quite easily. But what if I now have access to weather forecasts for the next 24 hours too? (and historical forecast data too).
The architecture I came up with looks like this:

So I train the left upper branch on forecast data, then train the right upper branch on out-turn data, then freeze their layers to and join them to form the final network in the picture and train that on both forecasts and out-turns. (when I say train, the output is always the forecast for the next 24 hours). This method does in fact have better performance than using forecasts or out-turns alone.
I guess my question is, has anyone seen any literature around on this topic and/ or knows a better way to solve these sort of multivariate time series forecasting problems and is my method okay or completely flawed?

I have not come across a model like this yet. BUT
If you have not tried smaller models I'd recommend trying that first.
Justification: this lets you use learning curves to diagnose what to do next.
Also, you might try starting with a GRU (the LSTM overhead may not be needed).
One Idea for a starting model
Observe that turnout does not affect weather. So the weather forecast model will be independent of the turnout model but not vice versa.
(Note: I'm using RNN to denote which ever recurrent architecture you choose)
Formulation for simultaneous prediction:
Weather
$\text{RNN}_w$ is your weather prediction model
$\hat w_t = \text{RNN}_w(\hat w_{t-1})$ be the predicted weather at time $t$
Turnout: The idea here is that people will or will not go to the beach for various reasons (overcrowded, too hot or stormy etc). So in the beach population prediction task we use all these as features to predict the population at the next time-step. This reduces the problem to any one of the classical models already developed.
$\text{RNN}_p$ is your turnout (beach population) prediction model
$\hat p_t$ be the predicted turnout at time $t$
$\hat c_t=[\hat p_t, \hat w_t]$ is the concatenation at time t
$\hat p_{t+1} = \text{RNN}_p(\hat c_t)$
Formulation in the presence of weather forecast:
Simply replace $\hat w_t$ with the true weather forecasts $\hat f_t$.
A final warning
Unless you think your RNN is better at weather prediction than the forecasting systems I would not recommend using $\text{RNN}_w$ in a production application.
I hope this helps.


text:
Is it possible for a neural network to be used to compress data?
When training a neural network, we often run into the issue of overfitting.
However, is it possible to put overfitting to use? Basically, my idea is, instead of storing a large dataset in a database, you can just train a neural network on the entire dataset until it overfits as much as possible, then retrieve data "stored" in the neural network like it's a hashing function.

The auto-encoder (AE) can be used to learn a compressed representation (a vectorised hash value) of each observation in the training dataset, $z$, which can then be used to later retrieve the original (or similar) observation.
The variational auto-encoder (VAE), a statistical variation of AE, can also be used to generate objects similar to the observations (or inputs) in the training set.
There are other data compressor models, for example, Helmholtz machine, which precedes both the AE and VAE.

Train a network that has large input and small output. Turn it upside down (yes, you may do that). By giving the small outputs, corresponging to input, the ideally- trained network will generate those large data.
But you see in all compression there will be data lost, so generated data will be slightly :DD different then original dataset. So its suitable for statistical data, like images, whatever, but not for structured like text or the most unsuitable example - program source code.

An Auto-Encoder is probably what you are looking for. AE is a very powerful Neural Network when you want to compress data and get a lower dimensional representation of the data with maximum information retained.
If you are interested to know how it works -- Think of it as training a NN to predict the data itself. Which means your input and output layers would exactly be the same. So how does this help in compression ? The Hidden Layer -- Here is where the magic happens. The compression comes from the fact that we use lesser number of neurons in the hidden layer than in the i/p layer. Assuming that the NN we build nicely predicts the i/p at the end of the training,  the output of the hidden layers can be thought of as newly engineered features which are less in dimension yet powerful enough to represent the information in original data.

See What is the difference between encoders and auto-encoders? For a working example of a neural network being used to compress (and decompress) images.


text:
How to define cost function for custom nonlinear functions?
For logistic regression, the Cost function is defined as:
\begin{equation}
Cost(h_{\theta}(x)-y) = -ylog(h_{\theta}(x))-(1-y)log(1-h_{\theta}(x))
\end{equation}
I now have a nonlinear function
\begin{equation}
 h_{\theta}^{(i)}(x)=xe^{-j\theta_i|x|^2}
\end{equation}
where $i$ denotes the $i$th training sample. How should I define cost function for this particular nonlinear function?

Hi Yifan. I might be able to answer this question later. Meanwhile, if you need immediate help, you might want to ask the question also on https://stats.stackexchange.com/.

Hi nbro, thank you very much for the suggestion~~ However I'll wait for your response~~ I'll ask there next time : )

Hi again~~ I would really like to ask if you could give me a hint on this problem~ Thank you so much

Hi Yifan! I am sorry I only answer now. It had been a while since I had worked with logistic regression, so I needed some time to review it. Have a look at generalised linear models (link function and prediction/mean function). In the usual case of logistic regression, the link function is the logarithm of the odds ratio, that is, it is $\ln \left( \frac{p}{1 - p} \right)$, because the mean function is $\frac{1}{1 + e^{-x^T \beta}}$. In your case, you have a different mean function, which will correspond to some link function.

Hi~~~ thank you so much for this. I'll have a deeper look into it and let you know~ Thank you so much for all the help!


text:
How do I encrypt and decrypt my model when I run inference on it?
I want my models to be accessible only by my programs. How do I encrypt and decrypt my model when I run inference on it? Is there any existing technology that is widely used?

It's not really possible. If your program runs on the users' machines, they will have access to the encryption keys (as your program will have them). You can make it hard, but if someone is determined enough, they will be able to crack it. I wouldn't waste time on encrypting it.

While @Oliver Mason's comment is correct, and your proposed method won't provide perfect security, you can still protect your models at rest, so that they are stored encrypted in the memory, and your software feed the key at runtime to decrypt it.
On whatever DL inference engine that you have, once it supports loading the model from a buffer (e.g. void*) rather than a file path, you can do the following: read the encrypted model, decrypt it into a buffer, and initialize your neural network model from the decrypted buffer. OpenVino supports loading the model from a buffer.
For encrypt / decrypt the model, any framework such as OpenSSL, or even tiny-AES, can work.
Mentioning again, how to store and use the key is something that should be handled carefully, and a user with sufficient knowledge can read the model and the keys at runtime from the application's memory.


text:
Can you learn parameters in nonlinear function?
In the paper Nonlinear Interference Mitigation via Deep Neural Networks, the the following network is illustrated.
The network structure is
The network parameters are $\theta = \{W_1^{1},...,W_1^{l-1},W_2^{1},...,W_2^{l-1},W^{l},\alpha_1,...,\alpha_{l-1}\}$, where $W_1$ and $W_2$ are linear matrices and $\rho^{(i)}(x)=xe^{-j\alpha_i|x|^2}$ is element-wise nonlinear function ($i$ is the index of layer).
Where should I add this $\rho^{(i)}(x)$? Is it possible to learn the parameter $\alpha$? I don't think it is the same idea as activation function since it is positioned in the middle of two linear matrices... Or can it be added as embedding layer?

Hi and welcome to this community! Can you please tell us which paper you found these expressions in?

Hi thanks! It's in the field of optical communication. check https://arxiv.org/pdf/1710.06234.pdf Figure 1 bottom branch

After a quick look at the relevant parts of the paper, I think that $\rho$ are the activation functions that act element-wise (so if the input to $\rho$ is a vector, the result is also a vector with the same shape as the input vector). However, the activation functions are also followed by another linear transformation, the matrices $W_2^{l}$, where $l$ is the layer. $\alpha$ should be learnable parameters (complex numbers), according to the paper.

Thank you very much, to my understanding, at the end of each layer there should be activation function, so how do you add another linear transformation after that? And is there a way to learn parameters in the activation function?

It is usual the case that the non-linearity is applied after the linear transformations, but, in this architecture, it seems not to be the case: I have not read more than a few lines of the paper, so maybe they explain why they are doing this. They say that the non-linearity is a differentiable function. So, you can take the derivative of this function w.r.t. to its parameter $\alpha$. Once you have the derivative of a differentiable function w.r.t. to one of its parameters, you can use gradient descent to learn those parameters.

Thank you again. And is it possible to learn $W_1$ and $W_2$ at the same time? From what I learned there is only one weight matrix for each layer, so two weight matrices still make sense then? Learning parameters can be $\alpha$ in addition to $W$ and $b$?

You can have more than one matrix associated with a single layer. For example, in a recurrent neural network, for each layer, you have the weight matrix associated with the feed-forward connections and you have another weight matrix associated with the recurrent connections. So, in general, you can have more than one weight matrix for each layer. The way you combine these matrices with the inputs defines also the architecture. So, yes, you can learn both $W_1$ and $W_2$ at the same time (that is, you can update them at the same gradient descent step).

Yes, in general, you can have different parameters and not just the weight matrices. You just need to be able to calculate the partial derivatives of the loss function w.r.t. those parameters.

Thank you so much for your answer! So basically I have to construct my own network functions and not use keras to add dense layer and activation function that way right?

Yes, you may need to do that, that is, create a custom layer with your specific learnable parameters (but I am not super familiar with Keras).

Thank you so much for your help! I should learn more and come back later!! Many thanks!

One more question, could you please recommend some study material for this specific question? I know it's a lot to ask and I really appreciate your help!

The tensorflow tutorials are mostly based on high-level API and there's not much room for customization.

You should be able to implement this model in TensorFlow (but I am not sure), given it's one of the more complete ML frameworks.

Really appreciate it!

In general, you can learn any parameter of the network, provided you can find the partial derivative of the loss function with respect to the desired parameter. Given that $\rho$ is assumed to be differentiable (as the authors state in the paper), you can take the partial derivative of the loss function with respect to the parameter $\alpha$.
In this paper, $\rho$ is a non-linear function (that is, a function that is not linear, e.g. the sigmoid function) that applies element-wise to its input. So, if you pass a vector to this $\rho$, you will get a vector of the same shape out of it. The authors do not explicitly call it an "activation function", but $\rho$ does an analogous job of an activation function, that is, it introduces non-linearity. Furthermore, in this architecture, $\rho$ is also followed by a matrix. In general, this is not forbidden, even though it is not common.
In general, in each layer of a neural network, you can have several different learnable parameters or weights. A parameter is learnable if you can differentiate the loss function with respect to it. You can have more than one weight matrix. For example, recurrent neural networks have usually more than one weight matrix associated with each layer: one matrix is associated with the feed-forward connections and the other matrix is associated with the recurrent connections.


text:
How do we give a kick start to the Facenet network?
I read the Facenet paper and one thing I am not sure about (it might be trivial and I missed it) is how do we give the kick start to the network.
The embeddings, in the beginning, are random, so picking hard (or semi-hard) negatives, based on the Euclidean distance, would give random images in the beginning.
Do we hope that over time this will converge to the actual desired hard images? Is it any reason to expect that this convergence will be attained?

What do you mean by "kick start to the network"? I also don't get what you mean by "picking hard (or semi-hard) negatives". What are "negatives"?


text:
What could be causing the drastic performance drop of the DQN model on the Pong environment?
I am running a basic DQN (Deep Q-Network) on the Pong environment. Not a CNN, just a 3 layer linear neural net with ReLUs.
It seems to work for the most part, but at some point, my model suffers from catastrophic performance loss:

What is really the reason for that?
What are the common ways to avoid this? Clipping the gradients? What else?

(Reloading from previous successful checkpoints feels more like a hack, rather than a proper solution to this issue.)

Try to increase size of replay buffer or increase time update interval for target network.

hmmm, tried both. It still happens. I did analyse the replays though, and it looks like basically a lot of the early strong gains come from learning a single trick to win. And then later the net tries to win without using that trick (i guess it forgets about that at some point), leading to that strong underperformance, until it learns that trick (or another) again. Overall stability improves eventually.

There was good paper from deep mind about similar phenomena - contradicting objectives causing plateau in learning -"Ray Interference: a Source of Plateaus in Deep Reinforcement Learning"

@mirror2image That sounds like it could be a good starting point for an answer. Please consider writing one based around a short summary of that paper.


text:
Will BERT embedding be always same for a given document when used as a feature extractor
When we use BERT embeddings for a classification task, would we get different embeddings every time we pass the same text through the BERT architecture? If yes, is it the right way to use the embeddings as features? Ideally, while using any feature extraction technique, features values should be consistent. How do I handle this if we want BERT to be used as a feature extractor?

BERT is deterministic. There is no variation unless you parse your tokens differently in succeeding runs. Here is the original paper the model architecture is based off of Transformer Paper. Note that in every layer, the only operations used for the most part are matrix multiplications, concatenations, basic ops, and layer normalizations, all of which are deterministic.

Thats what I thought. But when I used the method 1 mentioned in the answers for the question here : https://stackoverflow.com/questions/55369821/how-to-train-a-neural-network-model-with-bert-embeddings-instead-of-static-embed , I am getting different embeddings evrytime I run the code on same data with same tokenizer.

Would you share the web link of "BERT is deterministic."? Or at least where in the paper?

@CloudCho "Bert is deterministic" is not a paper, but a fact derived from everything up to logits are matrix multiplications and normalizations that dont inject any randomness

That's interesting view. Would you share your definition of randomness/?  For "don't inject any randomness", is there other neural network model based on Transformer having the randomness?

@CloudCho my definition of "has randomness" = can take more than one possible value with nonzero probability. and dont know off top of my head of a transformer with injected randomness, but im sure it exists. Is your goal to add priors to activations and/or weights?

Still not clear where we could make input (inject), which part of Transformer structure is/are you mentioning? If you find a Transformer based model with injected randomness, please share. Sorry for confusion, my question was to know why called deterministic.

@CloudCho anywhere. e.g. y=x^2 is deterministic. but if i do y~N(x^2, .1) i added white noise to it. deterministic means no matter how many times i run it, it will produce same output. like y=x^2. if x=3, y is always 9


text:
What is the actual quality of machine translations?
As an AI layman, till today I am confused by the promised and achieved improvements of automated translation.
My impression is: there is still a very, very far way to go. Or are there other explanations why the automated translations (offered and provided e.g. by Google) of quite simple Wikipedia articles still read and sound mainly silly, are hardly readable, and only very partially helpful and useful?
It may depend on personal preferences (concerning readability, helpfulness, and usefulness), but my personal expectations are disappointed sorely.
The other way around: Are Google's translations nevertheless readable, helpful, and useful for a majority of users?
Or does Google have reasons to retain its achievements (and not to show to the users the best they can show)?

Comments are not for extended discussion; this conversation has been moved to chat.

Who claimed that machine translation is as good as a human translator? For me, as a professional translator who makes his living on translation for 35 years now, MT means that my daily production of human quality translation has grown by factor 3 to 5, depending on complexity of the source text.
I cannot agree that the quality of MT goes down with the length of the foreign language input. That used to be true for the old systems with semantic and grammatical analyses. I don't think that I know all of the old systems (I know Systran, a trashy tool from Siemens that was sold from one company to the next like a Danaer's gift, XL8, Personal Translator and Translate), but even a professional system in which I invested 28.000 DM (!!!!) failed miserably.
For example, the sentence:

On this hot summer day I had to work and it was a pain in the ass.

can be translated using several MT tools to German.
Personal Translator 20:

Auf diesem heißen Sommertag musste ich arbeiten, und es war ein Schmerz im Esel.

Prompt:

An diesem heißen Sommertag musste ich arbeiten, und es war ein Schmerz im Esel.

DeepL:

An diesem heißen Sommertag musste ich arbeiten und es war eine Qual.

Google:

An diesem heißen Sommertag musste ich arbeiten und es war ein Schmerz im Arsch.

Today, Google usually presents me with readable, nearly correct translations and DeepL is even better. Just this morning I translated 3500 words in 3 hours and the result is flawless although the source text was full of mistakes (written by Chinese).

Comments are not for extended discussion; this conversation has been moved to chat.

Google's translations can be useful, especially if you know that the translations are not perfect and if you just want to have an initial idea of the meaning of the text (whose Google's translations can sometimes be quite misleading or incorrect). I wouldn't recommend Google's translate (or any other non-human translator) to perform a serious translation, unless it's possibly a common sentence or word, it does not involve very long texts and informal language (or slang), the translations involve the English language or you do not have access to a human translator.
Google Translate currently uses a neural machine translation system. To evaluate this model (and similar models), the BLEU metric (a scale from $0$ to $100$, where $100$ corresponds to the human gold-standard translation) and side-by-side evaluations (a human rates the translations) have been used. If you use only the BLEU metric, the machine traslations are quite poor (but the BLEU metric is also not a perfect evaluation metric, because there's often more than one translation of a given sentence). However, GNMT reduces the translation errors compared to phrase-based machine translation (PBMT).
In the paper Making AI Meaningful Again, the authors also discuss the difficulty of the task of translation (which is believed to be an AI-complete problem). They also mention the transformer (another state-of-the-art machine translation model), which achieves quite poor results (evaluated using the BLEU metric).
To conclude, machine translation is a hard problem and current machine translation systems definitely do not perform as well as a professional human translator.

100 BLEU score doesn't mean human gold-standard translation, it means it matches the reference translation exactly. As there are usually multiple ways to translate a sentence, even human translation usually does not have 100 BLEU, but more like 50-60.

@justhalf Read my answer again.

Thanks for the reply, and sorry if my previous comment appeared rude. My point in my previous comment was that it is inaccurate to give the impression that human translation will get 100 BLEU points, which your current answer seems to do.

@justhalf I just said that $100$ corresponds to a human "gold-standard" translation. However, I also state that the BLUE metric is not perfect, because often there is more than one translation of a given text.

You have asked quite a lot of questions, some of which cannot be answered definitively . To give an insight of the quality (and its history) of machine translations I like  to refer to Christopher Manning his 'one sentence benchmark' as presented in his lecture. It contains one Chinese to English example which is compared with Google Translate output. The correct translation for the example would be:

In 1519, six hundred Spaniards landed in Mexico to conquer the Aztec Empire with a population of a few million. They lost two thirds of their soldiers in the first clash.

Google Translate returned the following translations.

2009 1519 600 Spaniards landed in Mexico, millions of people to conquer the Aztec empire, the first two-thirds of soldiers against their loss.
2011 1519 600 Spaniards landed in Mexico, millions of people to conquer the Aztec empire, the initial loss of soldiers, two thirds of their encounters.
2013 1519 600 Spaniards landed in Mexico to conquer the Aztec empire, hundreds of millions of people, the initial confrontation loss of soldiers two-thirds.
2015 1519 600 Spaniards landed in Mexico, millions of people to conquer the Aztec empire, the first two-thirds of the loss of soldiers they clash.
2017 In 1519, 600 Spaniards landed in Mexico, to conquer the millions of people of the Aztec empire, the first confrontation they killed two-thirds.

Whether Google  retains or 'hides' their best results: I doubt it. There are many excellent researchers working in the field of natural language processing (NLP). If Google would have a 'greatest achievement' for translation, the researchers would figure it out sooner or later. (Why would Google hide their 'greatest achievement' anyway? They seem to see the benefit of open source, see the Transformer[1] or BERT[2])
NB. For an updated list of state-of-the-art algorithms in NLP, see the
SQuAD2.0 leaderboard.
[1] Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems. 2017.
[2] Devlin, Jacob, et al. "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint arXiv:1810.04805 (2018).

Thanks a lot for the link to "well compensated researchers". Having compensations in mind always helps to understand things better (even though I don't know what you had in mind when setting this link).

Argument was not very solid either. Have removed the link and tried to improve the argument. I have been reading a lot of NLP papers and am quite confident in my findings, but its difficult to find support for the argument.

Please let me know about your findings (if you don't mind). My mail address can be found on my profile page.

2019: In 1519, 600 Spaniards landed in Mexico to conquer the Aztec empire of millions of people, and they first met two-thirds of their soldiers.

It really depends on the language pair and the topic of the content. Translating to/from English to any other language usually is the best supported. Translating to and from popular languages works better, for example, translating from English to Romanian is a poorer translation than English to Russian. But translating from English to Russian or Romanian is better than translating Russian to Romanian. And translating Romanian to English is better than translating English to Romanian.
But if you are used to working with translators and you have a passing familiarity with the languages, translation mistakes and the topic, it's easy to understand what was supposed to be there. And, at that point, sometimes its easier to read something translated into your native language for quick scanning than it is to read it in a second language.
Less popular languages (for translation not necessarily in number of speakers) are much much closer to literal translations only slightly better than what you personally would do using a dictionary for two languages you do not know.

Apologies for not writing in English. Please find the adapted translation here:
To give interested people an idea of the quality of MT (DeepL) please see this example from a text I was working on this morning (6,300 words, started at 9 am, delivery today around 1 pm and still find time for this post). I was working on this sentence (201 words) when I posted my comment.

"You further represent, warrant and undertake to ABC that you shall not: (a) Conduct any fraudulent, abusive, or otherwise illegal activity which may be grounds for termination of your right to access or use this Website and/or the Services; or (b) Post or transmit, or cause to be posted or transmitted, any communication or solicitation designed or intended to obtain password, account, or private information from any other user of this Website; or (c) Violate the security of any computer network, crack passwords or security encryption codes, transfer or store illegal material (including material that may be considered threatening or obscene), or engage in any kind of illegal activity that is expressly prohibited; or (d) Run maillist, listserv, or any other form of auto-responder, or "spam" on this Website, or any processes that run or are activated while you are not logged on to this Website, or that otherwise interfere with the proper working of or place an unreasonable load on this Website’s infrastructure; or (e) Use manual or automated software, devices, or other processes to "crawl," "scrape," or "spider" any page of this Website; or (f) Decompile, reverse engineer, or otherwise attempt to obtain the source code of this Website."

DeepL returns this:

Sie versichern, garantieren und verpflichten sich gegenüber ABC, dass Sie dies nicht tun werden: (a) betrügerische, missbräuchliche oder anderweitig illegale Aktivitäten durchzuführen, die Anlass für die Beendigung Ihres Rechts auf Zugang oder Nutzung dieser Website und/oder der Dienste sein können; oder (b) Mitteilungen oder Aufforderungen, die dazu bestimmt sind oder sind, Passwörter, Konten oder private Informationen von anderen Nutzern dieser Website zu erhalten, zu posten oder zu übertragen oder posten oder zu posten oder zu übertragen; oder (c) die Sicherheit eines Computernetzwerks verletzen, Passwörter oder Sicherheitsverschlüsselungscodes knacken, illegales Material (einschließlich Material, das als bedrohlich oder obszön angesehen werden kann) übertragen oder speichern oder sich an illegalen Aktivitäten beteiligen, die ausdrücklich verboten sind; oder (d) Mailliste, Listenserver oder jede andere Form von Auto-Responder oder "Spam" auf dieser Website oder Prozesse, die ausgeführt werden oder aktiviert werden, während Sie nicht auf dieser Website angemeldet sind, oder die anderweitig das ordnungsgemäße Funktionieren oder eine unangemessene Belastung der Infrastruktur dieser Website stören; oder (e) manuelle oder automatisierte Software, Geräte oder andere Prozesse verwenden, um eine Seite dieser Website zu "crawlen", zu kratzen, zu spinnen oder zu spinnen; oder (f) dekompilieren, zurückzuentwickeln oder anderweitig zu versuchen, den Quellcode dieser Website zu erhalten.

It took me about 5 to 10 minutes to adjust this paragraph.
As a translator, I know that I cannot rely on the machine translation, but I learnt the specifics and capabilities of the different systems over time and I know what to pay attention for.
MT helps me a lot in my work.

Notice that legal texts yield better automatic translations, since there's a bucketload of multilingual texts in this area.

Am I wrong and Google's translations are nevertheless readable, helpful and useful for a majority of users?

Yes, they are somewhat helpful and allow you to translate faster.

Or does Google have reasons to retain its greatest achievements (and not to
  show to the users the best they can show)?

Maybe, I don't know. If you search for info, Google does really do a lot of horrible stupid stuff, like learning from what users say on the internet, taking unsuitable data as trusted input data sets.

Surprisingly all the other answers are very vague and try to approach this from the human translator POV. Let's switch over to ML engineer.
When creating a translation tool, one of the first questions that we should consider is "How do we measure that our tool works?".
Which is essentially what the OP is asking.
Now this is not an easy task (some other answers explain why). There is a Wikipedia Article that mentions different ways to evaluate machine translation results - both human and automatic scores exist (such as BLEU, NIST, LEPOR).
With rise of neural network techniques, those scores improved significantly.
Translation is a complex problem. There are many things that can go right(or wrong), and computer translation system often ignores some of the subtleties, which stands out for a human speaker.
I think if we are to think about the future, there are few things that we can rely on:

Our techniques are getting better, wider known and tested. This is going to improve the accuracy in the long run.
We are developing new techniques which can take into account variables previously ignored or just do a better job.
Many of currently existing translation models are often "reused" to translate other languages (for example, try translating "JEDEN" from Polish to Chinese(traditional) using Google Translator - you will end up with "ONE", which is an evidence pointing out the fact that Google translates Polish to English, and then English to Chinese).
This is obviously not a good approach - you are going to lose some information in the process - but it's a one that will still work, so companies like Google use it for languages where they don't have enough workpower or data.
With time, more specialized models will appear, which will improve the situation.
Also, as previous point stated, more and more data will only help improving the machine translation.

To summarize, this complex problem, although not solved, is certainly on a good way and allows for some impressive results for well-researched language pairs.

"Surprisingly all the other answers...", not all other answers. I would say "Some other answers" or "Most other answers".

This will be not so much an answer as a commentary.
The quality depends on several things, including (as Aaron said above) 1) the language pair and 2) the topic, but also 3) the genera and 4) the style of the original, and 5) the amount of parallel text you have to train the MT system.
To set the stage, virtually all MT these days is based off of parallel texts, that is a text in two different languages, with one presumably being a translation of the other (or both being a translation of some third language); and potentially using dictionaries (perhaps assisted by morphological processes) as backoff when the parallel texts don't contain particular words.
Moreover, as others have said, an MT system in no way understands the texts it's translating; it just sees strings of characters, and sequences of words made up of characters, and it looks for similar strings and sequences in texts it's translated before.  (Ok, it's slightly more complicated than that, and there have been attempts to get at semantics in computational systems, but for now it's mostly strings.)
1) Languages vary.  Some languages have lots of morphology, which means they do things with a single word that other languages do with several words.  A simple example would be Spanish 'cantaremos' = English "we will sing".  And one language may do things that the other language doesn't even bother with, like the informal/formal (tu/ usted) distinction in Spanish, which English doesn't have an equivalent to.  Or one language may do things with morphology that another language does with word order.  Or the script that the language uses may not even mark word boundaries (Chinese, and a few others).  The more different the two languages, the harder it will be for the MT system to translate between them.  The first experiments in statistical MT were done between French and English, which are (believe it or not) very similar languages, particularly in their syntax.
2) Topic: If you have parallel texts in the Bible (which is true for nearly any pair of written languages), and you train your MT system off of those, don't expect it to do well on engineering texts.  (Well, the Bible is a relatively small amount of text by the standards of training MT systems anyway, but pretend :-).)  The vocabulary of the Bible is very different from that of engineering texts, and so is the frequency of various grammatical constructions.  (The grammar is essentially the same, but in English, for example, you get lots more passive voice and more compound nouns in scientific and engineering texts.)
3) Genera: If your parallel text is all declarative (like tractor manuals, say), trying to use the resulting MT system on dialog won't get you good results.
4) Style: Think Hilary vs. Donald; erudite vs. popular.  Training on one won't get good results on the other.  Likewise training the MT system on adult-level novels and using it on children's books.
5) Language pair: English has lots of texts, and the chances of finding texts in some other language which are parallel to a given English text are much higher than the chances of finding parallel texts in, say, Russian and Igbo.  (That said, there may be exceptions, like languages of India.)  As a gross generalization, the more such parallel texts you have to train the MT system, the better results.
In sum, language is complicated (which is why I love it--I'm a linguist).  So it's no surprise that MT systems don't always work well.
BTW, human translators don't always do so well, either.  A decade or two ago, I was getting translations of documents from human translators into English, to be used as training materials for MT systems.  Some of the translations were hard to understand, and in some cases where we got translations from two (or more) human translators, it was hard to believe the translators had been reading the same documents.
And finally, there's (almost) never just one correct translation; there are multiple ways of translating a passage, which may be more or less good, depending on what features (grammatical correctness, style, consistency of usage,...) you want.  There's no easy measure of "accuracy".

"Or does Google have reasons to retain its achievements (and not to show to the users the best they can show)"

If they were, then what they're holding back would be amazing. Google publishes a lot of strong papers in Natural Language Processing, including ones that get state of the art results or make significant conceptual breakthroughs.
They have also released very useful datasets and tools. Google is one of the few companies out there that is not only using the cutting edge of current research, but is actively contributing to the literature.
Machine translation is just a hard problem. A good human translator needs to be fluent in both languages to do the job well. Each language will have its own idioms and non-literal or context-dependent meanings. Just working from a dual-language dictionary would yield terrible results (for a human or computer), so we need to train our models on existing corpora that exist in multiple languages in order to learn how words are actually used (n.b. hand-compiled phrase translation tables can be used as features; they just can't be the whole story). For some language pairs, parallel corpora are plentiful (e.g. for EU languages, we have the complete proceedings of the European Parliament). For other pairs, training data is much sparser.  And even if we have training data, there will exist lesser used words and phrases that don't appear often enough to be learned.
This used to be an even bigger problem, since synonyms were hard to account for. If our training data had sentences for "The dog caught the ball", but not "The puppy caught the ball", we would end up with a low probability for the second sentence. Indeed, significant smoothing would be needed to prevent the probability from being zero in many such cases.
The emergence of neural language models in the last 15 years or so has massively helped with this problem, by allowing words to be mapped to a real-valued semantic space before learning the connections between words. This allows models to be learned in which words that are close together in meaning are also close together in the semantic space, and thus switching a word for its synonym will not greatly affect the probability of the containing sentence.  word2vec is a model that illustrated this very well; it showed that you could, e.g., take the semantic vector for "king", subtract the vector for "man", add the vector for "woman", and find that the nearest word to the resulting vector was "queen". Once the research in neural language models began in earnest, we started seeing immediate and massive drops in perplexity (i.e. how confused the models were by natural text) and we're seeing corresponding increases in BLEU score (i.e. quality of translation) now that those language models are being integrated into machine translation systems.
Machine translations are still not as good as quality human translations, and quite possibly won't be that good until we crack fully sapient AI. But good human translators are expensive, while everyone with Internet access has machine translators available. The question isn't whether the human translation is better, but rather how close the machine gets to that level of quality. That gap has been shrinking and is continuing to shrink.

I don't like this approach - but that's a matter of taste and opinion. Doing without a "learned/savant/understanding" translation just because "human translators are expensive" makes me feel sad. What then is translation all about?

@Hans-PeterStricker Translation is about being able to communicate with people with whom you do not share a common language. Machine translation is currently good enough to allow us to do that somewhat well, although the resulting translations are often ungrammatical or sound like a non-native speaker. (continued...)

Depending on what you mean by "learned/savant/understanding", we may already be doing that. That's what the mapping to a semantic vector is; the words are embedded in a vector space that represents their underlying meaning. The Sutskever paper I linked (as "conceptual") actually does translation by mapping the entire sentence onto a semantic vector and then converting that vector into a sentence in the target language. So "understanding" of a sort is definitely happening there. (continued...)

There also exist models that learn the underlying syntax (i.e. sentence structure) , and there has been work on integrating that into neural models, although at the moment, models that learn what parts of the sentence they should pay attention to at any given moment seem to be more effective at handling that sort of thing than the explicit syntactic models. (continued...)

If you don't think that any of this sort of "understanding" counts as True Understanding, then what would count other than an AI that passes the Turing Test, i.e. a fully sapient one? Do note that I never said we can't make a fully sapient AI (I couldn't say how long it'll take; that's not my part of the field. But I have little doubt we'll get there eventually). But the models I'm describing here are what we're using now, and they work fairly well at allowing people to communicate. AI research is all about getting successively better versions of "good enough"


text:
How do I identify a monologue or dialogue in a conversation?
How do I identify monologues and dialogues in a conversation (or transcript) using natural language processing? How do I distinguish between the two?

There are a few different ways you could frame the problem...
For example, the simplest, yet probably not the most effective, would be to treat it as a supervised classification task.
In this case, you would gather data, split it into 2 classes(binary). With one dataset consisting of dialogues and the other monologues. Obviously, this framing comes with it's own set of problems. Namely, how do you prepare the dataset, or how do you deal with live examples?
Of course, there are other ways you could tackle the problem, perhaps the detection context. It all depends on what you feel is the easiest tenable solution given the variables involved in your specific instance.


text:
Can a CNN be trained incrementally?
Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2,  Instead of cleaning all of learned memory, and relearn handwriting data 0, 1, and 2 at the same time.
Can CNN do the same thing? Can CNN learn something new, but keep the previous memory? If yes, the efficiency could be high.  Right now, I have to give all of data at the same time, the efficiency is very very low.

You are looking for incremental (or online) learning.
A CNN can be trained incrementally. For example, in the paper Incremental Learning of Convolutional Neural Networks, the authors propose an incremental learning algorithm (inspired by AdaBoost and Learn++, which is another incremental learning algorithm for supervised learning of neural networks) for CNNs.
However, note that incremental learning is a challenging task, given the  stability-plasticity dilemma: a completely stable model, in order to keep being stable, will attempt to preserve the existing knowledge, so it will not learn new knowledge; similarly, a completely plastic model, in order to keep being plastic, it will keep forgetting previously acquired knowledge so that to learn new information.


text:
Which other loss functions for hierarchical multi-label classification could I use?
I am looking to try different loss functions for a hierarchical multi-label classification problem. So far, I have been training different models or submodels like multilayer perceptron (MLP) branch inside a bigger model which deals with different levels of classification, yielding a binary vector. I have been also using Binary Cross-Entropy (BCE) and summing all the losses existing in the model before backpropagating.
I am considering trying other losses like MultiLabelSoftMarginLoss and MultiLabelMarginLoss.
What other loss functions are worth trying? Hamming loss perhaps or a variation? Is it better to sum all the losses and backpropagate or do multiple backpropagations?

What hierarchical loss functions are worth trying
There are a number of hierarchical loss functions that have been described (see  articles #1 and #2), such as hierarchical win and normalized LCA.  The loss functions that you noted will not by themselves leverage the hierarchical information.


text:
Can machine learning be used to pass the Turing test?
Can we say that the Turing test aims to develop machines or methods to reach human-level performance in all cognitive tasks and that machine learning is one of these methods that can pass the Turing test?

See also Why are deep neural networks and deep learning insufficient to achieve general intelligence?

Essentially yes.
The Turing Test is essentially a benchmark or challenge problem. It is a task that AI researchers would like to be able to solve.
Machine learning is a technique. It is a tool developed by AI researchers to solve various problems. Some kinds of machine learning are applicable to the Turing Test, but others are not. Machine learning is also applicable to a wide range of other problems.

Jobst Landgrebe and Barry Smith, in the paper Making AI meaningful again (2019), argue that machine learning is not sufficient to build an AI that is able to fully (like humans) understand language. They state that current machine learning (or stochastic) approaches might not take into account a lot of contexts while understanding language (e.g. at machine translation), because of the eventual lack of associated data or because of the big number of possible solutions to a language understanding problem.
If understanding language is a necessary skill to pass the Turing test, then current machine learning is not sufficient to build an AI that will pass the Turing test (according to them).


text:
How do I segment each part of a DICOM image?
As I'm beginner in image processing, I am having difficulty in segmenting all the parts in DICOM image.
Currently, I'm applying watershed algorithm, but it segments only that part that has tumour.
I have to segment all parts in the image. Which algorithm will be helpful to perform this task?
The image below contains the tumour.

This image is the actual DICOM image

I looked at some picture from DICOM, and it seems that these images usually don't have many local minima, so I guess that's why watershed isn't optimal.
However, it seems that in DICOM the color gradients are often strong, so I suggest to use techniques that works well with gradients:

Direct Gradient computations, either convolutional gradients or morphological gradients (selects shape of your different objects)
Normalized Cut on Similarity Graphs

If you want to know more about these you can read this post.


text:
In the inception neural network, how is an image of shape $224 \times 224 \times 3$ converted into one of shape $112 \times 112 \times 64$?
According to the original paper on page 4, $224 \times 224 \times 3$ image is reduced to $112 \times 112 \times 64$ using a filter $7 \times 7$ and stride $2$ after convolution.

$n \times n = 224 \times 224$
$f \times f = 7 \times 7$
stride: $s = 2$
padding: $p = 0$

The output of the convolution is $(((n+2p-f)/s)+1)$ (according to this), so we have $(n+2p-f)=(224+0-7)=217$, then we divide by the stride, i.e. $217/2=108.5$ (taking the lower value), then we add 1, i.e. $118+1=119$.
How do we get an output image of $112$ now?

The padding is not size zero* in the inception CNN layers. In fact it is deliberately chosen to pad so that the convolution by itself would produce an image the same size as the original. I.e. $p=(f−1)/2$, in some libraries this is called "same" padding.
So, $p=3$
The stride is not 2. It is $s=1$ for the convolution. The Inception CNN does not use strided convolutions. Instead the stride of 2 is associated with a later max-pooling layer.
Therefore, using $(((n+2p-f)/s)+1)$ with the correct values $(((224 + 6 - 7)/1)+1 = 224$
Then apply max-pooling, with stride 2. $224/2 = 112$.

* Not to be confused with "zero padding" which means pad using $0$ as the value to insert into the new area. So you can have "zero padding with $p=3$"

thank you for your opinion. Even with p=1 which is obviously isn't the case.  Still doesn't explain 112x112

@SanthoshDhaipuleChandrakanth: 112 is just half 224 . . . easy to explain after using "same" padding. Stride is 1 for convolution, the stride of 2 is a max pooling layer, it is not a strided convolution done in one step. I have updated the answer.

@santhosh: what Neil presents here is not an opinion. It is a fact-based answer to your question. The important difference to an opinion is that fact-based answers can be correct or wrong. In contrast to opinions. Opinions are inherently vague and subject to discussion and cannot be right or wrong.


text:
Can the recurrent neural network's input come from a short-time Fourier transform?
Can the recurrent neural network input come from a short-time Fourier transform? I mean the input is not from the time-series domain.

Yes you can apply RNN to any sequence of same data type. The sequence can be in space, time, or any arbitrary ordered list. The items in the sequence can have any data at all, the only requirement is that each represents that same kind of thing (if you have multiple types of thing to process as a sequence, you just need to expand the definition so that the input features can represent all types unambiguously - essentially creating a "base class" that can represent them all).
The RNN will consume the sequence as a time-based sequence, one item per time step of the RNN. However, you can think of that as the same as a processor clock for a computer . . . an RNN is essentially a trainable Turing machine, and in principle can learn to accumulate any data about the sequence it has seen, and output any function of that accumulated data. Although in practice this learning process might be too hard for our current systems, require immense amounts of data etc . . .
In your case, STFT does create a time-based sequence. Each item in the sequence is a frequency analysis for a short period of time, and each time step of the sequence represents a fixed time difference between STFT frames (the windows usually overlap a little), where frequencies in the signal may change. Typically each STFT frame is a single time step input to a RNN. You could input the frequency-domain values in fixed order (e.g. low to high frequency) one at a time into a RNN too, but that would be unusual and would make most learning tasks harder.

@DuttaA: Yes I am sure of my answer. A RNN/LSTM run over a sequence of STFT or MFCC frames is a standard audio processing architecture, used in multiple projects. If you convert a piece fully to frequency domain, then an RNN could still process it - actually that's not so useful for many tasks, but that's not a limitation of RNNs, just a limitation of the representation, there aren't so many target features that are clear from such a conversion, it's just as hard to process as a raw waveform.

@DuttaA: In this case we don't have a fixed sized input, but a series of them.

@DuttaA: I would probably use a CNN (as you suggested in your answer) to process e.g. outputs of a spectrometer for chemical analysis, or other stable signal that could be converted to frequency domain, for exactly the issues you mention.


text:
How to handle rectangular images in convolutional neural networks?
Almost all the convolutional neural network architecture I have come across have a square input size of an image, like $32 \times 32$, $64 \times 64$ or $128 \times 128$. Ideally, we might not have a square image for all kinds of scenarios. For example, we could have an image of size $384 \times 256$
My question is how to we handle such images during

training,
development, and
testing

of a neural network?
Do we force the image to resize to the input of the neural network or just crop the image to the required input size?

I think the squared image is more a choice for simplicity.
There are two types of convolutional neural networks

Traditional CNNs: CNNs that have fully connected layers at the end, and
fully convolutional networks (FCNs): they are only made of convolutional layers (and subsampling and upsampling layers), so they do not contain fully connected layers

With traditional CNNs, the inputs always need to have the same shape, because you flatten the last convolutional layer, with a fixed size. As the flatten layer has a fixed size, the feature map shape from the layer before has to be the same shape, and so the inputs (images) also have to.
However, in FCN, you don't flatten the last convolutional layer, so you don't need a fixed feature map shape, and so you don't need an input with a fixed size.
In both cases, you don't need a squared image. You just have to be careful in the case you use CNN with a fully connected layer, to have the right shape for the flatten layer.
For instance, if you have an input size $320 \times 160$, and you have 3 pooling layers, so your output in the last convolutional layer is $40 \times 20 \times c$ (with $c$ the number of filters/channels), then you just need the flatten layer to have $40*20*c$ neurons.
If you create a new network, just design it to handle a rectangle image.
If you want to use an already pre-trained one, I think the better choice is to resize the image.
If the information in the cropped parts is important, maybe your prediction can be wrong (it depends if the object of interest is in the parts of the image that is cropped). Actually, in Yolo (an object recognition network), images are resized if they don't fit the input requirements. See figure 1 of the YOLO paper. It's because you don't need a high resolution to detect an object (for example, the CIFAR dataset has images of shape $32 \times 32$, but the network can still predict the correct label). So, I think that resizing your image may not affect the prediction much (unless the new size is very different from the original)

If you have a rectangular image and you are using existing models (or existing code), then you have to add an input pre-processing pipeline which transforms the image to standard dimensions. This is very common in computer vision and both PyTorch and Tensorflow have support for easily adding input pre-processing input pipeline for such a transformation.
Also, if you have a fixed size rectangular image data, then you can design your own network architecture (or initial module) which takes image features into account by using asymmetric pooling and convolutions.


text:
How do we stack two U-Nets to yield one final prediction?
I am trying to reproduce the model described in the paper DocUNet: Document Image Unwarping via A Stacked U-Net, i.e. stacking two U-Nets to yield one final prediction. The paper mentions that:

The deconvolution features of the first U-Net and the intermediate prediction y1 are concatenated together as the input of the second U-Net.

What does it mean by concatenating deconvolution features and the prediction (which is an array? cm)?
The next paragraph says that:

The second U-Net finally gives a refined prediction y2, which we use as the final output of our network. We apply the same loss function to both y1 and y2 during training.

It leads to the next question: Does it mean that I have to train U-Net twice?


text:
Which artificial intelligence algorithms could use tensor specific hardware?
AI algorithms involving neural networks can use tensor specific hardware. Are there any other artificial intelligence algorithms that could benefit from many tensor calculations in parallel?  Are there any other computer science algorithms (not part of AI) that could benefit from many tensor calculations in parallel?
Have also a look at TensorApplications and Application Theory.

One place is shaders for graphics. It was the original usage for GPUs anyways.

There are a lot of other potential applications. It's a good idea to start with GPU related problems, since GPUs are essentially doing a slightly wider set of operations, slightly slower. Some possible problems where TPUs might be advantageous are:

Shaders are algorithms for rendering graphics in one style or another. Since computer graphics can be understood as mostly linear algebra, it is natural to view this as operations over tensors.
Physical simulations, which again involve multiplication of vectors by a series of matrices.
Options Pricing, which again involves the multiplication of vectors by a series of matrices, especially when more complex derivatives are prices and the lattice becomes multi-dimensional.

Within AI, there are many other algorithms optimized to work with GPUs, and that could be modified to work with tensor specific hardware. For example, we have:

Genetic Programming on GPUs
Multiagent Systems on GPUs
SAT solving on GPUs

There does not yet seem to be much work optimizing these other problems for Tensor Processing units, but TPUs are also not yet very old. It took several years following the availability of inexpensive consumer GPUs before we started to see widespread use in AI. I suspect we will see more TPU-tailored code for other problems soon.

I think that might also be an application, although I think the advantages of doing so are probably small. In most RL algorithms (e.g. TD-$\lambda$), the change we make in our estimate of the value of one state will depend on the changes we make to the estimates for other states. This means there is an inherently sequential nature to RL updates. I think you might be able to come up with a parallel approach, but it's not as obviously useful as the other domains that were mentioned.


text:
What is the purpose of the "gamma" parameter in SVMs?
I want to understand what the gamma parameter does in an SVM. According to this page.

Intuitively, the gamma parameter defines how far the influence of a single training example reaches, with low values meaning ‘far’ and high values meaning ‘close’. The gamma parameters can be seen as the inverse of the radius of influence of samples selected by the model as support vectors.

I don't understand this part "of a single training example reaches", does it refer to the training dataset?

I've summarized the key ideas of SVMs. So this is how $\gamma$ is used with a gaussian Kernel:
$$K_{\text{Gauss}}(\mathbf{x}_i, \mathbf{x}_j) = e^{\frac{-\gamma\|\mathbf{x}_i - \mathbf{x}_j\|^2}{2 \sigma^2}}$$
The bigger the $\gamma$, the more "linear" the decision boundary will be. The closer to 0, the more support vectors you have/the more non-linear the boundary is (see interactive example)
You can also find it on udacity: SVM Gamma Parameter
In practice, you can use a grid search or random search to get good values.

I think you meant:
the bigger the $\sigma$, or equivalently, the smaller the $\gamma$, the more "linear" the decision boundary will be

Rougly speaking, the higher the gamma, the more complex the model, the higher the risk of overfitting.
In fact, as you can read on the page you linked:

If gamma is too large, the radius of the area of influence of the
  support vectors only includes the support vector itself[...]
When gamma is very small, the model is too constrained and cannot
  capture the complexity or “shape” of the data. The region of influence
  of any selected support vector would include the whole training set.

i not understanding this "of a single training example reaches"

Replace "a single" with "each individual".


text:
Which marketing-related classification challenges is a feed forward neural network suited to solve?
I am trying to think of some marketing-related classification challenges that a feed-forward neural network would be suited for.
Any ideas?

Has a data set been chosen? There's a wide variety of data out there (see this, this, or this).  Analytical possibilities are largely determined by the available data, so choosing your data would probably help to clarify your objectives.


text:
Questions on the identifiability issue and equations 8 and 9 in the D3QN paper
I have difficulty understanding the following paragraph in the below excerpts from page 4 to page 5 from the paper Dueling Network Architectures for Deep Reinforcement Learning.
The author said "we can force the advantage function estimator to have zero advantage at the chosen action."
For the equation $(8)$ below, is it correct that $A - \max A$ is at most zero?

... lack of identifiability is mirrored by poor practical performance when this equation is used directly.

To address this issue of identifiability, we can force the advantage
function estimator to have zero advantage at the chosen action. That is, we let the last module of the network implement the forward mapping

$$Q(s, a; \theta, \alpha, \beta) = V(s; \theta, \beta) + \left( A(s, a; \theta, \alpha) - \max_{a' \in | \mathcal{A} |} A(s, a'; \theta, \alpha) \right). \tag{8}$$

Now, for $a^∗ = \text{arg max}_{a' \in \mathcal{A}} Q(s, a'; \theta, \alpha, \beta) = \text{arg max}_{a' \in \mathcal{A}} A(s, a'; \theta, \alpha)$, we obtain $Q(s, a^∗; \theta, \alpha, \beta) = V (s; \theta, \beta)$.  Hence, the stream $V(s; \theta, \beta)$ provides an estimate of the value function, while the other stream produces an estimate of the advantage function.

I would like to request further explanation on Equation 9, when the author wrote what is bracketed between the red parentheses below.

An alternative module replaces the max operator with an average:

$$Q(s, a; \theta, \alpha, \beta) = V (s; \theta, \beta) + \left( A(s, a; \theta, \alpha) − \frac {1} {|A|} \sum_{a' \in \mathcal{A}} A(s, a'; \theta, \alpha) \right). \tag{9}$$

On the one hand this loses the original semantics of $V$ and $A$ because they are now off-target by a constant,
but on the other hand it increases the stability of the optimization: with (9) the advantages only need to change as fast as the mean, instead of having to compensate any change to the optimal action’s advantage in (8).

In the paper, to address the identifiability issue, there are two equations used. My understanding is both equations are trying to fix the advantage part - the last module.
For equation $(8)$, are we trying to make $V(s) = Q^*(s)$, as the last module is zero?
For equation $(9)$, the resulting $V(s)$ = true $V(s)$ + mean$(A)$? As the author said "On the one hand this loses the original semantics of $V$ and $A$ because they are now off-target by a constant". And the constant refers to mean$(A)$? Is my understanding correct?

Please, next time, ask only one question per post. Even if your questions are related, you should ask each in its separate post, so that people can focus on 1 question at time and future users/readers can find the answer to their specific question more quickly.

Yes, you're correct, if Equation 8 is used it will only be possible to get estimates $\leq 0$ out of the term
$$\left( A(s, a; \theta, \alpha) - \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha) \right).$$
This matches the meaning that we intuitively assign to the $Q(s, a)$, $V(s)$, and $A(s, a)$ estimators (I'm leaving the parameters $\theta$, $\alpha$, and $\beta$ out of those parentheses for the sake of notational brevity). Intuitively, we want:

$Q(s, a)$ to estimate the value of being in state $s$ and executing action $a$ for the policy that we are learning about.
$V(s)$ to estimate the value of being in state $s$ for the policy that we are learning about.
$A(s, a)$ to estimate the advantage of executing action $a$ in state $s$ for the policy that we are learning about.

In the above three points, "the policy that we are learning about" is the greedy policy, the "optimal" policy given what we have learned so far (ideally this would be truly the optimal policy after a long period of training).
In the last point of the three points above, advantage can intuitively be understood as the gain in estimated value if we choose action $a$ over whatever the expected value would be if we were following the policy that we are learning about.
Since we are trying to learn about the greedy policy, we'll ideally (according to our intuition) want the maximum advantage $A(s, a)$ to be equal to $0$; intuitively, the best action is precisely the one we want to execute in our greedy policy, so that best action should not have any relative "advantage". Similarly, all non-optimal actions should have a negative advantage, because they are estimated to be worse than what we estimate to be the optimal action(s).
This intuition is mathematically enforced by using Equation 8 from the paper for training:
$$Q(s, a; \theta, \alpha, \beta) = V(s; \theta, \beta) + \left( A(s, a; \theta, \alpha) - \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha) \right).$$
We can consider two cases to explain what this is doing:

Suppose that action $a$ is the best action we could have selected in state $s$ according to our current estimates, i.e. $a = \arg \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha)$. Then, the two terms in the large brackets are equal to each other, so the subtraction yields $0$, and the state-action value estimate $Q(s, a)$ equals the state value estimate $V(s)$. This is exactly what we want because we are trying to learn about the greedy policy.
Suppose that action $a$ is worse than the best action we could have selected in state $s$ according to our current estimates, i.e. $A(s, a; \theta, \alpha) < \max_{a' \in \vert \mathcal{A} \vert} A(s, a'; \theta, \alpha)$. Clearly, I've just stated here that the first term in our subtraction is less than the second term in our subtraction... so the subtraction yields a negative number. This means that the state-action value estimate $Q(s, a)$ becomes less than the estimated state value $V(s)$. This is also what we want intuitively, because we started with the assumption that action $a$ was a suboptimal action. Clearly, if we assume that the action $a$ is suboptimal, that should lead to a reduction in the estimated value.

Note that afterwards, when they start explaining Equation 9, they actually intentionally deviate from these standard, intuitive understandings that we have of what the three estimators should represent.

Concerning the additional question about Equation 9:
A major problem in the stability of training processes for Deep Reinforcement Learning algorithms (such as these DQN-based algorithms) is that the update targets contain components that are predictions made by the NN that is being trained. For example, the Dueling DQN architecture in this paper generates $V(s)$ and $A(s, a)$ predictions, which are combined into $Q(s, a)$ predictions, and those $Q(s, a)$ predictions of the network itself are also used (combined with some non-prediction reward observations $r$) in the loss function defined to train the Neural Network.
In other words, the Neural Network's own predictions are a part of its training signal. When these are used to update the Network, this will likely change its future predictions in similar situations, which means that its update target will also actually change when it reaches a similar situation again; this is a moving target problem. We do not have a consistent set of update targets as we would in a traditional supervised learning setting for example (where we have a dataset collected offline with fixed labels as prediction targets). Our targets are moving around during the training process, and this can destabilize learning.
Now, in that explanation following Equation 9, they essentially argue that this "moving target" problem is less bad with Equation 9 than it is with Equation 8, which can result in more stable training. I'm not sure if there is a formal proof of this, but intuitively it does make sense that this would happen in practice.
Suppose that you update your network once based on Equation 8. If your learning step changes the prediction of the advantage $A(s, a)$ of the best action $a$ by a magnitude of $1$ (kind of informal here, hopefully it makes sense what I'm trying to say), this will in turn move future targets for updates also roughly by a magnitude of $1$ (again, quite informal here).
Now, suppose that you update your network once based on Equation 9. It is unlikely that all of the different actions $a$ have their advantage $A(s, a)$ move by the same magnitude and in the same direction as a result of this update. It is more likely that some will move up, some will move down, etc. And even if they all move in the same direction, some will likely move by a smaller magnitude than others. In some sense, Equation 9 "averages out" the movements triggered by the learning update in all of these different advantage estimates, which causes the network's prediction targets overall to simply move more slowly, reducing the moving target problem. At least, that's the intuitive idea. Again, I don't think there is a formal proof that this happens, but it does turn out to often help in practice.

Really appreciate your detailed explanation. I I have updated my question with further comments. Could you help me out further? Thanks.

@Cheng See edit in answer for explanation of Equation 9. As for vector or matrix notation of Equation 8, yeah, that's fine. When talking about the Equations, I find it easier to consider them in scalar representation rather than vector representation, but they can be interpreted as vectors.

@Cheng Does my edit help to make things more clear? or is something unclear still?

Thanks @Dennis Soemers again for the detailed explanation. I replied in the updated question.

@Cheng Short explanation: Yes, your thoughts in your last edit are correct. Eq. 8 and Eq. 9 are both intended to address the identifiability issue, in different ways. The advantage of Eq. 8 is that it more closely matches our intuition of what the estimates should "mean". The advantage of Eq. 9 over Eq. 8 is that it suffers less from the "moving target" problem explained in my answer, and hence tends to result in a more stable learning process in practice. Does that help?

Thanks @Dennis Soemers, I updated the question again to include another reply from different source, and my further question. How do you think about my last question?

@Cheng Yes I think your understanding is correct. Maybe the "+ mean(A)" should be "- mean(A)" instead in your last edit, not 100% sure right now. It's a bit confusing with the informal shorthand notations (without full arguments). Subtracting mean advantages instead of max advantages from all $Q$ estimates makes all $Q$ estimates slightly larger (by the same constant) than they "should" intuitively be... so yes, I think you're right.

Thanks @Dennis Soemers, I have different opinion from yours. The Q estimates should remain the same. My understanding is for Equation 9, estimates for V(s) will becomes slightly larger(by a constant). And the constant is mean(A(s)).

@Cheng Yes you're technically right, $Q$ estimates are still the same since they're the final outputs that we train using the standard loss function etc. I was just comparing Eq 8 to Eq 9; in Eq 9, we subtract a smaller $A$ value than in Eq 8 (mean rather than max), so since there's a $Q$ before the $=$ sign, I automatically read that as "well, the quantity on the left-hand side of the equals sign must then also be a bit larger". But technically you're right, the difference would be in the $V$ rather than the $Q$ due to how the learning algorithm works.

@Cheng SE format is not a discussion board. You should not keep updating the question, and thus mutating the topic. If you have another question, just post a follow-up as a separate Q&A.

@BartoszKP, I disagree with you. I believe SE is a knowledge service in the form of Q&A. It should be a form that is easy accessible for a user and also the future questioner. I don't think the same question of identifiability should be divided. That is my understanding.

@Cheng Yes, the purpose of the site is to create a knowledge base for everyone. It should however be a knowledge base consisting of specific questions and specific answers, not chat history between you and the one who answers. Please see: https://meta.stackexchange.com/questions/43478/exit-strategies-for-chameleon-questions and https://ai.stackexchange.com/help/how-to-ask . Your question with all the alterations and responses is really hard to follow. The question post should contain only the question. If you want a clarification use comments, if you want to chat - use chat.

I believe that is explained on the prior page:

"Intuitively, the value function $V$ measures the how good it is to be in a particular state $s$. The $Q$ function, however, measures the the value of choosing a particular action when in this state. The advantage function subtracts the value of the state from the $Q$
function to obtain a relative measure of the importance of
each action."

Then two paragraphs above were you started your quote:

"However, we need to keep in mind that $Q(s, a; \theta, \alpha, \beta)$ is only a parameterized estimate of the true $Q$-function. Moreover, it would be wrong to conclude that $V (s; \theta, \beta)$ is a good estimator of the state-value function, or likewise that $A(s, a; \theta, \alpha)$ provides a reasonable estimate of the advantage function.
Equation (7) is unidentifiable in the sense that given $Q$ we cannot recover $V$ and $A$ uniquely. To see this, add a constant to $V (s; \theta, \beta)$ and subtract the same constant from $A(s, a; \theta, \alpha)$. This constant cancels out resulting in the same $Q$ value. This lack of identifiability is mirrored by poor practical performance when this equation is used directly."

Another way of looking at it would be:

You receive answers to your question

Answers receive votes

Answerers have reputation

In a perfect world people could vote based on reputation, with a weighing based upon the correctness of the answer.
You could simply look at which answer received the most votes and choose it as correct.
In the real world things don't work that way, things are correct or incorrect whether they are measured or not (think quantum mechanics) and measurement doesn't always reveal the true answer.
See: Parameter Estimation.
The estimate of the advantage is only so good, sometimes it's useful to consider it and in other instances it's useful to reject it - intelligently doing both maximizes it's usefulness.

Thanks @Rob. I commented in the updated question. If you understand the paper, please do help me out. Thanks.


text:
Is a linear activation function (in the output layer) equivalent to an identity function?
I have a simple question about the choice of activation function for the output layer in feed-forward neural networks.
I have seen several codes where the choice of the activation function for the output layer is linear.
Now, it might well be that I am wrong about this, but isn't that simply equivalent to a rescaling of the weights connecting the last hidden layer to the output layer? And following this point, aren't you just as well off with just using the identity function as your output activation function?

Do you mean using a linear activation function that is not the identity function? Like f(x)=2x?

And following this point, aren't you just as well off with just using the identity function as your output activation function?

When someone declares that the output of a neural network layer is linear this is exactly what they mean. It can also be described as "no activation function".
Saying that a NN layer has linear activation is a kind of short-hand for saying "the whole layer is a linear function of its inputs", which is true without adding any activation function, just using the weights and bias.
There is usually no separate linear function applied, and libraries such as Keras include the term 'linear' only for completeness, or so that the choice can be made explicit in the code, as opposed to an unseen default.
Note that the link to Keras activation definition above says:

Linear (i.e. identity) activation function.


text:
Which evaluation methods can I use for image segmentation?
I implemented an image segmentation pipeline and I trained it on the DICOM dataset. I compared the results of the model with manual segmentation to find the accuracy. Is there other methods for evaluation?

See:

Martin Thoma: A Survey of Semantic Segmentation, Section III

Subsection A is about metrics and B is about datasets.
Metrics include: accuracy, IoU, frequency weighted IoU, F-beta score, speed, ...

The paper referenced by Martin Thoma is the go-to for semantic segmentation. However I will also like to add the Panoptic Segmentation metric as an aggregated method to measure both the detection task and segmentation task of the model.
It is a very well-known and widely used metric since it is the standard metric for COCO dataset (segmentation)
This is the paper where the metric is proposed.
And here is the metric: