text:
How does the "Lorem Ipsum" generator work?
I've seen many Lorem Ipsum generators on the web, but not only, there is also "bacon ispum", "space ispum", etc. So, how do these generators generate the text? Are they powered by an AI?

Lorem ipsum generators don't typically use anything considered as AI. Usually they just store large pieces of text and select sections from it randomly - they are very simple. The main goal is to produce "nonsense" text that fills space but does not distract from issues of layout and design. The variations of it are usually just for fun, and like the original, are mostly simple generators which select strings of text from a core data source randomly and without using any AI techniques.
It is possible to build more sophisticated random text generators that work using data structures from Natural Language Processing (NLP).
One popular and easy-to-code data structure is N-grams, which store the frequencies/probabilities of the Nth word given words 1 to N-1. E.g. a bigram structure can tell you all the possible words to come after "fish" e.g. "fish" => ["food" => 0.2, "swims" => 0.3, "and" => 0.4, "scale" => 0.1] To use that structure to generate text, use a random number generator to select a word based on looking up the Nth word's frequency, then shift the list of words being considered and repeat.
A more recent text generating NLP model is recurrent neural networks (RNNs), which have a variety of designs. Popular right now are LSTM networks, and these are capable of some quite sophisticated generation, provided they are trained with enough data for long enough. The blog The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Karpathy has quite a few really interesting examples of using RNNs for text generation. In practice this works similarly to n-grams: Use the RNN to suggest probabilities for next word given words so far, choose one randomly, then feed back the generated word into the RNN and repeat.

If you wanted to generate more I guess you could take the string and convert to a list then you could randomly select as many words as you want, from the list.
Using Python
import numpy as np

lorem = "lLorem ipsum dolor sit amet, consectetur adipiscing elit. ".split()

number_of_words_needed = 20

new_text = []

for i in range(number_of_words_needed):
    new_text.append(lorem[np.random.randint(len(lorem))])

print(new_text)

ipsum ['sit', 'dolor', 'elit.', 'sit', 'sit', 'sit', 'elit.', 'dolor', 'amet,', 'ipsum', 'amet,', 'ipsum', 'dolor', 'Lorem', 'Lorem', 'adipiscing', 'sit', 'elit.', 'consectetur', 'adipiscing']

for reference
Source: "Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source. Lorem Ipsum comes from sections 1.10.32 and 1.10.33 of "de Finibus Bonorum et Malorum" (The Extremes of Good and Evil) by Cicero, written in 45 BC. This book is a treatise on the theory of ethics, very popular during the Renaissance. The first line of Lorem Ipsum, "Lorem ipsum dolor sit amet..", comes from a line in section 1.10.32.
The standard chunk of Lorem Ipsum used since the 1500s is reproduced below for those interested. Sections 1.10.32 and 1.10.33 from "de Finibus Bonorum et Malorum" by Cicero are also reproduced in their exact original form, accompanied by English versions from the 1914 translation by H. Rackham."

Full Latin text of de Finibus Bonorum et Malorum  Original reads "dolorem ipsum, quia dolor sit, amet"


text:
How do I train a bot to solve Katona style problems?
Cognitive psychology is researched since the 1940s. The idea was to understand human problem solving and the importance of heuristics in it. George Katona (an early psychologist) published in the 1940s a paper about human learning and teaching. He mentioned the so-called Katona-Problem, which is a geometric task.
Squares
Katona style problems are the ones where you remove straws in a given configuration of straws to create n unit squares in the end. In the end, every straw is an edge to a unit square. Some variations include 2x2 or 3x3 sizes of squares allowed as well as long as no two squares are overlapping, i.e. a bigger square 2x2 can't contain a smaller square of size 1x1. Some problems use matchsticks as a variation, some use straws, others use lines. Some variations allow bigger squares to contain smaller ones, as long as they don't share an edge viz. https://puzzling.stackexchange.com/questions/59316/matchstick-squares

Is there a way we can view it as a graph and removing straws/matchsticks as deleting edges between nodes in a graph?

If so, can I train a bot where I can plugin some random, yet valid conditions for the game and goal state to get the required solution?

Edit #1: The following problem is just a sample to show where I am getting at. The requirement for my game is much larger. Also, I chose uninformed search to make things simpler without bothering about complex heuristics and optimization techniques. Please be free to explore ideas with me.
Scenario #1:
Consider this scenario. In the following diagram, each dashed line or pipe line represents a straw. Numbers and alphabet denote junctions where straw meet. Let's say, my bot can explore each junction, remove zero, one, two, three or four straws such that resultant state has

no straw that dangles off by being not connected to a square.
a small mxm square isn't contained in a larger nxn square (m<n)
Once straw is removed, it can't be put back.

Initial configuration is shown here. I always need to start from top left corner node P and optimization... the objective is to remove straws in minimum hops from node to node using minimum number of moves, by the time goal state is reached.
       P------Q------R------S------T
       |      |      |      |      |
       |      |      |      |      |
       E------A------B------F------G
       |      |      |      |      |
       |      |      |      |      |
       J------C------D------H------I
       |      |      |      |      |
       |      |      |      |      |
       K------L------M------N------O
       |      |      |      |      |
       |      |      |      |      |
       U------V------W------X------Y

Goal 1 : I wish to create a large 2x2 square.
At some point during, say BFS search (although it could be any uninformed search on partially observable universe i.e. viewing one node at a time), I could technically reach A, blow out all edges on A to create the following.
       P------Q------R------S------T
       |             |      |      |
       |             |      |      |
       E      A      B------F------G
       |             |      |      |
       |             |      |      |
       J------C------D------H------I
       |      |      |      |      |
       |      |      |      |      |
       K------L------M------N------O
       |      |      |      |      |
       |      |      |      |      |
       U------V------W------X------Y

That is one move.
Goal 2 : I want to create a 3x3 square instead.
I can't do that in one move. I need the record of successive nodes to be explored and then possibly backtrack to given point as well if the state fails to produce desired result. Each intermediate state might produce rectangles which are not allowed (also, how would one know how many more and which straws to remove to get to a square) or dangle a straw or worse get stuck in an infinite loop as I can choose to not remove any straw. How do I approach this problem?
#Edit 2:
For validation, figures 3, 4 and 5 are given below.
       P------Q------R------S------T
       |             |      |      |
       |             |      |      |
       E      A      B------F      G
       |             |      |      |
       |             |      |      |
       J------C------D------H      I
       |      |      |      |      |
       |      |      |      |      |
       K------L------M------N      O
       |      |      |      |      |
       |      |      |      |      |
       U------V------W------X      Y

The above figure (3) is invalid as we can't have dangling sticks TG,GI etc.
       P------Q------R------S------T
       |      |                    |
       |      |                    |
       E------A                    G
       |                           |
       |                           |
       J                           I
       |                           |
       |                           |
       K                           O
       |                           |
       |                           |
       U------V------W------X------Y

The above figure (4) is invalid as we can't have overlapping squares
       P------Q------R      S      T
       |             |
       |             |
       E      A      B------F------G
       |             |      |      |
       |             |      |      |
       J------C------D------H------I
       |      |      |      |      |
       |      |      |      |      |
       K------L------M------N------O
       |      |      |      |      |
       |      |      |      |      |
       U------V------W------X------Y

Figure (5) is valid configuration.

Your intuition is right: this is fundamentally a problem for combinatorial search.
You're also right that problems are created by the fact that not every move is valid at state. To fix this, you need to add a function that can determine whether a given state is valid or not, in addition to the usual function that checks whether it is your goal state or not. Before adding each node to the queue of your search algorithm, check whether it is a valid state. If it isn't, don't add it.
The second issue you raise is that your search might enter an infinite loop. Since it is possible to remove zero edges from a state, this is a serious concern. There are two approaches to solving this. First, you can try storing all states that you have already visited in a fast data structure like a Hash Table. Before adding a node to your queue, check if it's already been processed. If it has been, don't add it. This may work, but the memory requirements grow exponentially in the number of moves required for a solution. It's sometimes worth it, but I think you can likely skip it for this problem.
A better approach if you're worried about speed is to switch your algorithm to something like iterative deepening, which has the good properties of BFS, but with much lower memory requirements; or to A* search if you can come up with an admissible heuristic for your domain (a good starting point: counting the number of junctions you'd need to remove sticks from to finish, if the robot could teleport, would be admissible).
Hope this helps!
Edit: Here's some pseudo-code for filtering out invalid moves:
function valid_state(State s){
    for stick in s.remaining_sticks:
        if stick is vertical:
           1. let side = walk up from the middle of stick until it becomes possible to turn right.
           2. let side += walk down from the middle of stick until it becomes possible to turn right.
           3. From the first junction above stick where we can turn right, try to walk *side* steps to the right.
           4. Then try to walk side steps down.
           5. Then try to walk side steps left.
           6. Repeat previous 5 steps but for the nearest junctions where we can turn left instead of right.
        else:
           Do exactly what's in the if above, but substitute "left" for "up" and "right" for "down".
    if we could walk in a square successfully for every stick, this is a valid state, so return true. Otherwise, return false.
}

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Where can I find pre-trained language models in English and German?
Where can I find (more) pre-trained language models? I am especially interested in neural network-based models for English and German.
I am aware only of Language Model on One Billion Word Benchmark and TF-LM: TensorFlow-based Language Modeling Toolkit.
I am surprised not to find a greater wealth of models for different frameworks and languages.

Of course now there has been a huge development:
Huggingface published pytorch-transformers, a library for the so successful Transformer models (BERT and its variants, GPT-2, XLNet, etc.), including many pretrained (mostly English or multilingual) models (docs here). It also includes one German BERT model. SpaCy offers a convenient wrapper (blog post).
Update: Now, Salesforce published the English model CTRL, which allows for use of "control codes" that influence the style, genre and content of the generated text.
For completeness, here is the old, now less relevant version of my answer:

Since I posed the question, I found this pretrained German language model:
https://lernapparat.de/german-lm/
It is an instance of a 3-layer "averaged stochastic descent weight-dropped" LSTM which was implemented based on an implementation by Salesforce.

As a note: transformers (previously known as pytorch-transformers) currently also supports Tensorflow.

This will depend to some extent on what you want to do with the language models.
Some possible resources are:
TensorFlow offers 3 pre-trained language models in the research package.
Cafe's ModelZoo has a single pre-trained model that does video -> captions.
Other packages like Cafe2  offer pre-trained models, but the documentation does not suggest any of them are suitable for language.
Failing this, a good approach might be to email the authors of a paper that adopts an approach you like. Some (but far from all) researchers will be happy to share their models, which you can then use as a starting point for your own.

I mean language model in the standard sense. I updated my question accordingly.

Hmm. It looks like you found a few. I suspect DuttA is correct: if these work well, they are extremely valuable right now. No one is likely to publish then free of change. Your best bet is to contact the authors of relevant papers and see if they'll share something with you.

Yeah, in the meantime, HuggingFace has established https://huggingface.co/models, where many research papers and other projects publish their models.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
What does "hard for AI" look like?
In theoretical computer science, there is a massive categorization of the difficulty of various computational problems in terms of their asymptotic worst-time computational complexity. There doesn't seem to be any analogous analysis of what problems are "hard for AI" or even "impossible for AI." This is in some sense quite reasonable, because most research is focused on what can be solved. I'm interested in the opposite. What I do need to prove about a problem to prove that it is "not reasonably solvable" by AI?
Many papers say something along the lines of

AI allows us to find real-world solutions to real-world instances of NP-complete problems.

Is there a theoretical, principled reason for saying this instead of "... PSPACE-complete problems"? Is there some sense in which AI doesn't work on PSPACE-complete, or EXPTIME-complete, or Turing complete problems?
My idea answer would be a reference to a paper that shows AI cannot be used to solve a particular kind of problem based on theoretical or statistical reasoning. Any answer exhibiting and justifying a benchmark for "too hard for AI" would be fine though (bonus points if the benchmark has a connection to complexity and computability theory).
If this question doesn't have an answer in general, answers about specific techniques would also be interesting to me.

Nice Question!
This is a perennial topic of discussion among AI researchers. The short answer is "we don't really know which topics are hard in general, but we do know which we haven't got good techniques for yet."
Let's start by explaining why AI is not concerned with notions of computational complexity like NP-Completeness. AI researchers figured out in the 90's that most problems that are computationally hard in theory aren't actually hard in practice at all! For example, Boolean Satisfiability, the canonical NP-Hard problem, is known to have hard instances, but in practice, these almost never show up. Even when they do, we can usually get good approximate solutions with minimal computation time. Since many AI problems are reducible to SAT, whole areas of the field just use these approximation techniques and solvers. There's a good, if a bit old, survey here. Since 2008, things have only gotten better. Basically, NP-Hard stuff just isn't that hard. Worst-case complexity is therefore probably the wrong tool to guess at which problems are hard for AI.
At the other end of the scale, we have subjective complexity based on things like how "large" the problem is. This has proven to be pretty unreliable. One example is Go, which I was told would "never be solved by AI" as late as 2010. Clearly, we were completely wrong. We just didn't know which techniques to use yet. Another example is language. Rule-based AI'ers tried at it for decades with minimal success. Probabilistic methods have achieved essentially human-level performance in less time. If you asked a researcher in the 1970s if language was hard for AI, they'd have said yes, but they'd have been wrong. This has been closely related to the advances in computing hardware: techniques that seemed wasteful and slow 40 years ago are now entirely practical. Sometimes they turn out to solve problems really well.
Part of this ties into the issue that AI'ers don't really know or agree on what intelligence is, or what it means to solve a problem. Some AI'ers maintain that language really is hard, and that the systems we use now aren't really solving language, they're just making blind guesses that happen to be right a lot. Under that view, language is still hard. Fodor was a strong proponent of this view in the past, and his writings are a good place to read about it, but I've still heard people espouse views like this at AI conferences as recently as 2015.
Something this usually considered hard right now is making a good general intelligence, that integrates knowledge across many domains and could plausibly pass something like the Turing test. Yet, there has been some progress here (look at virtual assistants), and some AI'ers reckon this might not be as hard as we thought either.
So, basically, we don't know what's hard because we don't know when someone will come up with a new exciting technique that solves problems previously thought to be hard, we know that worst-case complexity isn't a good measurement system for "intelligence" requirements, and we don't know what intelligence really is.

Comments are not for extended discussion; this conversation has been moved to chat.


text:
Why does the clipped surrogate objective work in Proximal Policy Optimization?
In Proximal Policy Optimization Algorithms (2017), Schulman et al. write

With this scheme, we only ignore the change in probability ratio when it would make the objective improve, and we include it when it makes the objective worse.

I don't understand why the clipped surrogate objective works. How can it work if it doesn't take into account the objective improvements?

Ok, so I think I have a better understanding of this now.
Firstly, let's remind the main idea of the PPO : staying close to the previous policy. It's the same idea than in TRPO, but the L function is improved.
So, you wanna make "small but safe steps". With clipped surrogate objective, you don't give too much importance to promising actions. You learn that bad actions are bad, so you decrease their probability according to "how bad" they are. But for good actions, you only learn that they are "a little bit good", and their probability will be just slightly increased.
This mechanism allows you to perform small but relevant updates of your policy.
hope this will help someone :)

Yes I think you understood it well. See Figure 1 on page 3. The red dots depict "where the policy" currently is. If we move down along the line (reduction in objective), we do not do any clipping at all. If we move up along the line (improvement in objective), we also do actually allow that for a little bit, but start clipping if we move too far from the existing policy.

I think @16Aghnar explains the concept quite well. However, by clipping the surrogate objective alone doesn't ensure the trust region as stated in the paper:

Engstrom et al., 2020, Implementation Matters in Deep RL: A Case Study on PPO and TRPO.

The authors inspected OpenAI's implementation of PPO and find many code-level optimizations, I'll list the most important optimizations below:

Clipped surrogate objective
Value function clipping
Reward scaling
Orthogonal initialization and layer scaling
Adam learning rate and annealing

They find that:

PPO-M (2.-5.) alone can maintain the trust region.
(PPO-M: PPO without Clipped surrogate objective, but with code-level optimizations)
PPO-Clip (1. only) cannot maintain the trust region.
(PPO-Clip: PPO without code-level optimizations, but with Clipped surrogate objective)
TRPO+ has better performance comparing to TRPO, and with similar performance comparing to PPO
(TRPO+: TRPO with code-level optimizations used in PPO OpenAI implementation)

A intuitive thought on why Clipped surrogate objective alone does not work is: The first step we take is unclipped.

As a result, since we initialize $\pi_\theta$ as $\pi$ (and thus the ratios start all equal to one) the first step we take is identical to a maximization step over the unclipped surrogate reward. Therefore, the size of step we take is determined solely be the steepness of the surrogate landscape (i.e. Lipschitz constant of the  ptimization problem we solve), and we can end up moving arbitrarily far from the trust region. -- Engstrom et al., 2020


text:
Which layer in a CNN consumes more training time: convolution layers or fully connected layers?
In a convolutional neural network, which layer consumes more training time: convolution layers or fully connected layers?
We can take AlexNet architecture to understand this. I want to see the time breakup of the training process. I want a relative time comparison so we can take any constant GPU configuration.

As CNN contains convolution operation, but DNN uses constructive divergence for training. CNN is more complex in terms of Big O notation.
For reference:

See Convolutional Neural Networks at Constrained Time Cost for more details about the time complexity of CNNs
See What is the time complexity of the forward pass algorithm of a neural network? and What is the time complexity for training a neural network using back-propagation? for more details about the time complexity of the forward and backward passes of an MLP

Indeed, purely empirical experience tells me that fully connected layers train much faster, even with hugely more parameters.  The operations in them are simple, as is the backprop.  CNNs appear to use huge amounts of memory to store all the intermediate gradient terms.


text:
Does depth-first search always stop when it has found the leftmost solution?
I'm a fresh learner of AI. I was told that depth-first search is not an optimal searching algorithm since "it finds the 'leftmost' solution, regardless of depth or cost". Therefore, does it mean that in practice, when we implement DFS, we should always have a checker to stop the search when it finds the first solution (also the leftmost one)?

One of the more standard assumptions when first introducing new students to search algorithms (like Depth-First Search, Breadth-First Search which you've also likely heard about or will hear about soon, etc.) is indeed that our goal is to find some sort of solution, and only find one.
If our intention is to find just a single solution, then yes, you will need to check at every node whether that is a solution node, and you can stop the search process once you've found one.
In practice, there can be all kinds of variants of this idea. Maybe in a different case you are interested in finding all solutions, rather than a single one; in such a case you would naturally not stop the search process after finding the first one, but continue searching.
So, to conclude, it really depends on exactly what you want, why are you using a search algorithm. If you only care about finding a solution, you can stop when you have one.


text:
Which nodes are expanded in the expansion phase of MCTS?
I'm confused regarding a specific detail of MCTS.
To illustrate my question, let's take the simple example of tic-tac-toe.
After the selection phase, when a leaf node is reached, the tree is expanded in the so-called expansion phase. Let's say a particular leaf node has 6 children. Would the expansion phase expand all the children and run the simulation on them? Or would the expansion phase only pick a single child at random and run simulation, and only expand the other children if the selection policy arrives at them at some later point?
Alternatively, if both of these are accepted variants, what are the pros/cons of each one?

By far the most common (and likely also the most simple / straightforward) implementation is to expand exactly one node in the Expansion Phase; specifically, the node corresponding to the very first state selected (semi-)randomly by the Play-Out Phase. This is also pretty much the bare minimum you have to do if you want any form of tree growing at all (which you do).

Other variants are possible too, but are much less common. The variant you suggest in the question is to expand all the children of the final node encountered during the Selection Phase, and run a Play-Out for all of them. I am not familiar with any literature on such a strategy really, never tried that myself. Intuitively, I would expect it to perform very similarly, perhaps slightly worse. Essentially what this would do is that it moves the "behaviour" of the search algorithm slightly more towards Breadth-First Search behaviour, rather than Best-First search behaviour. You spend a bit less of your computation time in the Selection Phase, because every Selection Phase is followed up by for example 6 (or whatever branching factor you have) Play-Outs instead of just a single one. On average I'd expect this to be slightly worse, because the Selection Phase is the primary source of the "Best-First Search" behaviour of the algorithm. I certainly don't expect a change like this to cause a large difference in performance though, if any. It will also likely be domain-dependent; worse in some cases, better in other cases.

A different variant that I did once use myself is to expand every node in the complete line of play followed by the Play-Out phase. You can visualize this as a very "thin", but "deep" expansion, whereas your suggestion discussed above would be visualized as a "shallow" but "wide" expansion (and the conventional expansion strategy of a single node would be "thin" and "shallow").
For this strategy, it is much easier to clearly define the advantages and disadvantages that it has in comparison to the standard strategy. The main advantage is that you retain more information from your Play-Outs, you throw less information away. This is because the Backpropagation phase, after the Play-Out terminates, can only store information in nodes that exist. If you immediately expand the complete line of play followed in the Play-Out Phase, you can store the result (the evaluation in the terminal state) in all of those nodes. If you don't expand the complete Play-Out (e.g. only expand the very first node), you'll have to "skip" all of those nodes which you didn't expand yet (they don't exist), and you can't store results in there yet.
The main disadvantage of this approach is that it requires more memory, the tree grows a lot more quickly.
I would personally recommend this approach if you have very strict limitations on computation time, if you expect to be able to only run very few iterations of the MCTS algorithm. For example, I personally used this in my General Video Game AI agent; this is a real-time game where you have to make decisions every ~40 milliseconds. In such a low amount of time, you cannot run many MCTS simulations. This means that:

You do not expect to run out of memory, even if you grow your tree very quickly, so the increased memory requirements become a non-issue.
Due to the low expected number of iterations, it is extremely important to retain as much information as possible, not throw any information away. If we can't run many simulations, we want to make sure to squeeze every little bit of information we can out of each of them.

For contrast, if you're developing an agent to play a board game, and it has in the order of multiple minutes of thinking time per turn, the standard approach of only expanding a single node per Expansion Phase becomes a lot more appealing. If you're capable of running tens of thousands of iterations or more, it really doesn't hurt if you "forget" about a little bit of information deep down in the tree. The risk of running out of memory also becomes a lot more serious if you're running many iterations, so you don't want to grow the tree too quickly.

Excellent answer. Well formulated and clear. Your alternative variation at the end is fascinating. Quick followup: In the most common variation, nodes can have some children explored, some unexplored. Lets say I'm using UCT to balance exploration/exploitation. How would that work? I assume I would first look at unexplored nodes, and only apply confidence intervals once all children have been explored at least once. Is that correct? Thank you

@chessprogrammer Yes that's correct. Prioritize children that have not been visited yet, only really use the UCB1 equation in the Selection phase if all children already have a visit count of at least 1.


text:
How to parse conjunctions in natural language processing?
Is there an accepted way in NLP to parse conjunctions (and/or) in a sentence?
By following the example below, how would I parse

I drink orange juice if its the weekend or if it's late and I'm tired.

into

it's the weekend

and

it's late

and

I'm tired

?
Implying an action will be taken when one of the above elements at the 1st level of depth is true.
I know when I hear the sentence that it means "its the weekend" OR ("it's late" AND "I'm tired"), but how could this be determined computationally?
Can an existing python/other library do this?

This seems not easy for NLP. I doubt that state-of-the-art NLP tools can reliably determine the correct hierarchical structure of independent clauses. Examples below.
The Berkeley parser gets it basically right in the sense that it can put its late and I'm tired on parallel, and they together on parallel with the weekend. But still not perfect (the weekend should be in the same subtree with It's rather than if its late and I'm tired)

The Stanford parser, which is available in Python (NLTK), incorrectly parsed I\'m tired to the same level of I drink orange juice.

Would you mind sharing your grammar rule for nltk.RegexpParser()? I'm still trying to wrap my head around that.

I didn't use RegexpParser().

The question and answer both ignore that "its" is a PRP$ (possessive pronoun), while "it's" is required for a  legitimate parse.


text:
Why does the value of state change depending on the policy used to get to that state?
From what I understand, the value function estimates how 'good' it is for an agent to be in a state, and a policy is a mapping of actions to state.
If I have understood these concepts correctly, why does the value of a state change with the policy with which an agent gets there?
I guess I'm having difficulty grasping the concept that the goodness of a state changes depending on how an agent got there (different policies may have different ways, and hence different values, for getting to a particular state).
If there can be a concrete example (perhaps on a grid world or on a chessboard), that might make it clear why that might be the case.

I guess I'm having difficulty grasping the concept that the goodness of a state changes depending on how an agent got there

It doesn't.
The value of a state changes depending on what the agent will do next. That is where the dependency on the policy comes in, not in past behaviour, but expectations of future behaviour. The future behaviour depends on the state transitions and rewards presented by the environment, plus it depends on the distribution of actions chosen by the policy.
More formally, the value function of a state is not just a relative and arbitrary scoring system, but equals the expected (discounted) sum of rewards, assuming the MDP follows the given dynamics, including action selection:
$$v_{\pi}(s) = \mathbb{E}_{A \sim \pi}[\sum_{k=0}^\infty \gamma^k R_{t+k+1} | S_t = s]$$
Without identifying a policy, it is not possible to assess a value function. In value-based control methods, the policy to evaluate can be implied, somewhat self-referentially, as the policy that acts greedily (or maybe $\epsilon$-greedily) according to the current estimates of the value function.

If there can be a concrete example (perhaps on a GridWorld or on a chess board), that might make it clear why that might be the case

A very simple deterministic MDP with a start state and two terminal states illustrates this:

Start in state B. Taking the left action is followed by a transition (with $p=1$) to terminal state A, and a reward of $0$. Taking the right action is followed by a transition (with $p=1$) to terminal state C, and a reward of $3$.
What is the value of state B? It depends on what the policy chooses. A deterministic left policy $\pi_1$ has $v_{\pi_1}(B) = 0$, a random policy $\pi_2$ choosing left and right with $p=0.5$ has $v_{\pi_2}(B) = 1.5$. The optimal policy chooses action right always and has $v_{\pi_3}(B) = v^*(B) = 3.0$

I guess this does answer my question of why does a value function explicitly depend on a policy. There's no way of knowing how an agent will behave once it leaves the current state unless it has a policy defining its behaviour.

Just out of curiosity though, why do states having an absolute value (independent of policy) does not make sense? Why is the measure of how good a state is dependent upon how an agent will act once it is in the state?

Is it because we define how good a state is in terms of expected future rewards which implies that by definition a value function needs a policy with it?

@user3079474: Yes, the value function in RL is defined that way, so that's precisely what it means. You could invent some function that allocated an absolute "value" to each state, but that would be a different function. Possibly the closest in RL, would be if immediate reward is allocated according only to state (which is quite common situation - e.g. think of winning any board game, this is often purely a function of state) - that would not be the value function though, but how the reward function was set up.

Some heuristic systems may also allocate values to states, and use them to guide search for game and problem-solving tree searches (like A*). There is a little bit of overlap here, as you can use a learned RL value function in place of a heuristic - that is essentially what AlphaGo and its variants do.


text:
Is there a way that helps me to architect my CNN fundamentally before training?
While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial & error process.

Is there a way that helps me to architect my model fundamentally before training?
Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be
changed - increased or decreased?

P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.

Don't forget the possibility that the relationship between the data and target is not strong in the first place. CNNs are constrained by reality. If you have a really great labeled dataset of cat pictures and traits of their owner, it is fairly unlikely you could perform better than statistical average of labels when tryin to infer e.g. the cat's owner's shoe size from a picture of the cat.

@NeilSlater you are right but in my case I have ruled out the possibility of non architectural flaw for the ease of the answer. However you are very right :)

See Analysis and Optimization of Convolutional Neural Network Architectures (my master's thesis).

Is there a way that helps me to architect my model fundamentally before training?

Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. https://autokeras.com/
See Chapter 3.

Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?

See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.


text:
How fast is TensorFlow compared to self written neural nets?
I made my first neural net in C++ without any libraries. It was a net to recognize numbers from the MNIST dataset. In a 784 - 784 - 10 net with sigmoid function and 5 epochs with every 60000 samples, it took about 2 hours to train. It was probably slow anyways, because I trained it on a laptop and I used classes for Neurons and Layers.
To be honest, I've never used TensorFlow, so I wanted to know how the performance of my net would be compared to the same in TensorFlow. Not too specific but just a rough approximation.

I wanted to know how the performance of my net would be compared to the same in Tensor Flow. Not to specific but just a rough aproximation.

This is very hard to answer in specific terms because benchmarking is very hard and is often wrong.
The main point of TensorFlow as I see it is to make it easier for you to use a GPU and further allows you to use a large supply of programs written in Python/JavaScript that still give C++ level performance.

How fast is TensorFlow compared to self written neural nets?

This is answering the general question of using TensorFlow/PyTorch vs a custom solution, rather than your specific question of how much of a speed up you'd get.
There was a relatively recent MIT paper, Differentiable Programming for Image Processing and Deep Learning in Halide trying to discuss the performance vs flexibility vs time spent coding a solution of 3 different languages.
Specifically they compared a solution in their language Halide vs PyTorch vs CUDA.

Consider the following example. A recent neural network-based
  image processing approximation algorithm was built around a new
  “bilateral slicing” layer based on the bilateral grid [Chen et al. 2007; Gharbi et al. 2017]. At the time it was published, neither PyTorch
  nor TensorFlow was even capable of practically expressing this
  computation. As a result, the authors had to define an entirely
  new operator, written by hand in about 100 lines of CUDA for the
  forward pass and 200 lines more for its manually-derived gradient
  (Fig. 2, right). This was a sizeable programming task which took
  significant time and expertise. While new operations—added in just
  the last six months before the submission of this paper—now make
  it possible to implement this operation in 42 lines of PyTorch, this
  yields less than 1/3rd the performance on small inputs and runs
  out of memory on realistically-sized images (Fig. 2, middle). The
  challenge of efficiently deriving and computing gradients for custom
  nodes remains a serious obstacle to deep learning.

So in general you'll probably get faster performance with TensorFlow/PyTorch than a custom C++ implementation, but for specific cases if you have CUDA knowledge on top of C++ then you will be able to write more performant programs.

A lot. There are all these optimizations that we might not have thought of like combining layers, functions, etc. I am a pytorch guy though, its clean and doesn't get in your way like tensorflow does.


text:
Why is the short-time Fourier transform used for preprocessing audio samples?
I've been told this is how I should be preprocessing audio samples, but what information does this method actually give me? What are the alternatives, and why shouldn't I use them?

OK, I just looked it up. https://ccrma.stanford.edu/~jos/sasp/Mathematical_Definition_STFT.html . . . it seems STFT includes a sliding window function. Probably you would implement STFT using FFT at a lower-level. I was just suspicious that something had been "lost in translation" with "Short-Time" having very similar meaning to "Fast" :-)

Fourier transform is used to transform audio data to get more information (features).
For example, raw audio data usually represented by a one-dimensional array, x[n], which has a length n (number of samples). x[i] is an amplitude value of the i-th sample point.
Using the Fourier transform, your audio data will be represented as a two-dimensional array. Now, x[i] is a not a single value of amplitude, but a list of frequencies which compose original value at the i-th frame (a frame consists of a few samples).
See the image below (from wikipedia), the red graph is an original value of n samples before transformed, and the blue graph is a transformed value of one frame.


text:
What is the AI discipline where an algorithm learns from an initial training set, but then refines its learning as it uses that training?
Imagine a system that is trained to manipulate dampers to manage air flow. The training data includes damper state and flow characteristics through a complex system of ducts. The system is then given an objective (e.g. maintain even flow to all outputs) and set loose to manage the dampers. As it performs those functions there are anomalies in the results which the system is able to detect. The algorithm CONTINUES to learn from its own empirical data, the result of implemented damper configurations, and refines its algorithm to improve performance seeking the optimum goal of perfectly even flow at all outputs.
What is that kind of learning or AI system called?

I believe this can best be done with reinforcement learning via Deep Q Learning. That's where I would start. Steps are:

Initialize a Q table.
Choose an action.
Perform the action.
Measure the reward.
Update the Q.

A neural net will approximate the Q function. See: https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0
Also consider policy gradients, actor critic, and inverse reinforcement learning.


text:
Can you help me understand how weight normalization works?
I am trying to dissect the paper Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks.
Unfortunately, because my math is a little bit rusty, I got a little bit stuck with the proof. Could you provide me with some clarification about proof of the topic?
What I understand is that we introduce, instead of the weight vector $w$ a scalar $g$ (magnitude of original $w$?) and $\frac{v}{\|v\|}$ (direction of original $w$?) of the vector.
$$\nabla_{g} L=\frac{\nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\|\mathbf{v}\|}$$
and
$$\nabla_{\mathbf{v}} L=\frac{g}{\|\mathbf{v}\|} \nabla_{\mathbf{w}} L-\frac{g \nabla_{g} L}{\|\mathbf{v}\|^{2}} \mathbf{v}$$
What I am not really sure about is:
If the gradients are noisy (does this mean that in some dimension we have small and in some high curvature or that error noise differs for very similar values of $w$?) the value will quickly increase here and effectively limit the speed of descent by decreasing value of ($\frac{g}{\|v\|}$). This means that we can choose larger learning rates and it will somehow adjust the effect of the learning rate during the training.
And what I completely miss is:
$$\nabla_{\mathbf{v}} L=\frac{g}{\|\mathbf{v}\|} M_{\mathbf{w}} \nabla_{\mathbf{w}} L$$
with
$$\mathrm{M}_{\mathrm{w}}=\mathrm{I}-\frac{\mathrm{w} \mathrm{w}^{\prime}}{\|\mathrm{w}\|^{2}}$$
It should somehow explain the reasoning behind the idea of the final effects. Unfortunately, I don't really understand this part of the paper, I probably lack some knowledge about Linear Algebra.
Can you verify that my understanding of the paper is correct?
Can you recommend some sources (books/videos) to help me to understand the second part of proof (related to the second set of formulas?)

If the gradients are noisy (does this mean that in some dimension we have small and in some high curvature or that error noise differs for very similar values of w?)

Gradients being noisy means that they are "inconsistent" across different epochs / training steps. With that I mean that they'll sometimes point in one direction, later in a different (maybe the opposite) direction, etc., that the gradients at different time steps give inconsistent/conflicting information, that we don't consistently keep following the same direction through gradient descent but keep jumping all over the place. Note that this is across epochs, it's not across different dimensions within the same epoch.
So, for example, we might have a vector of gradients that looks like $[1, 1, 1, \dots, 1]$ at one time step (i.e., positive gradients in all dimensions), and the next training step get a gradient more like $[-1, -1, -1, \dots, -1]$ (this example is pretty much the "most extreme" kind of noise you can have, completely conflicting directions, in reality it'd generally be less extreme).
Such "noisy gradients" can exist because, in practice, we almost always use estimates of the gradient of our loss function, rather than the true gradients (which we'll generally not even be able to compute). For example, if we have a very large training dataset in a supervised learning setting, we'd ideally use the gradient of the loss function computed across the complete dataset. That tends to be computationally expensive (requires forwards and backwards passes for every single instance in the dataset), so in practice we'll often only use a small minibatch to estimate the gradient; this can result in widely different estimates of the gradient for different minibatches. (note: there can also be different reasons for using minibatches rather than full dataset, such as avoiding overfitting, but that's not too important to consider for this specific question)
A different example of a setting where we can have noisy gradients is in Deep Reinforcement Learning. There, computing the loss function itself is often rather noisy (in the sense that our own predictions, which tend to still be incorrect during the training phase, are a component of the targets that we're updating towards), so our estimates of the gradients will also be noisy.

Derivation for your second question, about the gradient of the loss with respect to $\mathbf{v}$:
We start with the following, from Equation (3) in the paper (this gradient you understand, right?):
$$\nabla_{\mathbf{v}} L = \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g \nabla_g L}{\vert \vert \mathbf{v} \vert \vert^2} \mathbf{v}$$
Note that Equation (3) also gives:
$$\nabla_{g} L = \frac{\nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\vert \vert \mathbf{v} \vert \vert}$$
If we plug that into the previous Equation we get:
\begin{aligned}
\nabla_{\mathbf{v}} L &= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g \frac{\nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\vert \vert \mathbf{v} \vert \vert}}{\vert \vert \mathbf{v} \vert \vert^2} \mathbf{v} \\
&= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g \nabla_{\mathbf{w}} L \cdot \mathbf{v}}{\vert \vert \mathbf{v} \vert \vert^3} \mathbf{v} \\
\end{aligned}
The $\frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L$ term that shows up before the minus can also be isolated in the term after the minus:
\begin{aligned}
\nabla_{\mathbf{v}} L &= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert^2} \mathbf{v} \\
&= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert} \cdot \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert} \\
\end{aligned}
Equation (2) tells us that $\mathbf{w} = \frac{g}{\vert \vert \mathbf{v} \vert \vert} \mathbf{v}$, which means that $\frac{\mathbf{w}}{g} = \frac{\mathbf{v}}{\vert \vert \mathbf{v} \vert \vert}$. Plugging this into the above leads to:
\begin{aligned}
\nabla_{\mathbf{v}} L &= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{w}}{g} \cdot \frac{\mathbf{w}}{g} \\
\end{aligned}
Below Equation (2), they also explain that $\vert \vert \mathbf{w} \vert \vert = g$, so we can rewrite the above to:
\begin{aligned}
\nabla_{\mathbf{v}} L &= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{w}}{\vert \vert \mathbf{w} \vert \vert} \cdot \frac{\mathbf{w}}{\vert \vert \mathbf{w} \vert \vert} \\
&= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L - \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \frac{\mathbf{w} \mathbf{w}'}{\vert \vert \mathbf{w} \vert \vert^2} \\
\end{aligned}
In that last step, the $'$ symbol in $\mathbf{w}'$ denotes that we transpose that vector.
Finally, because we have that common $\frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L$ before and after the minus symbol, we can pull it out, multiplying it once by the identity matrix $\mathbf{I}$, and once by the remaining negated term. This is very similar to how you would simplify something like $a - ax$ to $a (1 - x)$ in "standard" algebra; the identity matrix $\mathbf{I}$ plays a very similar role here to the number $1$ in "standard" algebra:
\begin{aligned}
\nabla_{\mathbf{v}} L &= \frac{g}{\vert \vert \mathbf{v} \vert \vert} \nabla_{\mathbf{w}} L \cdot \left( \mathbf{I} - \frac{\mathbf{w} \mathbf{w}'}{\vert \vert \mathbf{w} \vert \vert^2} \right) \\
\end{aligned}
Note: in all of the above, I kind of neglected to pay attention to order of matrix/vector multiplications and their dimensions. Probably the $\nabla_{\mathbf{w}} L$ term should by now already have been moved to after those large brackets, instead of being before those brackets. In this case, it should work out fine regardless because $\mathbf{I}$ as well as $\mathbf{w} \mathbf{w}'$ are square and symmetric matrices. The only resulting difference is in whether you get a row or a column vector out at the end.
This can now finally be rewritten as the Equation you mentioned that you didn't understand yet, Equation (4) in the paper.

Hi Dennis, that's it! Thank you for the explanation, now I understand what was meant in the paper! It seems that I really need to work on my math and ml basics. Thank you again for the explanation!

@GrumpyC Well, that depends on what you want to do exactly. The ability to reproduce these kinds of math details is very different from the ability to implement the ideas and/or roughly understand the intuition. It's definitely useful to have a good basis in Linear Algebra though.

Hi Dennis, I think that both is necessary, intuition is probably enough for most of the practicioners, but I cannot but admire silent elegance of mathematics, in describing the world. Well I should properly read and practice! :D Thank you again!


text:
What is the name of an AI system that learns by trial and error?
Imagine a system that controls dampers in a complex vent system that has an objective to perfectly equalize the output from each vent. The system has sensors for damper position, flow at various locations and at each vent. The system is initially implemented using a rather small data set or even a formulaic algorithm to control the dampers. What if that algorithm were programmed to "try" different configurations of dampers to optimize the air flows, guided broadly by either the initial (weak) training or the formula? The system would try different configurations and learn what improved results, and what worsened results, in an effort to reduce error (differential outflow).
What is that kind of AI system called? What is that system of learning called? Are there systems that do that currently?

This question is very similar to this one, if not an exact duplicate.

I believe "Reinforcement Learning" is the term you are looking for (as mentioned by others as well) but keep in mind that the scope of your problem falls under the section of AI that is called Search.
Search algorithms are based upon experimenting with different actions (decisions) and selecting the one that minimizes an arbitrary cost function (reward), given the current and past problem states.

Near solution to your problem definition is reinforcement learning. You can define some reward using the objective function and define some possible state space for the machine and finally solve the problem by reinforcement learning techniques (near to trial and error by learning the preferences).

I think any learning algorithm probably uses trial and error and analysis of the results with the ultimate goal of maximizing utility.
It seems that the recent milestones in AI fall under the general umbrella of machine learning, which includes all forms of reinforcement learning. Essentially, any learning algorithm is using some form of statistical analysis.

For an umbrella term, I've been using "learning algorithm"

However, there is also a venerable history of less capable adaptive systems such as self-organizing networks.  (See also optimal control.)


text:
What is the name of the NLP technique that determines "who did what to whom" given a sentence?
Within a piece of text, I'm trying to detect who did what to whom.
For instance, in the following sentences:

CV hit IV. CV was hit by IV.

I'd like to know who hit whom.
I can't remember what this technique is called.

You might be referring to Semantic role labeling. SRL is the task of assigning labels to words or phrases in a sentence that shows their semantic role in that sentence.
In your example CV was hit by IV, the task is to identify the verb "hit" carried out by the actor "CV" affected "IV" the recipient.
Note: If you're only interested in the syntactic relationship among words or phrases in a sentence, not the semantic relationship between them, simple dependency parsing would do the job.


text:
How does L2 regularization make weights smaller?
I'm learning logistic regression and $L_2$ regularization.
The cost function looks like below.
$$J(w) = -\displaystyle\sum_{i=1}^{n} (y^{(i)}\log(\phi(z^{(i)})+(1-y^{(i)})\log(1-\phi(z^{(i)})))$$
And the regularization term is added. ($\lambda$ is a regularization strength)
$$J(w) = -\displaystyle\sum_{i=1}^{n} (y^{(i)}\log(\phi(z^{(i)})+(1-y^{(i)})\log(1-\phi(z^{(i)}))) + \frac{\lambda}{2}\| w \|$$
Intuitively, I know that if $\lambda$ becomes bigger, extreme weights are penalized and weights become closer to zero. However, I'm having a hard time to prove this mathematically.
$$\Delta{w} = -\eta\nabla{J(w)}$$
$$\frac{\partial}{\partial{w_j}}J(w) = (-y+\phi(z))x_j + \lambda{w_j}$$
$$\Delta{w} = \eta(\displaystyle\sum_{i=1}^{n}(y^{(i)}-\phi(z^{(i)}))x^{(i)} - \lambda{w_j})$$
This doesn't show the reason why incrementing $\lambda$ makes weight become closer to zero. It is not intuitive.

Here is my take.
The larger the $\lambda$, the more the corresponding regularization term for a coefficient will be big, so when minimizing the cost function, the coefficient will be reduced by a bigger factor, you can see this effect in the derivation of the update rule for gradient descent for example:
\begin{align*} \theta_j := \theta_j - \alpha\ \left[ \left( \frac{1}{m}\ \sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})x_j^{(i)} \right) + \frac{\lambda}{m}\theta_j \right] &\ \ \ \ \ \ \ \ \ \ j \in \lbrace 1,2...n\rbrace\newline & \end{align*}
\begin{align*} \theta_j := \theta_j (1- \alpha \frac{\lambda}{m})\left( \frac{\alpha}{m}\ \sum_{i=1}^m (h_\theta(x^{(i)}) - y^{(i)})x_j^{(i)} \right) &\ \ \ \ \ \ \ \ \ \ j \in \lbrace 1,2...n\rbrace\newline & \end{align*}
From this derivation, it is clear that at every update the coefficients get reduced by a factor that is usually a little less than 1 and directly proportional to $\lambda$, so, as $\lambda$ gets bigger, the weights get reduced more and more; eventually, for very big values of $\lambda$, we risk to totally underfit the data, since only the regularization term will remain in the cost function and all the weights will go to zero.
This is for linear regression, but it is essentially the same logic also for logistic regression.
This is taken from Andrew Ng's course on Coursera. A more mathematical precise (and complex) traction of the problem can be found in the Bloomberg machine learning course material.
PS: in the derivation of the update rule for gradient descent, the $\lambda$ should be divided by the number of training examples, this is important in choosing the right $\lambda$ because we want this relationship to be inversely proportional, otherwise the coefficient won't be decreased.


text:
Is there research that employs realistic models of neurons?
Is there research that employs realistic models of neurons? Usually, the model of a neuron for a neural network is quite simple as opposed to the realistic neuron, which involves hundreds of proteins and millions of molecules (or even greater numbers). Is there research that draws implications from this reality and tries to design realistic models of neurons?
Particularly, recently, Rosehip neuron was discovered. Such neuron can be found only in human brain cells (and in no other species). Are there some implications for neural network design and operation that can be drawn by realistically modelling this Rosehip neuron?

State of Rosehip Research
The Rosehip neuron is an important discovery, with vast implications to AI and its relationship to the dominant intelligence on earth for at least the last 50,000 years.  The paper that has spawned other articles is Transcriptomic and morphophysiological  evidence for a specialized human cortical GABAergic cell type, Buldog et. al., September 2018, Nature Neuroscience.
The relationship between this neuron type and its DNA expression is beginning.  No data is available regarding the impact of the Rosehop distinctions on neural activity during learning or leveraging what has been learned.  Surely, research along those lines is indicated, but the discovery was just published.
Benefit of the Interdisciplinary Approach to AI
That those who reference papers like this can see value in the unification or at least alignment of knowledge across disciplines is most likely beneficial to AI progress and progress in the other fields of cognitive science, bioinformatics, business automation, manufacturing and consumer robotics, psychology, and even law, ethics and philosophy.
That such interest in aligning understanding along interdisciplinary lines is present in AI Stack Exchange is certainly beneficial to the community growth in both professional and social dimensions.
Disparity Between What Works
In the human brain, neurons work.  Whether Rosehip neurons are a prerequisite to language, the building of and leveraging of complex models, or transcendent emotions such as love in homo sapiens is unknown and will remain so in the near future.  However, we have a fifty millennia long proof of concept.
We also know that artificial networks work.  We use them in business, finance, industry, consumer products, and a variety of web services today.  When a pop-up asks whether the answer given was helpful, our answer becomes a label in a set of real data from which samples are extracted for machine learning.
Nonetheless, the cells that are working are offspring of the 1957 perceptron with the addition of the application of gradient descent using an efficient corrective signal distribution strategy we call back propagation.  The comprehension of neuron function in 1957 was grossly short of what we now know to be functional features of mammalian brain neurons.  The Rosehip discovery may widen that gap.
Spiking Networks
The spiking network research more realistically models neurons, and neuromorphic research and development has been placing improved models into VLSI chips.  The joint venture between IBM and MIT is another.
Correlating Neural Function to Brain Function
The relationship intelligence and the number of proteins or molecules may not be the most telling.  These are more likely relationships between metrics and features and the intelligence of the system.

Genetic features that have been identified (22 of them) that directly affect intelligence testing results — For instance the correlation between polymorphisms of the oxytocin receptor genes OXTR rs53576, rs2254298, and rs2228485 and intelligence is known — See the question containing references to discovery of 22 genes that affect intelligence test results significantly
Neurochemical expression resulting from environmental factors varying the levels of oxytosin, dopamine, serotonin, neuropeptide Y, and canabinoids which is involved in global and regional functional behavior in the human brain
Signal topology (distinct from sizes and counts and distinct from the topology of created by packaging neural nets in the cranial region) — Signal topology is now being identified.  Scanning technology has developed to the point where signal paths can be identified by tracking pulses in temporal space and determine causality.
Synaptic plasticity, a type of neural plasticity
Total number of neurons applied to a particular brain function
Impact on axon and cell body thermodynamics on signal transmission, a key element in modelling a brain neuron

None of these are yet modelled in such a way that simulation accuracy has been confirmed, but the need to research along these lines is clearly indicated as this question implies.

It looks like you really have two questions here. I'll try to answer the first one, and you should think about making a separate question for the second.
There is research into using simulated models of biologically realistic neurons. While there are large projects like the Human Brain Project aimed at simulating human brains, there is also a lot of lower-level AI research. SPAWN is an interesting system that got a lot of press a few years ago, and has continued to be developed since then. It uses realistic neurons to simulate several brain-regions at once, creating a surprisingly general AI system that could perform many types of motor and vision tasks using the same basic design.

@DuttA, there's a spectrum. See Eliasmith's book Neural Engineering for more. Basically, you call build a far more accurate simulation of a neuron than the usual RLU or Sigmoid models without doing the full biochemical simulation. These more realistic simulations are useful with modern hardware, and can be simulated in very large numbers.

It is true that the current Machine learning is based on treating neurons as a component in the whole complexity , mesh of neurons. The focus is more on the architecture rather than understanding or imitating the basic block of it more clearly , i.e. the neurons.
Anirban Bandhopadhyay is a biologist and Neurologist who has studied how the harmony changes the memory element and decision making power in microtubles inside the neurons.
Here, is the snippet of him explaining , and trying to see what exactly computation is , and how the brain does computation.
How does the Brain Act?


text:
How can I apply reinforcement learning to solve this asteroid game?
Introduction
An attractive asteroid game was described in the paper Learning Policies for Embodied Virtual Agents through Demonstration (2017, Jonathan Dinerstein et al.):

In our first experiment, the virtual agent is a spaceship pilot, The pilot's task is to maneuver the spaceship through random asteroid fields

In theory, this game can be solved with reinforcement learning or, more specifically, with a support vector machine (SVM) and epsilon-regression scheme with a Gaussian kernel. But it seems that this task is harder than it looks like, as the authors of the same paper write

Although many powerful AI and machine learning techniques exist, it remains difficult to quickly create AI for embodied virtual agents.

it is quite challenging to achieve natural-looking behavior since these aesthetic goals must be integrated into the fitness function

Questions
I really want to understand how reinforcement learning works. I built a simple game to test this. There are squares falling from the sky and you have the arrow keys to escape.

How could I code the RL algorithm to solve this game? Can I do this manually in Javascript according to what I think should happen? How can I do this without having to map the positions of the rectangles and mine, just giving the agent the keyboard arrows to interact and three information:

Player life
Survival time
Maximum survival time

A closely related question and a minimal implementation written in Python.
That program implements the reinforcement learning technique 'Q-Learning'.
The idea is for the program to take in an observation of the environment (which could be a screenshot if learning a computer game, or sensor data for a robot) and output a decision in the form of a vector of values. Each cell in that output vector corresponds to a possible action (left, right, shoot, etc) and the highest valued cell shows the action that the agent/player should take. The values in that output vector are called the Q-Values, and it is the mapping from the input data to the vector of Q-Values that we are trying to learn.
In your case, the function that takes in an observation of the environment and spits out a vector of decision choices is your SVM.
The question linked above contains a description of the training algorithm. It boils down to playing the game multiple times while storing the input vector, output vector and output decision for each step until you reach a termination condition (ie hitting an asteroid) and receiving a score (in this case a negative score since you want to avoid the asteroids). Then, going backwards through the the output vectors, assigning a slowly reducing amount of the output score to the particular cell in each output vector corresponding to the decision that was made at that step of the game.
The array of input vectors from the game, and the array of output decision vectors that you have just created become the training data for your machine learning system. Once you have completed a training run, play the game again with your trained system and keep repeating until it is a good as you need it to be, or it stops getting any better.
To address your question directly, I don't think that you are going to be able to train your system without giving it information about the position of the asteroids (assuming that they are positioned at random). However, you could try just having a very simple input vector such as 3 integer values where each value corresponds to the presence (1) or absence (0) of an asteroid in the 3 squares to the above left, above and above right of the player. That might be enough to encourage it to dodge away from falling rocks...
There are some Support Vector Machines implemented in JavaScript, although I have never used them, but since SVMs are basically two-class classifiers you would have to check that the library can easily support multi-class SVMs. In your case the SVM would be trying to classify each input into a particular player choice (left, right, shoot, etc).


text:
Should I be decaying the learning rate and the exploration rate in the same manner?
Should I be decaying the learning rate and the exploration rate in the same manner? What's too slow and too fast of an exploration and learning rate decay? Or is it specific from model to model?

@16Aghnar in what scenario would I use a lower learning rate? I've looked at the ATARI papers and games like Super Mario Bros. They used learning rates of 0.00025. Is it because we want it to not get hooked onto the same decision again and again? As in, with a higher learning rate, it would assume action x would be best. I hope I made sense!

So, a lower LR means a slower convergence but an improved asymptote (limit of the learning curve). So tuning it depends on the time you have, and also on your model. You can begin with, for example, 0.001, see the learning curve, and if you reach quickly the asymptote, you can try with a lower LR, see again the learning curve, and so on. (and notice that the 0.98 - 0.997 values I mentioned in my answer are for the LR decay, not for the LR)

First of all, I'd say that there is a reason to give Learning Rate (LR) and Exploration Rate (ER) the same decay: they play at the same scale (the number of successive batches you'll train your model on). But if I refine the analysis, I would rather say that it's a reason to choose them in the same range, i.e. close to 1, but not specifically at the same number.

For LR decay, people often choose it very close to one (which can mean really different things like 0.98 or 0.997), because it plays on a large scale, and you don't want the LR to disappear too brutally.

However, the choice of ER decay can have more variation from model to model. It depends on the initial value of ER (you don't wanna decay fastly ER if your ER is initially low), and also to the "learning speed" of your model: if your model learns efficiently at the beginning, you could want to fastly decrease ER in order to reduce the noise on the action, supposing that you did enough exploration at the beginning (but I think this last opinion is more controversial). You can find an interesting paper here, where the author tries different ER decay and finds out that 0.99 is the best, for CartPole environment.


text:
Does AI rely on determinism?
I don’t believe in free will, but most people do. Although I’m not sure how an act of free will could even be described (let alone replicated), is libertarian freewill something that is considered for AI? Or is AI understood to be deterministic?

It looks like the definition you have used requires that "free will is logically incompatible with a deterministic universe" - if we accept that definition, and also accept that free will exists, then probably it does not prevent the creation of non-deterministic machines which can have that form of free will. But either way, it allows a discussion to go forward.

See the Strong Free Will Theorem.  I feel like I'm starting to get a handle on this (although that feeling may be illusory;) but the bits about locality are interesting.
I've been studying games formally for a little while and it seems that Indeterminacy only arises from inaccessible information, including intractability and random number generation.  Sans indeterminacy, the concept free will seems meaningless, but that even deterministic models (such as non-chance, perfect information games) can produce indeterminacy.

I'm going to assume that by free will, you mean something like the philosophical concept of libertarian free will, which is defended by philosophers like Robert Kane. In Libertarian Free Will, individuals have some capability to make choices about their actions. The classic way to argue this is by assuming some kind of spirit-stuff (e.g. a soul) that exists outside the material world, and that this spirit-stuff constitutes the consciousness of a person. Kane tries some mental gymnastics to avoid this, but then concedes something like it in a footnote. I'm not aware of any serious work that doesn't make some kind of non-physical assumption to justify this view. If someone can point at one, I'll update the answer.
By determinism, I'm going to assume you mean the usual notion of philosophical determinism: since people's decisions depend on what happened in the past, and where they are in the present, they don't really have a choice in any meaningful sense. Philosopher's like Dennett adopt a slightly softer view (compatibilism, essentially: you don't get to make big choices, but you do get to make small ones). Appeals to Quantum Mechanics are common to justify that view. In this context, free action means something more like "did something we couldn't predict exactly". An example might be: you are pre-destined to put a can of campbell's brand tomato soup in your shopping cart, but "make a choice" about exactly which of the dozens of cans you will put in. Since small choices can have large impacts (maybe that can will give you food poisoning, and the others wouldn't), this can make all sorts of things impossible to predict exactly.
I think most AI researchers don't worry too much about these issues, but Turing actually addresses them in his paper right at the start of the field, Computing Machinary and Intelligence. The deterministic/compatibilist view point is introduced as Lady Lovelace's objection: Computers only know how to do what we tell them to, so they can't be called intelligence.
Turing's counterargument uses two prongs. First, Turing notes that computers can probably be made to learn (he was right!). If they can learn, they can do things that we don't expect, just like children do. Second, Turing notes that computers already do all sorts of things we don't expect: they have bugs. Anytime a computer exhibits a bug, it did something that was unexpected. Since we cannot generally rule out bugs in programs, computers will always do surprising things. Therefore, computers satisfy the deterministic notion of free will.
Turing also addresses the libertarian notion of free will, which is part of what he calls the "Theological Objection". The objection is that intelligence requires some kind of divine spark (like free will). Turing argues that we can't detect sparks like this right now (he actually thought we would be able to one day, and spent a lot of time looking at supernatural phenomena too). However, there's no reason to suppose that computers with the right programs won't be endowed with them. A divine creator could decide that anytime you build something brain-like, it gets a spark. If we build a program that's brain-like, maybe it gets a spark too. In the absence of some way to detect souls, it seems like we ought to just agree to treat things that seem intelligent as though they had these souls, since otherwise we don't have a really clear way to decide who is and isn't intelligent. The only remaining way is to say "only things made of human meat have souls and are intelligent". While a lot of people do actually say things like this (e.g. animals have no souls), this is a pretty arbitrary view, and I think there's no hope arguing against it. Turing suggests the "Anthropic Principal": we shouldn't assume that we're special, because the earth isn't in a special place in the galaxy, or in the universe, and we have pretty compelling evidence that we're an evolved version of other animals around us, but some groups (e.g. biblical literalists) find this unconvincing.

Comments are not for extended discussion; this conversation has been moved to chat.

AI is "deterministic" in the sense that it follows exactly the algorithm. "Deterministic" means different things to a data scientist/programmer, but let's not go into details here.
There is no "freewill" in AI, it's all about mathematics and algorithms. Don't watch too many scientific movies!

How are you defining "deterministic" and "freewill" in this answer? As these are difficult concepts, your answer needs some working definitions.

I'm wondering though, if a learning algorithm develops novel goals, does that demonstrate volition/intentionality, analogous to the free will we humans perceive ourselves as having?  (PS- I'm thinking you meant to say "scientific" movies, in terms of misleading, quasi-science?)

Although I’m not sure how an act of freewill could even be described (let alone replicated),

Well, one popular definition goes like this:

[Free will is] the freedom to act according to one's motives without arbitrary hindrance from other individuals or institutions

Source - Wikipedia entry on Compatibilism
Note that this definition is perfectly compatible with determinism (hence the name "Compatibilism"). Actually, proponents usually argue that free will requires determinism, because if your choices were ultimately random, like rolling a dice, how could they be your free choices?
Now, if you assume that an AI can be said to have "motives", then according to this view, it would have free will - if noone hinders it.
The contrary view, Incompatibilism, has been described in another answer by John Doucette. I agree with him that most AI researchers probably don't worry about philosphical questions like that. All the proponents of indeterminism (sometimes called metaphysical free will or libertarianism) I'm aware of assume that there exist "causes" that are neither deterministic / physical, nor purely random. (e.g. Agent causation). Since AIs are more or less by definition based on physical processes, deterministic algorithms and possibly some random number generator, I don't see how they could posess this kind of freedom.

Thanks. Does this mean, according to Compatibilism, if something should cause an AI computer to “decide” to destroy all humans, and nobody pulls the cord, then the AI would be free to carry out this decision- thus having freewill?

@anonymouswho: Probably yes. If nothing forces you to pull the trigger (e.g. by holding your hand, threatening you etc.), then it's your free decision. Which is pretty close to our everyday intuition.

What would cause an AI to decide to destroy humanity? Perhaps a virus? Or maybe it’s battery is low so it malfunctions? Would we say these things forced the AI to do something it wasn’t programmed to do?

Excellent answer.  I want to pose the same question I posed on another answer: If a learning algorithm develops novel goals, does that demonstrate volition/intentionality, analogous to the free will we humans perceive ourselves as having?

@DukeZhou: I guess that depends on how you define the terms "goal", "volition" and "intentionality". Otherwise it's just a question about semantics.

The free will in the spiritual world would be for you to have the right to follow or not the main path (God's way). Taking into consideration that whatever path you follow, you will have its consquences (good or bad).
However, an artificial intelligence created to solve problems does not need these distractions. Determinism considers the feeling of freedom a subjective illusion. Trying to apply free will to an AI is to create an inner inner space within the "mind" .. but creating that is already impossible because we can not understand exactly what is going on in our minds. The big question is, how to program something that we can not explain accurately? How to induce an AI to learn something that does not make sense?
I believe that it is not just code and database that will lead the AI to have faith in its existence or to believe in free will. But to construct an AI that reads and understands every human mind, every thought, house illusion, every paranoia, every confusion, every sadness, every lie ... may be a good source of studies, but whoever qualifies for this kind of experiment? What would be the conclusion of an AI by understanding the whole confused and troubled human mind?
It's a very complex question ... let's continue studying and raising this kind of subject. I spent my last years trying to recreate human thoughts and actions in an AI .. it's a study for a lifetime .. my fear is the disappointment at the end of everything :(

You bring up some interesting questions. If an AI is programmed to fear, and it acquires a fear of death- or being shut off- how would it react? I don’t think AI can have human thoughts unless it’s lived a human life. I don’t remember being potty-trained, but it’s something that drastically changed my life. And even after the fact that billions of people have come and gone, the outcomes are still endless.

@Cannabijoy it would react in the way it was told to, or told to learn to, nothing more, nothing else.

Love question. Firstly, as pointed out in other answers Narrow AI today are mostly algorithms following their procedures given some inputs. No need for philosophy here as they are following reproducible steps.
However, if your referring to general AI or AI a kin to human level intelligence or better then maybe the question holds some weight. But again as pointed out it would come back to whether you believe you or I indeed have free will?
For me I believe free will can be modelled as a sort of entropy. If you look on the macro level things are blurry, agents are making decisions and moving around in a unpredicatable way. On the micro level however, given all the data in one state one could predict the next state shattering the idea of free will. I guess it’s up to you to decide whether this fits with your free wil definition or not.

No, because there is no utility in building a "libertarian free AI" as far as I know of.
AI is another tool. What is the purpose in building an AI with such a distinction?
The reason for that question is this. Let's say you want an AI to accomplish some kind of task you want machine assistance in. That's what tools do- assisting with tasks. What exactly would this task be that a "non-libertarian free AI" couldn't achieve but a "libertarian free AI" could?

You guys are missing the point completely.
Everything I read assumes that being is based on science.  You are trying to fit the reality of being into a brain, that is, into a machine. You know that you exist. A billion sensors of every imaginable type cannot equate to reality. Even with a billion more neurons than what's in the brain.
It's like the mathematics you use. You believe you can be accurate when the truth is, you cannot be. The position between 2 numbers can never be accurate, they are infinite.
It's like time. You believe it to be organic when the truth is, there is only the now.
Try as you may, you will never fit being into an algorithm. The real illusion is that you believe you can.

I was very bored.
The short answer is yes and the long answer is basically yes.
I'm going to gloss over what you could possibly mean by AI and instead focus on what AI programs pretty much are:
A mish-mash of algorithms and methods borrowed from mathematics, perhaps more specifically statistics, inserted into a good old-fashioned program.
Let's start from the very beginning. The very beginning.
For the sake of this argument, everything is a language and there exists an omnipotent alphabet which contains all possible symbols you can come up with to convey a message. Such an alphabet would contain everything you've ever known, all their permutations, and then some. It would contain your clothes as well. Forget countable infinity or all those concepts, those come way later.
It is important to realise that right now I'm communicating with you through not just through the power of a string of characters. You're also experiencing other stimuli that you, or whatever part of "you", is/are capable of interpreting into another language. To be crude, everything you decide to classify as a 'standalone thing' is a compiler that runs in perpetual execution, translating all the 'things' it 'receives' into other 'things' it 'spits out' for other 'things' to 'receive'.
Think about a modern day computer. You write a program in your fancy little language. You hit compile. A compiler goes through your code, and spits out more code. Except this time this code is written in another language, sometimes "closer to the metal", sometimes "just about the same abstract level" and sometimes "even more abstract", and this process repeats itself until somewhere along the line that code you started with, has been interpreted to mere electrical signals, which then themselves are being 'compiled' by an entity we will call 'the universe' and that's up to empirical observation to determine what and what's not going on. (Except "the universe" was always responsible. But we will partition things for the sake of ..partitioning things. You get what I mean, I know you do.)
Now let's jump back to languages. Mathematics is a language in the sense that:

It is built of symbols contained in an alphabet we will call X
The several fields specify their own grammatical structure through which we can decide whether or not a statement is well-formed. This encompasses everything from where you can put the + operator in high school algebra, to how you can write a proof in formal logic. There needs to be no justification for how you build a grammar. You can always just make a new grammar and use it instead. Of course, it might not be capable of forming statements which are compatible with other grammars.

What's interesting about X is that its definition is not fixed. Throughout time, we've introduced new symbols into the mathematical alphabet to be able to express more concepts while keeping things separate. (Or rather some people have had the sense to keep it this way.) For example, whenever you see Leibniz's integration symbol, you know you're probably dealing with some kind of integration and not something novel that you've never heard of before.
Now here's where I actually answer your question:

I assume that by "program" you are referring to the mathematical construct as defined in theoretical computer science: A string of characters from an alphabet.
This string is then fed to a compiler (lexer|parser|semantic analyser) which spits out another string (mainly the job of the semantic analyser). This string usually is built from characters of a different alphabet. That is to say, the compiler is a function which maps a well formed string of language A to a string of language B
The end goal of compiling a program is to execute it, which basically means a succession of compilers will take the output of the previous compiler and spit back yet another string, until the string is essentially electrons moving about the circuitry of the computer in your bedroom and producing fancy lights on your monitor

So whenever you write an "AI program", you're just writing a "program" that contains some "AI algorithms" which are really just applications of things we've known in mathematics for 100s of years, which again, are really just a string of characters that are about to be translated by a compiler.
In other words, nothing you can ever write is not deterministic, provided you look at the bigger picture.
A common argument I see is that since AI programs usually "adapt" and "self-optimise" when solving the problem, they're not quite deterministic, in the sense that feeding the program the same input twice will (hopefully) yield better results the second time.
Except what really happened is that you had an input string that you partitioned into inputs A and B, and fed them to the algorithm in succession. Had you fed AB initially, you would've obtained the same results.

"there's no such thing as true random numbers anyway." that's all you really need to deter any idea of AI on any computer. Any software or set of functions on a computer is pretty much (right now at least) all set code, by humans.
Also the execution of actions based on variables not listed is not artificial intelligence, it's just simpler code.
Any REAL Artificial Intelligence will not be made on a board of 1's and 0's, that's defeating the purpose, all of the actions are predetermined (even if they are extended intricately to cover many possibilities) so they have no chance to create something non-deterministic.
Real independent intellect is most likely (in my eyes) found, not made.

AI is algorithmic not free willed in a sense that humans have free will.  So in that sense it is deterministic.  Give it the same data each time you would expect the same result.  Change something (ie feed it new data to learn from) and then it will give a different result.  Hence the determinism.
EDIT: Some algorithms do use some randomising - ie some versions of hill climbing - but if we want to get technical there's no such thing as true random numbers anyway.  (Unless you're using one of those supercomputers that use the radiation from the sun as a seeding factor)

In what sense do humans have "free will" as you use it in this answer? For instance, are you referring to Cartesian Dualism? You appear to be equating free will with "true random numbers" - could you explain why? Your assertion about using radiation from the sun is odd - what makes that "true random" and, say quantum noise as used in commercial TRNG chips not "true random".

I agree with Neil. This answers just asserts that we have freewill, but doesn’t explain how. If a human thinks berries are food (initial data), and is about to eat a poisonous berry, but another human tells them it’s poisonous (introducing new data) so that they do not eat, how does that equate to freewill?

Free will as a philosophical debate has been ongoing for centuries, probably beyond the scope of the comments. Think of it from a simplistic view, I could have chosen to respond to these comments or I could have ignored them. My free will. Based on random factors (how tired I am right now, how busy I am, wether I'm in the mood to respond nicely or not -which itself can rely on thousands of factors including the weather- etc) I exercised a certain degree of free will.  You can't replicate that to the same extent.

As for random numbers have a look at random.org which uses radiation from the sun and atmospheric noise to seed true (to an extent) random numbers.  otherwise a computer can not create true random numbers due to hardware and mathematical limitations.

I think what Neil means about randomness is that committing a random act does not equate to a “free will”. If a robot starts to randomly wave it’s arm in the air, and cannot provide a reason for this bizarre behavior, then we couldn’t really say it freely wanted to wave its arms. Why did you choose to respond to these comments? Did you have any reason to? Did you have a reason not to?

Of course I had a reason. My thoughts flowed in a way to convict me that I was right and what I was presented with as an alternative fact was wrong. As having my brain wired (or my neurons trained) in a stubborn manner I automatically responded.  So this would be an argument against free will. This behavior can be replicated in a machine. With this world view there is no such thing as love, fear, nostalgia etc.  It is I guess very nihilistic.

It doesn’t have to nihilistic, and I don’t really see how having a free will gives life any more meaning. If we have a purpose to live, love, etc., then there was a reason- and our life and love will create their own causes and effects that will permeate the world. En archē ēn ho logos...

Or we could easily say life is a pursuit to feed dopamine, seratonin and endorphines to a 2 pound chunk of meat inside our skulls. En arche en ho science.

@solarflare I tend to couch it as "we (humans) appear to have free will."  (imo, I'm not sure how the appearance of and actuality are materially different in that validating or disproving free will wouldn't seem to change anything;)

@DukeZhou I agree. From our perspective we have free will.  Wether we actually have it or not are somewhat irrelevant to the actual experience.

@solarflare As a side note, true random numbers are not hard to generate, e.g. using single photon sources. You don't need anything fancy for this, just basic quantum mechanics.


text:
Why do we use $D(x \mid y)$ and not $D(x,y)$ in conditional generative adversarial networks?
In conditional generative adversarial networks (GAN), the objective function (of a two-player minimax game) would be
$$\min _{G} \max _{D} V(D, G)=\mathbb{E}_{\boldsymbol{x} \sim p_{\text {data }}(\boldsymbol{x})}[\log D(\boldsymbol{x} | \boldsymbol{y})]+\mathbb{E}_{\boldsymbol{z} \sim p_{\boldsymbol{z}}(\boldsymbol{z})}[\log (1-D(G(\boldsymbol{z} | \boldsymbol{y})))]$$
The discriminator and generator both take $y$, the auxiliary information.
I am confused as to what will be the difference by using $\log D(x,y)$ and $\log(1-D(G(z,y))$, as $y$ goes in input to $D$ and $G$ in addition to $x$ and $z$?

It looks like you're asking about the difference between using conditional and joint probabilities.
The joint probability $$D(x,y)$$ is the probability of x and y both happening together.
The conditional probability $$D(x | y)$$ is the probability that x happens, given that y has already happened. So, $$D(x,y) = D(y) * D(x | y)$$.
Notice that, in a C-GAN, we have some extra information that is given, like a class label $y$. We actually don't care at all about how likely that information is to appear. We care only about how likely it is to appear with a given $x$ from the source distribution, versus how likely it is to appear with a given $z$ from the generated distribution.
If you tried to minimize the joint probabilities, you would be attempting to change something that the networks have no ability to control (the chance of $y$ appearing).

So if some information say y (which is given and not being modeled), is given to discriminator only. Then for each generated and real frame, can we say it , D(x/y) and D(G(z)/y)?

Careful, the bar | is not the same as the slash / (use shift + \ to get the bar), and has a very different meaning. In that scenario, then we'd model it with D(x|y) and D(G(z) | y), but that's a very strange situation to imagine. If x is not conditionally independent of y, then it is very unlikely that your generator will produce challenging examples, because it cannot produce examples that vary with y, but the discriminator observes y for every sample.


text:
Do convolutional neural networks also have recurrent connections?
I asked my self this simple question while reading "Comment Abuse Classification with Deep Learning" by Chu and Jue. Indeed, they say at the end of the that

It is clear that RNNs, specifically LSTMs, and CNNs are state-of-the-art  architectures for sentiment analysis

To my mind CNNs were only neurons arranged so that they correspond to overlapping regions when paving the input field. It wasn't that recurrent at all.

You are right. I think you are just misinterpreting the part of the sentence ('specifically LSTMs'). LSTMs are an example of a popular type of RNN.
RNNs and CNNs are different architectures but they can be used together.
Here is another sentence with the same structure:

It is clear than dogs, specifically corgis, and cats are very common in online memes.

Both CNN and RNN fall into the super set of neural networks,however applications of the two matters.
So to branch them off in terms of applications,

I would say CNN’s are mainly used for vision related applications,
  whereas, RNN’s are mainly used for language processing applications.

You can refer to these links for further details.
Comparative Study of CNN and RNN for Natural Language Processing

How are recurrent neural networks different from convolutional neural networks?
The unreasonable effectiveness of Recurrent Neural Networks
Hope this can give you a glimpse!

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Why do we need 10 bits to represent the 1000 classes in AlexNet?
I'm reading the AlexNet paper. In section 4, where the authors explain how they prevent overfitting, they mention

Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label".

What does this mean?

You need 10-bits ($2^{10} = 1024$) to represent 1000 classes.

It takes at least 10 bits to represent any number between $1-1000$ because $2^{10} = 1024$. This means that if one was trying to represent 1 of the 1000 classes, one would need at least 10 bits. However, having these 10 bits set correctly for each input is really hard and would require overfitting to ensure it.