text:
Does humanism grant moral consideration to sentient artificial general intelligences?
Some formulations of humanism already grant moral consideration to sentient non-human animals (e.g. https://humanists.international/what-is-humanism/). Does humanism also extend to granting rights to AGIs, should they become sentient?
Sentientism is a closely related philosophy that makes this explicit https://en.wikipedia.org/wiki/Sentientism.

There is no product here. It's a genuine question. The FB group is just a discussion forum. I'm new to SE so apologies if I've framed this wrong.

Thanks @DuttaA.  I've tightened the question and removed the links to my other article + the community web site.  Hope this is better.

I like this question and will answer if I have anything to add to Neil's.  (My sense is there is a deeper question of whether humanity, here in the general sense of Ren 仁, is an exclusive characteristic of homo sapiens.  Evolutionary game theory suggests that it may not be.  PKD explores this in a Christian context of empathy.)

Does humanism also extend to granting rights to AGIs, should they become sentient?

I think that is going to be something that will divide self-identifying humanists into a few different groups, based on understanding about what AGI is.
The front page you link includes "aiming toward the welfare and fulfillment of living things", which does not explicitly include non-living sentient things. However, the humanist philosophy naturally extends to granting rights and protections based on reasoning and scientific evidence to anything that can be argued as needing them.
The big problems are:

Recognising and measuring sentience.
Attaching moral value to events that happen to artificial entities, where subjective measures such as pain, stress, happiness, fulfilment might not be relevant.

These measurement problems already cause a lot of variability in moral stances towards farming animals for instance. These biological creatures share enough with humans that even though we don't fully understand the nature of their subjective experience, it is possible to extrapolate with some backing from scientific studies. For example, we can measure whether a creature suffers stress or pain and reacts similarly to a human, and even if we don't understand what that means subjectively (what it feels like to be such an animal), a combination of Occam's Razor and Precautionary Principle can be argued:

Occam's Razor the simplest interpretation is that if an animal is based on same cellular structure, has same brain regions for experiencing pain and reward, uses same stress hormones etc, that its subjective experience is likely somehow comparable to a humans. This allows us to apply empathy to an animal's situation with a guess that doing so is not 100% inaccurate.
Precautionary Principle the consequences for inhumanely treating a sentient creature are morally worse than the waste of humanely treating a non-sentient creature (provided doing the latter is not causing undue suffering to some other thing). So faced with a lack of precise knowledge, there is still a strong rational ethics argument for extending rights and humane treatment if there is any uncertainty.

When it comes to artificially created entities, we lose the first argument of Occam's Razor, and can only get it back with a better understanding of what sentience actually is, and which parts of it we should value. We need a much firmer theory here than with animals, because there is so little shared between our construction and that of a human-built machine. In addition, just as there are many different kinds of organism which exhibit different amounts of apparent intelligence and possible sentience, there will be different versions of AGI, probably starting with purpose-built non-narrow AIs that combine features that are practically useful and make them seem intelligent (parsing human speech and responding appropriately, which Google Home, Alexa, Siri already do to some depth) with some common-sense understanding of broad task portfolios. These things are near future, and most developers who work on them would not consider them sentient - however, they will be designed to act as if they were, because that makes them practical and usable, which in turn will make it much harder to argue Occam's Razor due to similarity in behaviour.
We still keep the second argument, but it is confounded by not understanding what the analogs are for humane and inhumane when dealing with artificial entities. It is unlikely that anyone would have a problem with switching off a running instance of the next 2 or 3 generations of Siri assistants - but at what point does that become deliberate incapacitation of a sentient being, and does it even matter if the system is designed to be switched on and off, and expresses no particular desire to be in either state?
In summary, the humanist philosophy could extend to cover the welfare of artificial general intelligences, but it is far from clear whether it can go there in practice. Until we have more complete theoretical models of AGIs, or practical working devices, then it is premature to consider it. Most debate is going to be based on imagined qualities of AGI, and any two people discussing this are likely to be starting from different assumptions about what those are.


text:
Where can I find people solving this smoothing, filtering, temporal learning problem?
Consider a prediction problem for example. This is a loss function (negative log likelihood) that I am roughly talking about:
\begin{align*}
    J_{\text{train}} &= -\sum_t \log L\left(\theta_t, X_t, Y_t\right) \\
    J_{\text{test}} &= -\sum_t \log L\left(\theta_t, X_{t+1}, Y_{t+1}\right) \\
    J_{\text{dyn}} &= - \sum_t \frac{\left(\theta_{t+1} - \text{stop_gradient}({\theta_t})\right)^2}{2 \sigma_{\theta}^2} \\
    \theta_0 &\sim \Psi
\end{align*}
Here, the dynamics are simply first-order smoothness (Brownian motion) in the time-dependent parameters $\text{d}\theta_t = \sigma_{\theta} \text{d} W_t$.
This is basically what they do in weather systems and in papers like A Bayesian Approach to Data Assimilation.
The main difference is, we now have the ability to use stop gradients in the smoothness, which changes the problem somewhat. Other issues might arise.
Who else is doing this? Feel free to also provide links to papers on related work. I am not finding enough hits, so maybe I am missing something. Are RNNs, if twisted slightly, falling into this framework and I am no seeing it?

Please, provide more context about your initial equations. Where did you find them and in which context?

I created them. There is literally nothing else needed for the formulation but the context is, for example, minimize the negative log likelihood where you have a model for observations. The dynamics log likelihood is a model for the dynamics.
This is typical in data assimilation (see the paper above) but I have not found people using stop_gradient style constructions.

@nbro Not sure if that update went through.

I will maybe look at your question later. Meanwhile, you could start a bounty to attract attention, but you will lose reputation!

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
Can Reinforcement Learning solve problems, where certain elements in the environement are randomly located?
I want to solve a problem using Reinforcement Learning on a 20x20 board. An agent (a mouse) has to get the highest possible rewards as fast as possible by collecting cheese, which there are 10 in total. The agent has a fixed amount of time for solving this problem, namely 500 steps per game. The problem is, the cheeses will be randomly assigned to the fields on the board, the agent knows however, where the cheeses is located.
Is there any way how this could be solved using only Reinforcement Learning (and not training for an absurd amount of time)? Or is the only way to solve this problem to use algorithms like the A*-algorithm?
I've tried many different (deep)-q-learning models, but it always failed miserably.
Edit: I could not get any meaningful behavior after 6 hours of learning while using a GTX 950M. Maybe my implementation was off, but i don't think so.

Could you clarify what "an absurd amount of time" is for you? Do you have a GPU for accelerating training?

I don't think you will learn anything optimal in 6 hours on GTX 950M especially with millions of possible boards being generated one after another in each episode. I don't know if you would even learn optimal behavior on a single stationary board that doesn't change after every episode in that amount of time but that's just my opinion, maybe I'm wrong. I would recommend you to first try to solve a single fixed board and see how much time it takes.

Yes you can use RL for this. The trick is to include the location of the cheese as part of the state description. So as well as up to 400 states for the mouse location, you have (very roughly) $400^{10}$ possible cheese locations, meaning you have $400^{11}$ states in total.
So you are going to want some function approximation if you want to use RL - you would probably train using a convolutional neural network, with an "image" of the board including mouse and cheese positions, plus DQN to select actions.
Viewed like this, a game where the mouse tries to get the cheese in minimal time seems on the surface a lot simpler than many titles in the Atari game environments, which DQN has been shown to solve well for many games.
I would probably use two image channels - one for mouse location and one for cheese location. A third channel perhaps for walls/obstacles if there are any.

Or is the only way to solve this problem to use algorithms like the A*-algorithm?

A* plus some kind of sequence optimisation like a travelling salesman problem (TSP) solver would probably be optimal if you have been presented the problem and asked to solve it any way you want. With only 11 locations to resolve - mouse start plus 10 cheese locations - then you can brute force the movement combinations in a few seconds on a modern CPU, so that part may not be particularly exciting (whilst TSP solvers can get more involved and interesting).
The interesting thing about RL is how it will solve the problem. RL is a learning algorithm - the purpose of implementing it is to see what it takes for the machine to gain knowledge towards a solution. Whilst A* and combinatorial optimisers are where you have knowledge of how to solve the problem and do so as optimally as possible based on a higher level analysis. The chances are high that an A*/optimiser solution would be more robust, quicker to code, and quicker to run than a RL solution.
There is nothing inherently wrong with either approach, if all you want to do is solve the problem at hand. It depends on your goals for why you are bothering with the problem in the first place.
You could even combine A* and RL if you really wanted to. A* to find the paths, then RL to decide best sequence using the paths as part of the input to the CNN. The A* analysis of routes would likely help the RL stage a lot - add them as one or more additional channels.

But isn't having such a high state space not computationally really expensive and wouldn't it have to learn for a really long time? What i have tried so far is using a Feed-Forward-Network and encoding the fields with cheese as '1' and with no cheese as '0' and strenghtening the board so it becomes a list. I did the same thing with the agent, the mouse. This means i had inputted 800 numbers into my network and this did not work. Would using a convolutional neural network be much better than my approach?

@Pascal: That is your state space for this problem whether you like it or not. Yes, it may take some training time - I would guess at a couple of hours on a GPU-accelerated machine (whilst this will take a few seconds for dedicated solution such as A*). CNNs will probably work better because they allow for the "space like" nature of the board.

A assume here OP is familiar with DQN basics.
"Standard" way to solve this problem with Deep RL would be using convolutional DQN.
Make net from 2 to 4 convolutional layers with 1-2 fully connected on top.
The trick here is how you output Q-values. Input is board with cheese, without information about mouse. Instead net should output Q for every action from every field on the board (every possible position of mouse), that mean you output 20x20x(number_of_moves from the field) Q values. That would make net quite big, but that is most reliable way for DQN. For each move form the replay buffer only one Q value updated (gradient produced) with Time Difference equation
Because only one value form 20x20x(number_of_moves) updated per sample you need quite big replay buffer and a lot of training. After each episod (complete game) cheese should be randomly redistributed. Episodes should be mixed up in replay buffer, training on 1 episod is a big No.
Hopefully that should at least give you direction in which do research/development. Warning: DQN is slow to train, and with such big action space (20 x 20 x number_of_moves) could require million or tens of millions of even more moves.
Alternatively, if you don't want such big action space, is to use actor-critic architecture (or policy gradient, actor-critic  is a kind of policy gradient). Actor-critic network have small action space, with only number_of_moves outputs. On the down size complexity of method is much higher, and behavior could be difficult to predict or analyze. However if action space is too big it could be preferable solution. Practical issues and implementations of actor-critic is too huge area to go in depth here.
Edit: There is another way with lesser action space for DQN, but somehow less reliable and possibly more slow: shift the board in such way that mouse is in the center of the board and pad invalid parts with zero (size of the new board should be x2). In that case only number_of_moves should be in output.

Are you suggesting that the size of the output layer should be number_of_positions * number_of_actions?

Yes, that is standard practice for DQN. More exactly it's number_of_potential_actions.

but in the problem described (2d grid world) the number_of_potential_actions at any step is 4.

I mean all possible action, with all possible position of mouse.

so if the output layer is 20x20x4 values wide, how does the agent select which action to take?

according to current position of mouse.

@DrMcCleod This answer suggests an action representation similar to AlphaZero, which includes lots of non-valid moves and you need to filter. I would instead suggest something more like DQN as the problem feels intuitively more like the DQN Atari setup than like go or chess. For clarity - that means the actions are just the 4 (or 8 if you allow diagonal moves) movement directions you allow for the mouse. However, both representations are valid, used in highly successful systems, and neither mirror2image nor I has done the experiment on your test environment.

@NeilSlater I was not the OP, but thanks for the clarification.

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
Which model to use when selecting objects of interest?
I have a set of polygons for each image. Those polygons consist of four $x$ and $y$ coordinates. For each image, I need to extract the ones of interest. This could be formulated as an Image Segmentation task where, for example I want to extract the objects of interest, here: cars.

But since I already get the polygons through a different part of my pipeline I would like to create a simpler machine learning model. The input will not be the image but only the coordinates of the polygons.
In this model each sample should consist of multiple polygons (those can vary in number) and the model should output the ones of interest.
In my mind, I formulated the problem as follows:

The polygons are the features. Problem: Samples will have varying number of features.
The output will consist of the indices of the "features" (polygons) I am interested in.

First, I created a decision tree and classified each coordinate as $0$ (not interested in) or $1$ (of interest). But, by doing this, I don't consider the other coordinates that belong to the image. The information of the surrounding is lost.
Does someone have an idea of how to model this problem without using Image Segmentation?

Can you please explain in more detail, how the coordinates are selected? what makes them interesting and how is the relation between the coordinates and the sample? Does the "interestingness" of a coordinate in any way depend on its assignment to any sample? Supervised learning generally means, you can train your model with a set of data for which the decision is known. So you have such a labelled data set, right?


text:
Extract personal information about a person from a list of documents and summarize it
I need to extract personal information about a person from a list of documents and summarize it to the user. If there are 2 people with the same name, the correct person should be identified. If the person has a nickname, that also needs to be identified. The input to the program can be the name of the person, address, organization name etc.
I have extracted named entities like person, org, location etc from the text using NLTK library.
The output after extracting the named entities is mentioned below,
[('Michael', 'NNP', 'B-PERSON'), ('Joseph', 'NNP', 'B-PERSON'), ('Jackson', 'NNP', 'I-PERSON'), ('was', 'VBD', 'O'), ('born', 'VBN', 'O'), ('in', 'IN', 'O'), ('Gary', 'NNP', 'B-GPE'), (',', ',', 'O'), ('Indiana', 'NNP', 'B-GPE')....
Now, I want to extract relationships between those entities.

Hello Amey. You don't ask a question here, just state a project goal. No-one here is going to do your whole project for you, so please make it clear what part of your project you want to ask about. Also please explain what relevant knowledge you already have so that people don't waste their time writing up descriptions that you do not need.

hi @NeilSlater , I have edited my question.

https://www.nltk.org/book/ch07.html has information on your query. Please start and go all the way to "6   Relation Extraction". Hope atleast this points you in the right direction.


text:
How to chose dense layer size?
I am fine-tuning a VGG16 model on 20 classes with 500k images I was wondering how do you chose the size of the dense layer (the one before the prediction layer which has a size 20). I would prefer not to do a grid search seeing how long it take to train my model.
Also how many Dense layer should I put after my global average pooling ?
base_model = keras.applications.VGG16(weights='imagenet', include_top=False)

  x = base_model.output
  x = GlobalAveragePooling2D()(x)
  x = Dense(???, activation='relu')(x)
  x = Dropout(0.5, name='drop_fc1')(x)
  prediction_layer = Dense(class_number, activation='softmax')(x)

I haven't see particular rules about how its done, are there any ?
Is it link with the size of the convolution layer ?

It's depend more on number of classes. For 20 classes 2 layers 512 should be more then enough. If you want to experiment you can try also  2 x 256 and 2 x 1024. Less then 256 may work too, but you may underutilize power of previous conv layers.

Is it based on your own experience something else ? and why would 2 layers of 512  be better than for example a layer of 4096 ?

Just common practice. Historically 2 dense layers put on top of VGG/Inception. It works, so everyone use it. Intuition behind 2 layers instead of 1 bigger is that it provide more nonlinearity. But it's not proven. Also the number of weights would be quite different. It's different balance between weights and input/output pipe size

okay thanks :) I was wondering because I saw that a lot but didn't see anything written about

I am also wondering about this.
It must depend both on convolutional sub-network output size (N) and number of classes (M).
Maybe there are some rules of thumbs depending on (N, M).

Why 2 dense layers and not, say, 3 or 4 ?
Is it better to have all dense layers (except last) the same size ? or decreasing ? or increasing ? or pyramidal ?
Is it better to have small dense layers or larger ones with dropout between layers ?

And bonus question:

Should we use batch normalization between dense layers ?


text:
I am looking for research related to the use of AI and ML in automotive and aeronautics safety design
I am specifically interested in the topic of edge cases.
I have the presentation Edge Cases and Autonomous Vehicle Safety as a starting point, in particular on page 6:

Machine Learning (inductive training)

No design insight

Generally inscrutable; prone to gaming and brittleness.

I'd like to find more hard data on how ML may do very well until an edge case is encountered.

Hi and welcome to this community OneArb! Are you asking for research papers or what or are you looking for suggestions for topics to research?

@nbro see my update

Can you be more explicit? I don't think that your update makes this clearer.

Hope I covered it

Now, I don't see the relationship between "I'd like to find more hard data on how ML may do very well until an edge case is encountered." and "I am looking for research related to the use of AI and ML in car, aeronautics manufacturing design and safety". Try to be a bit clearer.

See page 15: "Edge cases are the stuff you didn’t think of!" Safety will be excellent as long as the software operates within the cases covered by the training data set, very poorly otherwise.
I am looking for any study quantifying edge cases. In other words software engineers attempt to account for all possible circumstances but fail at anticipating extraordinary events. Within those edge cases ML may result in an undefined behavior, for instance accelerating where breaking was the optimal solution.

So, you should want to read about the topic, so you want just links to papers or articles that discuss these issues, right?

I am looking for any insight. I see the matter under consideration by car and areonautics manufacturers. There are hearsay accounts that ML designed systems being overall safer all together failing spectacularly outside of the "safe operation state space" where traditional software would have performed better.


text:
How to translate algorithm from logic to equation, and back?
I just recently got into machine learning, and have been hitting a lot of obstacles understanding the algorithms involved in the programmings. My issue isnt with the programming, but how they're translated from math to code. ML is popular with python, and that's okay, but i dont like python, and i dont want to have to learn it to be able to use the programming language of my choice, to do the exact same thing but in a way i feel comfortable (i dont care if python is popular for math majors, because it's easier for them to understand -- it isnt for me, when nothing being done is explained thoroughly.).
I'm trying to decipher this model

this is the breakdown for the algorithm model

this is the math i was able to decipher for this particular model

(Left is terminology and their usage, the middle in black was something to do with programming arrays... below it is the equation used in bottom left, but more elaborate, and underneath that is a image that says the same thing the algorithm is doing.. because sometimes pictures are easier to understand that words VectorArray(Value) * VectorArray(Weight) + SingleUnit(Bias) = Neuron(Node))
But then everything stops at the middle layer of the second image. How do i get the full output to give me a yes or no response?
How do i enter in the variables and tables to go thru the math steps?
Is my understanding correct, or am i lacking somewhere?
This user is also sharing the same algorithm but our math dont look the same
How do i go from what i have, to what [s]he has?
At the end of all of these questions, i'm going to write everything into a programming script, that'll use a different language from python (and i would need to manually create resources from scratch, because no one else thinks machine learning should be done in other languages -- it seems...). I want to be able to understand the process itself, without just doing cookie-cutter actions (tools made by users for those too lazy to do the work -- which circumvent the learning/understanding process of what's going on behind scenes).

could you please rewrite to make it clearer, I think you can remove some information that I think isn't relevant to the main question (like your opinion with python) and explain each picture you put in there

i dont know how to make it clearer... my opinion was added because i stated i wanted to use programming as end goal but could not find any resources that applied to what it is i wanted to do. which was not use python cause it wasnt convenient enough for me to follow along with. If popularity dictates the results i query, then i have to specify i dont want "popular" results, and that i'm seeking something against the norm... All i want to do is understand the math better, so that i can write a code that does the same thing. But if i dont understand it, i cant explain it so that others can...

Are you trying to understand the maths for a normal feed-forward neural network used as a binary classifier? That's what it looks like, but you barely mention those terms, and you are asking a very generic (impossible to answer) question in your title. "How to translate algorithm from logic to equation, and back?" is impossible to answer. However, if you gave a specific part of the maths or the code for a neural network, many people on this site could easily convert between the two for any specific part.

I dont understand any of those terminologies, i tried to state that i am new to the technology, and that research into the subject is difficult because all resources around the subject is cluttered around python programming. If i google "Machine Learning" i get nothing but python results. If i google "Machine Learning -python" then i dont get anything that teaches me the math/concept. I went with what i was able to understand, if you can help fill in any gaps, i would be very much appreciative. It's something i would like to learn, but if i cant, then i just wont. :/

OK, your first goal then is to narrow down what you want to understand and explain it in the question in ways that answerers have a clue on what you are asking about. Otherwise this question will go nowhere. There are 100s of ML algorithms, and many variations of neural networks. You need to understand what you want to do at a high level - at least the name of it, before anyone can even know what you are trying to achieve

So, just to clarify, validating if the math is good or not will not be a good question to ask if i'm unable to specify the algorithm it came from? what about the linked stack question, on getting my math to look like theirs, is that also out of the scope? There are a lot of greek symbols floating around in ML math that arent clearly explained, and i'm not entirely sure how or where to fill in those gaps either. Only reason i understood sigmoid was because i deconstructed the formula above. But i got stuck when a python script i was rewriting stopped making sense to me... i wanted to keep going

If your math was presented more formally, it could maybe be validated of its own accord. But the photo of your workings on a whiteboard here is not really maths . . . it is just a bunch of equations and diagrams without much context (I could not verify that it matched the code you were looking at for instance, and it doesn't match the vector quantization diagram, but I suspect you have just accidentally chosen the wrong diagram here). For what it is worth, your workings do look like a correct interpretation of how a single NN layer feeds forward in order to make a prediction.

I suggest you drop the images, give relevant parts of the python script - including both the part you have interpreted already and the part that you are stuck on. Please use LaTex to show your interpretation as an equation, not photos of whiteboards - you can embed LaTex using $ $ sign e.g. $y_i = \sigma(W_{i,1}x_1 + W_{i,2}x_2 ... + b_i))$ becomes $y_i = \sigma(W_{i,1}x_1 + W_{i,2}x_2 ... + b_i))$

Actually I can see what I think is an error in your maths. However, not sure if just a typo or if I am misunderstanding it without the context that I am asking for

Thank you, will take some time to reevaluate the code and question, to adjust for feedback provided.


text:
What are the current open source text-to-audio libraries?
I am not new to AI and did some work for few months but completely new to text to audio. Yes I used text to audio tools a decade back... but I would like to know where exactly we stand in terms of text to audio today.
I already did some research, it seems like traditional way of text to audio is fading away and speech cloning seems to be emerging but my impression on this might be completely wrong.
What are the current open source text-to-audio libraries?

voice-builder is an opensource text-to-speech (TTS) voice building tool from Google.

There is one by Mozilla called Deep Voice
And another python library called pocket-sphinx

authors: Анастасия Бирсак


text:
Is there any GUI for per-neuron editing
I couldn't find GUI for precise "artificial neural-network alike" structures, which could supports neuron naming, synapse naming, import of external functions or code fragments and debugging. It would be ideal if synapses also could pass not only float values, but user-defined structures too. Optimization, GPU-computing is irrelevant (and probably impossible with such features).
Is such thing exists? I'm thinking about writing one myself, for my needs. And most probably I will...
Why do I need such features? For testing concept of construction of kind-of-neural logic programming. I'm sure there is also should exists programming languages with such paradigm, but I also don't know how to find them, google would mostly give me info about common artificial neural networks.

Yes, and besides showing weights, I mostly need to be able to see textual tags for neurons and their connections, not just "1, 2, 3 ... etc", but actual user defined names.

Well, kinda. I'm using C#+WPF with MVVM pattern and I've selected appropriate open source projects, currently I'm modelling data with UI. Interactive UI isn't a problem (at least in selected environment), but I want my project to support both UI+code editing mode (i.e. possibility to create and edit networks either by mouse or keyboard, depending on which way is more comfortable).

Also, I've found that this paradigm is called dataflow oriented programming. And I've found that most languages using this paradigm is mostly visual languages, which I find quite inconvenient (because it's much more convenient to write purely textual code for development and use visual for debugging and minor fixing).


text:
Is having near-duplicates in a training dataset a bad thing?
I am making a labeled dataset of images from web streams for a CNN classification. Pictures from the same stream are quite similar as far as background, but slightly different as far as the main object. The focus of what should be learned is in the main object.
My concern is that feeding similar images with the same features in the background will result into making those features more relevant and hence lower the weights of the features that matter.
So, should I be worried about removing similar images from a dataset, so that unrelated features are not learned?
An ideal answer would discuss the trade-offs.
I am aware of the practice of augmenting the training images by scaling/skewing/flipping them around. So it looks like people do it intentionally, but why?
I should also say that it's not about learning from a single stream, there are tons of them. So most images are very square-distant from one another since they are coming from different streams, except those ones that were snapped from the same stream.


text:
Are these visualisations the filters of the convolution layer or the convolved images with the filters?
There are several images related to convolutional networks on the Internet, an example of which I have given below

My question is: are these images the weights/filters of the convolution layer (the weights that are learned in the learning process), or the convolved images of the previous layer's image with the filters of the current layer?
image source:
https://stats.stackexchange.com/questions/362988/in-cnn-do-we-have-learn-kernel-values-at-every-convolution-layer

Only the first convolutional layer, with filters that process the input [colour] channels directly, can be rendered directly as image patches in the same domain as the input. The left-most panel in your example looks like that.
Further layers of the neural network cannot be rendered like this for two reasons:

They have a number of input channels based on the previous layer's output, for example they may process 32 or 128 channels. There is no simple mapping of these channels to colours.

They respond to a wider range of input stimuli than any single image patch. If you tried to render out all inputs that they respond to then you would likely get an indistinct-looking grey blob, if anything at all. This is different to the first layer which does directly react pixel-by-pixel according to the weights.

What is typically done to render patches like the middle and right-most panels in your example, is to find sample patches that trigger a strong response from that filter. This search can be done using gradient ascent - start with noise, then take gradient steps in the direction of increasing the signal for that filter. This is also the basis of "Deep Dream" images, which instead of doing that for small patches, apply it to whole images, and many filters at once.


text:
Algorithms and strategies to help judges rule cases
I'm a Rails developer with a lot of web experience, but none (still) in AI.
I'm working in a web text editor that judges use to writing their sentences.
The goal is to start to use AI to help the judge rule the case, either based on his own previous rulings, either based on his colleagues rulings.
The judge would provide the text for the plaintiffs and defendants petitions, and based on these two inputs the system would recommend previous rulings that apply for the case.
I already have a considerable dataset of judges rulings inside the database, and they can be easily attached to the plaintiffs and defendants petitions for training (so this plaintiff petition + this defendant petition = this ruling).
This is specially challenging because the complaints can contain different subjects combined into the same petition; but the fact is that many offices use the same standardized petitions, as the defendants do as well, so I think the system can have a great chance of prediction success.
What algorithms or strategies should I start studying to tackle this problem?
Any similar articles, white papers or repositories that could help in my goal?

I hope this isn't actually going to be used in production... AIs are actually used in the courthouse, and (unsurprisingly) they turn out to be extremely racist or otherwise biased, with disastrous results.

@forest the project is very proof of concept, and it's not planned to be used in criminal cases. Only in civil trials, like bank agreements disputes, with a lot of thesis repetition and very little (if not none) evidences / testimonials.

Genuine success in this area would be beyond the state-of-the-art in research, since it likely requires analogising from relational knowledge extracted from text. In recent years, techniques for working with natural language have tended to be statistical, and are therefore somewhat deficient in this respect. You could look at 'bag of words'/latent semantic analysis approaches, but they are likely to generate many false positives unless a lot of ad hoc conditions are added manually. More recent work on 'treenets' (paper here) is more structurally informed, but is still a relatively new area.


text:
Can there be applications of byzantine neural networks on quantum computers?
This question came after I connected 2 pieces of information :

I recently listened to The Byzantine Generals’ Problem, Poisoning, and Distributed Machine Learning with El Mahdi El Mhamdi (Beneficial AGI 2019).
Today, I was laughing with a friend at "quantum computing" being mentioned in a job offer.

Considering the probabilistic nature of results in quantum computers, what would be the advantages to using byzantine resistant neural networks on quantum computer? Has this already been attempted?


text:
Can the inputs and outputs of a neural network be a neural network?
Can the inputs and outputs of a neural network (NN) be a neural network (that is, neurons and connections), so that "if some NN exist, then edit any NN".
I think that by creating NNs with various inputs and outputs, interacting with each other, and optimizing them with evolution, we can create strong intelligence.

Comments are not for extended discussion; this conversation has been moved to chat.

A neural network essentially is a function:
$$\mathbf{y} = f(\mathbf{x}, \mathbf{\theta})$$
Where $\mathbf{x}$ is a vector input, $\mathbf{\theta}$ are changeable or learnable parameters, and $\mathbf{y}$ is a vector output.
There are some variations of this in practice, as you can make special arrangements of $\mathbf{x}$ and $\mathbf{y}$, or use an internal state and feedback loops to allow either or both $\mathbf{x}$, $\mathbf{y}$ to be sequences. However, the above function is basically what a neural network is; how it works beyond that summary are details that you can study.
In supervised learning, you are interested in fixed sizes/shapes of $\mathbf{x}, \mathbf{y}, \mathbf{\theta}$ and trying to find a value of $\mathbf{\theta}$ such that
$$f(\mathbf{x}, \mathbf{\theta}) \approx g(\mathbf{x})$$
where $g(\mathbf{x})$ is some "true" function that you care about, and can find or generate examples of, but typically don't fully know. The value $\mathbf{\theta}$ is called the parameters of the neural network. In addition to these parameters, there are also hyper-parameters of the neural network, which include how many neurons there are in each layer, the valid connections between them, which non-linear function is applied after summing connections between them, etc.
There are learning algorithms used to find $\mathbf{\theta}$ - the many variations of gradient descent being the most popular. Some algorithms - mainly evolutionary approaches - can also vary hyper-parameters, although a more common approach is to repeatedly find $\mathbf{\theta}$ using gradient descent, and vary hyper-parameters in different learning trials, using some metric of performance using test data to pick the best one.

Can the inputs and outputs of a neural network (NN) be a neural network (that is, neurons and connections), so that "if some NN exist, then edit any NN".

Yes - partially. This is a data representation issue. To use it as an input, you would need to express the state of a neural network as a vector - or sequence of vectors - for the input, and the output/edit would also need to be a vector. Probably the simplest way to do this would be to use one network directly output a fixed length vector for $\mathbf{\theta}$ of the target network given an existing value of $\mathbf{\theta}$. That would not allow you to change connections or layer sizes etc, but it would be a very straightforward way to express "one neural network altering another" (ignoring whether this was in any way useful for a task).
If the output was a full representation of the new network, then you would have a function that took as input the definition of one neural network, and output the definition of another network. It would be up to you to convert to/from implemented neural networks and the representations for input $\mathbf{x}$ and $\mathbf{y}$.
If the output $\mathbf{y}$ was an "edit" for a change $\mathbf{x} \rightarrow \mathbf{x}'$, then you would have to decide what edits are allowed, design the representation and write code that applied the edit (the NN would not actually make any changes to another NN by itself). There is no standard way to do this, although there are things you could base this on (such as NEAT).
The big unanswered question with both of the approaches though is what your "true function" $g(\mathbf{x})$ is supposed to be. Having a neural network that represents a neural network generator or edit function is only half of the problem. You also need a way to either generate "correct" outputs to learn from, or a way to assess outputs against a goal.
The goal cannot simply be "make a valid edit", as the number of valid edits that will do nothing useful vastly outnumbers the number of edits that have some specific purpose. This is a similar issue to the fact that there are roughly $2^{8000000}$ valid 1MB files, but only a small proportion of those will be valid image files, and a smaller proportion still will be valid images that represent a natural image that could be taken by a camera. Neural networks that generate natural images therefore must be trained using natural images as a reference, otherwise they will tend to produce meaningless static-like noise.

I think that by creating NNs with various inputs and outputs, interacting with each other, and optimizing them with evolution, we can create strong intelligence

This is very broadly compatible with an Articial Life approach to AI. Although there are two differences between what you are proposing and typical A-life approaches:

Evolutionary algorithms need some measure of fitness, in order to select the best performing individuals to take forward. A-life solves this by implementing a very open environment that makes no direct judgement on outputs of functions, but allows virtual creatures that collect enough resources (defined in the environment) to procreate.

Your suggestion contains no hint that you are thinking of any kind of measure of success or fitness for either the editing NN or target NN. You will need some measure of fitness at least for the target network (and maybe the editing network too) if you intend to use evolutionary algorithms

A-life typically does not treat the evolutionary algorithm itself (the editing or the NNs) as a learning goal. You will not see the results of a good or bad editor until many simulations have passed, so this "meta-search" is likely to be incredibly slow.

A-life simulations are typically already quite slow to reach behaviour which is interesting (because it has emerged without direction from the developer), but usually quite simple for the given environment such as predators chasing prey.

From what we know of the evolution of life, a simple feedback mechanism of RNA molecules editing other RNA molecules - the "RNA world" - is considered a likely pre-life step. This has some parallels with what you are suggesting - and this or something similar perhaps, has resulted in intelligent beings such as ourselves. So your idea perhaps has some merit in a theoretical sense. However, it took biology billions of years to go from such a basic stage to self-aware creatures, and this took the full processing power of large numbers of atoms interacting in ways that even all the computers in the world could not simulate in a fraction of real time.
To speed things up, and turn the idea into something feasible, you would need to look into viable environments and evaluations that would focus the learning towards direct measures of intelligence. Also, you would do well to compare your idea about NNs editing NNs with learning approaches that don't use such a feedback loop.
There is no theory that suggests a NN that can edit or output other NNs would offer any advantage for research into strong AI, compared to other search methods. The idea is basically "AI alchemy" - an idea of an experiment that could perhaps be done, but without any theory backing it as being better or worse than other ideas.
Personally, I would expect the search for a NN which is a good NN editor for a NN which has some other task, to be too slow to be useful when faced with very broad tasks such as exhibiting high level reasoning.


text:
Will training an AI still work if the input data is somewhat sparse?
I'm looking at writing an AI agent for pattern recognition.
I want to be able to constantly feed new data to the AI to continuously train it as new data may have new patterns.
My problem, though, is that my input feed may break once in a while (the data comes from a remote computer) and thus some of the data will go missing. The other computer sends me real-time data so when the connection goes down, any new data while disconnected goes missing as far as the AI agent is concerned. (at this point, I'm not looking at fixing the gaps, although ultimately, reducing them is one of my goals, at this point I have to pretend it's not possible to accomplish.)
What kind of impact missing data has on a pattern recognition AI?

First, the title mentions "sparse data". Recently the expression has taken a clear meaning: The agent input is data with mostly zeros. In the question a different meaning: A "sparse data stream", where data flows and vanishes sometimes. I understand the question as: "Will training an AI still work if the training data stream breaks?"
Note the explicit "training data stream": The question suggests the agent has at least 2 inputs: Training data you want to feed "anytime", and "inference data" sent to the agent for actual recognition.

This question enters (to my eye) the realm of distributed AI and multi-agent systems, and ultimately a common issue in distributed systems.
If we cast your problem to two humans S and L communicating, when S talks to L over a reliable channel, L gets all the information. When the channel breaks, L gets nothing. Does it prevent L from living normally? It merely cuts out whatever is expected out of the conversation from S to L.
Back to your scenario, whenever the data stream is broken (S), the learning agent (L) will just be unable to learn from that data source. The impact on the pattern recognition agent is bounded to what it could have learned from the new data. The agent recognition performance remains constant while the data stream is interrupted.
Now if the learning agent is just learning, and cannot perform recognition without learning, there is an architectural or implementation issue. Continuous learning entails the agent is active (performs actual recognitions) and learns out of what it does.

Update, for clarification:
The performance remains constant is "true", but subtle. At time t some metric like precision can be 99% with respect to what the agent has seen so far. Assuming continuous learning is interrupted and new recognition requests come in, the performance has "two faces":

As long as new recognition requests are "close" to what the agent has seen so far, performance is "constant"---the agent still scores 99%.
If the request is quite different, the performance will drop. The size of the drop depends on how different the input is.

A concrete example: The agent is trained to find mushrooms with a dataset where all images are taken in the forest. Assuming learning stops, when an image of mushroom on a concrete crack comes in, the agent will probably do worse. And it would then keep doing worse on such kind of image, as long as it cannot "refresh" by learning from this experience.

Yes, the agent is used against the incoming data. However, all the correct answers are not available immediately, so the teaching from the new data has a delay. Obviously, the recognition won't run for a while until the agent knows enough.Then it continues to improve as more data comes in. The delay could be around 1h, although that in itself is not too important. "the performance remaining constant" is certainly the bit I was looking for. Thank you.

@AlexisWilke As you mention that specific aspect, I have refined the answer to expand on that point.

authors: Ηλεκτρολόγος Μηχανικός


text:
Are there neural networks that accept graphs or trees as inputs?
As far I know, the RNN accepts a sequence as input and can produce as a sequence as output.
Are there neural networks that accept graphs or trees as inputs, so that to represent the relationships between the nodes of the graph or tree?

Yes, there are numerous, coming under the umbrella term Graph Neural Networks (GNN).
The most common input structures accepted by these techniques are the adjacency matrix of the graph (optionally accompanied by its node feature matrix and/or edge feature matrix, if the graph has such information).
A Comprehensive Survey on Graph Neural Networks, Wu et al (2019) divides GNN's into four subgroups:

Recurrent graph neural networks (RecGNN)
Convolutional graph neural networks (ConvGNN)
Graph autoencoders (GAE)
Spatial-temporal graph neural networks (STGNN)

ConvGNN's can themselves be classified by whether they use Spectral methods or Spatial methods, and GAE's by whether they are designed for Network embedding or Graph generation.

There are types of neural networks designed exactly for that purpose. For example, graph convolutional networks (GCN) by Thomas N. Kipf. The input to the network will be a matrix of size $N \times F$, where $N$ is the number of nodes and $F$ the number of features (for each node). You then can multiply the feature matrix with the adjacency matrix (each node is going to be a weighted sum of its first-degree neighbors). There are a lot of other variations, such as diffusion convolutional networks, gated graph neural networks, etc. There is a nice survey that describes most of the recent related work in the field Graph Neural Networks: A review of methods and applications by Jie Zhou et al.

'The input to the network will be a matrix of size NxF' of fixed size, i guess?

Yes, usually the input if fixed. Is that a problem for your setting?


text:
Machine learning with graph as input and output
In my application, I have inputs and outputs that could be represented as graphs. I have a number of acceptable pairs of input and output graphs. I want to use these to train a model.
I am looking for pointers where simple examples of learning methods with graphs as input are discussed. Please note that the graph size is not fixed.
A sample input is
Graph:
  Node A: Component X with parameter size = 12
  Node B: Component Y with parameter size = 30
  Node C: Component Y with parameter size = 30
  A connects to B
  A connects to C

Sample output:
Node A: x=0, y=0
Node B: x=-21, y=0
Node C: x=21, y=0

In this case, we expect the model to understand that input graph is symmetric and a particular way of arranging them is preferred. We want to train the model over a large set of such input-output pairs and then use it to generate output on new inputs.

Could you elaborate on the problem you are trying to solve? What does your data represent?

The input graph represents a circuit - the nodes being components and nets/connections are the edges. I thought providing these details in the question might confuse people.

Very interesting problem, so you're trying to verify the circuit and component values or is it like auto routing the circuit copper track?

Mostly to infer patterns in the input, and to generate the an acceptable output which we think depends on these patterns.

Does this answer your question? Are there neural networks that accept graphs or trees as inputs?

You can flatten the graph into a matrix and then train it like a normal neural network input. Perhaps an adjacency graph or maybe simply a series of linear equations representing the nodes and convert it into matrix form.

If the circuit has only passive electronic components like resistor, capacitor or coil then ML can figure out their characters with sufficient training but if there are programmable active components like MCU/FPGA then it will be a tricky problem to solve.
Even active components like transistors, SCR, TRIAC have well-defined response characteristics under a normal working condition, but programmable components could run different firmware and behave differently.
I think you should see this as a time series problem where you will find out the possible output signal for a given component values in the mesh and the input signal parameters like Voltage and Current.
And if your circuits are really big you might want to break it down into isolated sections, then run a model for each of them separately, feed the output of one model as input to next one and so on. I guess this is where you got the graph idea.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
What do you call a machine learning system that keeps on learning?
As I understand it from this video lecture, there are three types of deep learning:

Supervised
Unsupervised
Reinforcement

All these can serve to train a neural network either only prior to its deployment or during its operating.
For the latter case, it is referred to as continuous learning here and here and as dynamic learning here and here.
Which term should I use to refer to a machine learning algorithm that keeps on learning (even after deployment)? If it's "continuous", is there an opposing term (such as "static" for "dynamic") for those systems that stop learning before being deployed?

There are several terms or expressions related to such systems, such as

online learning
incremental learning
continuous learning
continual learning
lifelong learning

They are sometimes used interchangeably, but some of them have slightly different meanings. For example, online learning does not need to be incremental, which refers to algorithms that attempt not to forget previously learned information.
The opposite of online is offline. However, the expression batch learning is sometimes used as an antonym for online learning.

In such a thriving field like AI, many terms are not fully established and in some areas it will take more time until the community agrees on specific terms for specific technologies.
Therefore, it might not be possible to give an definitive answer to your question yet. But it is important to note, that the references you gave do not all speak about the same type of learning during operation. And this gives as cues to dig deeper into the topic.
The easiest way to learn during operation is simple reinforcement learning. DQN of Deepmind works exactly that way. It tries out different moves and learns what works by receiving feedback through the reward function. There is not necessarily a separate learning and working phase. The AI just gets better over time. This means, that the NN keeps adapting to the challenge at hand and if that challenge changes, it adopts to the new challenge over time and "forgets" what worked before, if this old strategy is no longer efficient.
Your first reference to Deepmind addresses the aspect of forgetting. When talking about continuous learning, it could also mean, that a system does not forget previous skills but can utilize them later after learning something totally different. This requires more advanced architectures of AI. And this approach doesn't mean, that the NN must necessarily keep learning during operation. It could use this technique during training and stop learning once it is productive.
And there is a third field, recursive networks, which do not pass their information from neuron to neuron in a straight fashion but can contain loops or other types of gates, that control the flow of information. Such architectures are usually more complex than normal NNs, but resemble the working of animal brains a little more. In this field the term dynamic is used frequently.
Therefore, I would say that the term you are looking for is continuous learning - I have also read online learning for this type of NN - when you want to say that the NN keeps learning during production. The term dynamic is more common for recursive or other more complex types of NNs and fits there better in my opinion.
To conclude my answer, I need to jump back to the beginning. Many terms in the field of AI are not finally settled. Therefore my analysis is just a summary of the trends I have seen in different publications and lectures and must not hold true in the years to come. Hope it helps nevertheless.

ML, being a relatively young and fast-developing field, has numerous (near-)synonyms for many concepts.
One paradigm difference is whether a model is learned from a static, pre-defined set of data, or whether it adapts as new data is presented to it over time.
Some of the terms used to describe these two paradigms respectively (with subtle differences in meaning between authors/terms) are:

Offline / batch / isolated learning
Online / continual / continuous / incremental / lifelong learning

Further, some familiar branches of ML (like Transfer Learning and Multi-task learning) have a lot of intersection with Continual Learning.

Related q's:

What is the difference between active learning and online learning?

https://stats.stackexchange.com/questions/897/online-vs-offline-learning

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Transfer learning to train only for a new class while not affecting the predictions of the other class
I am basically interested in vehicle on the road.
YoloV3 pytorch is giving a decent result.
So my interested Vehicles Car Motorbike Bicycle Truck and bus, I have a small vehicles being detected as truck.
Since the small vehicle is nicely being detected as truck. I have annotated this small vehicle as a different class.
Though, I could add an extra class say 81th class, since the current YoloV3 being used is trained on 80 classes.
81th class would contain the weight of the truck, I would freeze the weights such that the rest of the 80 classes remain unaltered and only the 81th class of this new data gets trained.
The problem is the final layer gets tuned according to the prediction of all the classes it learns.
I was not able to find any post that could actually mention this way of preserving the predictions of the other classes and introducing a new class using transfer learning.
The closest, I was able to get is this post of
Weight Sampling Tutorial SSD using keras
Its mentioned in
Option 1: Just ignore the fact that we need only 8 classes
This would work, and it wouldn't even be a terrible option. Since only 8 out of the 80 classes would get trained, the model might get gradually worse at predicting the other 72 clases in the second paragraph.
Is it possible to preserve the predictions, of the previous pre trained model while introducing the new class and use transfer learning to train only for that class?
I Feel that this is not possible, would like to know your opinion. Hope someone can prove me wrong.

Even if you want to re-train your model for just one new class you will have to prepare your training data such that it includes all or most of the classes which you want to predict. Most of the times last two layers of a network have the data of number of labels which are to be predicted and that should always be sum of the number of classes you already trained on and the number of classes you want to add for training.


text:
Can A3C update the policy / critic on a local machine without needing to copy?
To make A2C into A3C you make it asynchronous. From what I understand the 'correct' way to do that is to thread off workers with a copy of the policy and critic, and then return the state/action/reward tuples to the main thread, which then performs the gradients updates on the main policy and critic, and then repeat the process.
I understand why the copying would be necessary in a distributed environment, but if I were to always run it locally then could I just perform the updates on a global variable of the policy and critic, i.e. avoid the need for copying? Provided the concurrency of the updates was handled correctly, would that be fine?

I don't think it is possible. If the global net updates every N iterations but local net (each agent) updates every M iterations and each agent runs asynchronously, having one copy of the weights is not possible since each agent might need a different copy of the weights.

The global net would be updated every M iterations also. To be clear; the worker would need to 'freeze' a copy of the policy/critic in order to do the updates, it would just get an immediate/online copy through the global variables. The only difference, as I see it, is that each worker has direct access to global variables for the policy/critic to perform the updates, rather than needing to return out with the S, a, r tuples for the main handler thread to do the updates. As I say, I understand the need for that in distributed, but I don't see why it'd be needed on a purely local environment

If you allow each worker to share the global net and update on it live, it is possible to do with only one copy of the network. This is called "Hogwild" fashion. I have seen this in Pytorch tutorial itself.

You might want to check out this paper relating to Phizaz's comment: Asynchronous Methods for Deep Reinforcement Learning (specifically search for Hogwild).

Thank you. I've since discovered that A2C can still have multiple environments/agents to garner experience more quickly (as you would in A3C), but the difference being that A2C waits for all agents to complete their experiences and apply gradient changes across all agents' batches, whereas A3C performs the gradient updates as soon as it has collected the specified size of experience data for that one agent. The benefit of multi-agent A2C is the agent is always working with the most up-to-date policy, and possible speed improvements if you use a GPU to combine the batches into one large batch.


text:
How to report the solution path of a search algorithm on a graph?
I'm working on a problem where we are given a graph and asked to perform various search algorithms (BFS, DFS, UCS, A*, etc.) and the goal state is to visit all nodes in the graph. After all nodes are visited, we need to print out the "solution path." However, I am a bit confused on what "path" means in AI.
For simplicity, let's just consider a graph of 3 nodes: A, B and C with 2 undirected edges (A, B) (A, C). If we perform BFS on this graph starting at node A and traversing alphabetically, we'd visit A, then B, then C.  So, in this case, is the solution path A -> B -> C, i.e. the order in which the nodes are visited? Or is the solution path A -> B -> A -> C? Basically saying that we go from A to B, but to go from B to C, we must go through A again.

Defining Path
You are right to be confused if your professors did not clarify (see warning at end). The term "path" can mean a few things:
"Concrete" Path:
Recall, a graph is a collection of vertices and edges. A path on the graph is then:
$$v_1\xrightarrow{e_1} v_2\xrightarrow{e_2} \cdots \xrightarrow{e_{n-1}} v_n$$
Where $v_i$ are vertices on the graph and the arrows denote the direction traversed over the undirected edges $e_j$. Observe, this definition explicitly defines some "chain" of vertices connected by specific edges.
"Abstract" Path: Say there is some task that involves a serious of repeated actions but does not depend on specific objects. We can define this as a chain of composed functions:
$$f_n\circ\cdots \circ f_1$$
The idea, is that this definition is little more abstract. Though, there are many other ways one could define "path."
"Solution Path"
A Subtlety

So, in this case, is the solution path A -> B -> C, i.e. the order in which the nodes are visited? Or is the solution path A -> B -> A -> C? Basically saying that we go from A to B, but to go from B to C, we must go through A again.

BFS does not "go through A again." Considering you are using a queue based implementation, once A has been visited there will be no need to "visit" it again. That is, we have added all nodes we need to the queue and marked A as being seen. This leads into:

How to report the solution path of a search algorithm on a graph?

Since, these algorithms do not revisit nodes, what is most likely being asked of you is for you to report the order of the visits - this is what my professors wanted in my AI & Algorithms courses.
A Warning: I can only answer what my professors wanted. Please clarify this with your instructors.


text:
How to train chat bot on infinite non-stationary data?
I have continual simulated data of million sentences of two simulated persons talking to each other in a room and I want to model one of the persons speech. Now, during this period things in the room can change. Let's say, one of them says "Where is the book?" The other one responds "I placed the book on the bookshelf". Now during time, the position of the book changes, so the question Where is the book? does not have stationary answer i.e the answer changes during time. However, in general the answer has to be "The book is at some_location" and not something else. Also, the mentioning that the book is placed on the bookshelf can be sometimes 10, 100 or 1000 sentences before the question "Where is the book?"
How do you approach this kind of problem? Since the window can be too large I can not split data into training samples of 10, 100 or 1000 sentences. My guess is that I should use BPTT + LSTM and train in one shot without shuffling the data. I am not sure this is feasible, so I will greatly appreciate your help! I have also my doubts what if "Where is the book?" appears 20 sentences after (instead of 10,100 and 1000) in the test set (which is not same as the training set)? Also, should I use Reinforcement Learning (since I can generate the data) or Supervised learning?
Thanks a lot!


text:
Dialects classification using deep learning
Dialects differ a lot between cities in my country, Syria. People sometimes express themselves using different local phrases and idioms which refer to the same topic. So, I came up with the idea of creating an Android application shows a limited set of sentences or expressions while asking you to express them in the local dialect of your region orally, after that this application tries to figure out what your dialect is. For a short period of time, I'm going to launch an Android application in order to collect the needed dataset which will be a new contribution. First of all, I need some helpful answers to my questions:

In general, is a period of 6 months enough for such a project to be done by only one student who is a beginner in this field or it is harder than it seems?
Are the libraries and tools needed to do this project available for free?
I know that more training data leads to more accurate results. In order to obtain good results, what is the estimated minimum number of training data needed for this model?
How do you advise me to begin?
How much is my suggested project relevant to the project attached in this link?

kindly write down your suggested edits and recommendations if any.
Edit for the 5th question: also see this paper.

I think that your project is interesting, but if you have never implemented anything using any ML library/framework before, then it will take you sometime first to get used to the libraries. It also depends on your programming background. It's hard to answer your questions. We don't know what you know about ML. If you really don't know anything, then maybe this project is quite challenge (I would say). Anyway, I think it will not be easy to give you satisfactory answers to all of your questions.

@nbro Yes, I'm a very beginner student in this field. However, I have a strong background in algorithms and data structs using the C++ programming language. Can you answer 2, 3 or 5, please! Thanks for your kindness.

You will be luckier if you ask just one question per post. If you ask multiple questions, then the users will be discouraged because an eventual answer to your questions might come up to be very long, or they might know the answer to one question but not the others, etc. So, ideally, you should try to ask questions which should not produce very long answers.

Anyway, I think that the answer to your 2nd question is "yes, there are free tools which will help you to implement such system or most of it".

Answered the same question at https://ai.stackexchange.com/questions/11116/how-much-the-dialects-recognition-and-speech-recognition-are-relevant


text:
Converging to a wrong optimal policy if the agent is given more choices
I am a bit new to Reinforcement learning. So, I am extremely sorry if I am asking something obvious. I have written a small piece of code to find the optimal policy for a 5x5 grid problem.

Scenario 1. The agent is only given two choices (Up, Right). I believe, I am getting an optimal policy.
Scenario 2. The agent is given four choices (Up, Right, Down, Left). I am getting the wrong answer.

I have represented actions with numbers:
0 - Right
1 - Up
2 - Down
3 - Left

When the action Up is chosen, with 0.9 probability it will move up or 0.1 probability move right and vice-versa. When the action Down is chosen, with 0.9 probability it will move down or 0.1 probability move left and vice-versa.
I did not use any convergence criteria. Instead let it run for sufficient iterations. I have indeed confirmed that my optimal state values and policy is converging but to a wrong number. I am attaching the code below:
def take_right(state):
    if (state/n < n-1): state = state + n
    return state

def take_up(state):
    if (state%n!=n-1): state = state + 1
    return state

def take_left(state):
    if (state/n > 0): state = state - n
    return state

def take_down(state):
    if (state%n > 0): state = state - 1
    return state

Scenario 1 result:

Scenario 2 result:

Green has a reward of 100 and Blue has a penalty of 100. Rest of the states have a penalty of 1. Discount factor is chosen as 0.5
Edit:
This was really silly question. The problem with my code was more pythonic than RL. Check the comments to get the clue.

Welcome to AI Stack Exchange. Could you clarify your state representation? The left arrows would be correct in the second diagram if your logic accidentally allowed wrap-around (so going left from left edge would be optimal if it put the agent on the right hand side). But it is hard to tell whether your action processing has blocked that. The left arrow on the bottom row looks correct to me, better than selecting "up" since it avoids the p(0.1 ) of ending in the penalty location from actually travelling right.

Reinforcement Learning is really fun because the agent will find any bug in your implementation and will exploit it.
>>> take_left(0)
0
>>> take_left(1)
-4

The agent figured out your bug with negative values and exploits negative indexing to get to the target faster.

Thanks a lot for that. I am fairly a bit new to python too (coming from C++). Anyway glad that my code is otherwise fine.


text:
Grouping products and naming groups
I'm working on a home tool that will help create a shopping list from a list of recipes chosen for a coming week.
This boils down to:

Extracting ingredients and their quantities from recipes.
Grouping similar ingredients together.
Summing up quantities for similar ingredients.
Naming groups of similar products in a shopping list.

The tasks seem non-trivial for a few reasons.

Similar ingredients are described differently, depending on the recipe book/portal, e.g.:

5 lemons
5 lemons (to be squeezed)
5 fresh lemons
5 big yellow lemons

Recipes lists alternatives for ingredients (e.g., "3 lemons or 5 limes"), leaving decision up to a user.
Recipes involve some information about product-preprocessing. For instance, one has to buy lemons instead of lemon juice when the recipe says:

100ml lemon juice
100ml freshly squeezed lemon juice

My language has a complex inflection. For instance, there can be multiple plural forms of a noun and the form of an adjective must be agreed with a form of noun. Adapting NLP algorithms designed for English language might be not straightforward and require some lemmatizing/stemming but not for single words, but whole phrases.
Naming products group is hard. Once fresh lemons and big yellow lemons are group together and their quantities summed up, one need to decide how to name this group in a shopping list, e.g.: "10 lemons" or "10 fresh lemons".

Is there any research paper that would cover those challenges?
Especially applied in the same domain?

PS. There are other challenges like different ways to quantity quantities (volume-based: ml, spoon, bunch, etc. and weigth-based: kg, g, etc.), but let's leave them out for now.

I worked at a start-up doing this exact task for two years. There were several developers involved in it, and it took several years; by the end (with a lot of manual input, eg categorising ingredients in an ontology, converting measurements, and mapping ingredients to shopping items from a supermarket inventory) we had something that worked reasonably well.
You rightly recognise that it is not trivial... The first step would be to identify the list of ingredients from the recipe (assuming you have scraped it from a website), and turn it into a list; beware that sometimes there might be more than one ingredient in a line. Then you need to analyse the entry: we had a grammar that distinguished between quantities, attributes (fresh), processing instructions (peeled) and a number of other things. The main item would of course be the actual ingredient. Then we would look this up in a supermarket inventory and map it to the closest item (but I don't think this is part of your remit).
We tried different ways of analysing the recipe lines, and any grammar/parser is fine, as long as it can deal with partial analyses, as you cannot always rely on lines being complete and well-formed, and there is always scope for additional comments you want to ignore ("I only use the freshest lemons directly imported from Tuscany", "get your fishmonger to take the bones out"). In the simplest case you could have a list of items (lemon) with inflections etc, that you try to match in your recipe lines. If you don't care whether it's big lemons or not, don't bother using a full-blown grammar. And even then, you could just look at a list of adjectives preceding an ingredient to identify any modifiers. We used a grammar based on recursive transition networks (RTNs, where the parser was complex but the grammars easy to handle) and one that was a simple phrase structure one (where the parser was simpler, but the grammar was much more complex). Both worked fine.
As far as I am aware there is not really much written about this kind of work. The main advice I can give you is to look at as many recipes as possible to get an idea for what you will encounter, and cut corners wherever possible -- it's a big job to do it perfectly, and you can get very far by using simple methods.
We also transferred it into different languages (Polish too, which I assume you're working with?). We didn't find that inflections were that big an issue. You just have a list of all inflected forms linking to the lemma. As you have a limited domain, that is a perfectly feasible approach. Just have a master list of ingredients, with all alternate forms (courgette and zucchini are the same thing, for example).


text:
Why do we update parameters with respect to cost instead of loss?
Assuming that the cost function $J$ is the average of the loss function $\mathcal{L}$ over all training examples $m$
$J(w) = \frac{1}{m} \displaystyle\sum_{i=0}^{m} \mathcal{L}$
why do we update the parameters with respect to the cost function
$w := w - \alpha \frac{dJ}{dw}$
instead of updating the parameters at each iteration with respect to its loss?
for i in m:
    w = w - learning_rate*(dL/dw)

Can you provide the source where this is done? Do you know what is the derivative of a summation?

The objective in parametric ML is to find parameter values that minimize the overall cost function $J(w)$ across the entire training dataset, whether in terms of the framework of empirical risk minimization (ERM) or information-theoretic cross-entropy (CE). Intuitively minimizing the average cost over all examples leads to a more generalizable model as it helps the model generalize well to unseen data rather than being influenced too heavily by the noise or fluctuations in individual examples.
Having said that, in the common stochastic gradient descent (SGD) training method, if your minibatch size is 1 then effectively during each batch you're updating the parameters w.r.t the loss of the single current sample, and this can be proved to be convergent under certain conditions albeit at a lower rate as referenced here:

The sum-minimization problem also arises for empirical risk minimization. There,
${Q_{i}(w)}$ is the value of the loss function at ${i}$-th example, and ${Q(w)}$ is the empirical risk... However, in other cases, evaluating the sum-gradient may require expensive evaluations of the gradients from all summand functions. When the training set is enormous and no simple formulas exist, evaluating the sums of gradients becomes very expensive, because evaluating the gradient requires evaluating all the summand functions' gradients. To economize on the computational cost at every iteration, stochastic gradient descent samples a subset of summand functions at every step.

The convergence of stochastic gradient descent has been analyzed using the theories of convex minimization and of stochastic approximation. Briefly, when the learning rates ${\eta }$ decrease with an appropriate rate, and subject to relatively mild assumptions, stochastic gradient descent converges almost surely to a global minimum when the objective function is convex or pseudoconvex, and otherwise converges almost surely to a local minimum. This is in fact a consequence of the Robbins–Siegmund theorem.

Because you want to do ERM, which aims at minimizing:
$$
\mathbb{E}_{x,y \sim p}[L(h(x))]
$$
Given this, if we apply a monte carlo approach to estimate that expected value, you end up with the mean of $N$ samples... the higher it is $N$, the better is the approximation (thus the lower the variance)
You can for sure update at each iteration wrt the loss of the single current sample, however you will struggle to converge


text:
Easiest way to train a neural-network with neurons that deviate from $f_{nl}(x \cdot A)$
I want to model how a neural network would behave for a system of input-output devices that are only approximately similar to a neuron. I think I have a resonable plan for how to do this, but I'm looking for some help in seeing if this is something that needs to be completely coded from scratch, or if this is something that can use some out-of-the-box tools in ML that already exist.
So first I'll describe what I want in more detail:
For a neuron in a neural network, the neuron in the model can be described by the following picture:

the output of the neuron is  $output = f_{nl}(x \cdot A)$,
where the inputs X are dot-producted with the weights A and are then a nonlinear function $f_{nl}$ is applied.
Now I am considering a mathematical model that behaves similar to a neuron, but not exactly.
This new model systematically gives something that deviates from this nonlinear process. For example, consider an output that looks like
$output = f_{nl}(x \cdot A-\epsilon A x^2)$,
For some small value of $\epsilon$ this is approximately a neuron. Could such an input-output system be trained with the same back-propagation techniques to be able to reliably learn on a training set?
Furthermore, I am considering involves a real physical system that needs to be solved with a system of differential equations, it might not have an analytical representation, and cannot be represented by this simple example of $f_{nl}(x \cdot A-\epsilon A x^2)$ instead it is just a general relationship:
$output = f(x, A)$,
The function is continuous, but needs to be solved numerically. It's very similar to the behavior of a neuron, but not exactly. I would like to see if such a thing can be trained on. I am thinking that I can create a neural network to learn the behavior of one of these "almost neurons" and then chain these neurons together to make a neural network.
My plan is to perform a large set of simulations for a wide set of inputs to collect a set of data representing the input-output response. Then I will make a neural network that learns the behavior of one of these "almost neurons."
For example, I could train it with a single neuron:

Let's call the inputs to this neuron Q, which consist of the both the inputs and weights of the input-output function that looks approximately like a neuron. The output of this neuron is:
$output = f_{nl}(Q \cdot B)$
In this example I have a 1 layer NN that is trained to learn the input-output behavior of my nonlinear function f(x, A). In principle this doesn't have to be a 1-layer NN, and could be some neural netweork that I've trained to represent f(x, A) (which approximately looks like a typical neuron).
The values of B are learned via the training data based on how my input nodes and weights vary the output f(x, A). I will obtain the values of B by trainining them on the a large dataset describing the input-out relationship of f(x, A) (which is obtained by repeatedly solving some differential equations).
So now I have a trained neural network which describing my nonlinear function, $output = f(x, A)$. Now I want to fix these weights and chain a number of these nonlinear functions to form a neural network.
So I can take my trained output behavior and create a neural network based on chaining a bunch of these things in series, to look like:

The values of B are fixed-weights that have been predetermed by the previous training, and characterizes the input-output behavior each of the individual input-output devices that are chained to form the network. The values of A are the unfixed weights that will be trained for a general machine learning data set. So for instance this system of 3 input-output devices will be trained via gradient decent to see if it can learn on pictures of numbers.
In this case it seems like the tricky thing is setting up this neural network which has some fixed weights, while others are tunable.
So ultimately my question is what is the fastest or easiest way to do such a thing.
I am pretty new to coding machine learning projects, so I was wondering if people knew if such a type of problem is possible to handle with an "out-of-the-box" solution, or if I will really need to write my own code from scratch. For example will I need to write my own backpropagation algorithms, or does this seem like something like (for example) Keras will be able to handle?

there is no such a thing in this question, you made close to no point, zero formalization, you just throw there buzzwords with no exact meaning... the least you can do to hope that somebody understands what you are saying, is to post an example of computation

@AlbertoSinigaglia, can you specify what exactly you don't understand?
I cannot provide an example of the computation because it requires a simulation to do so. It's not an analytical solution.

@AlbertoSinigaglia, I think my question is well-posed (I have a plan to train a model on something that behaves similar to a conventional neuron, and I want to then train these neurons in a neural network), so it would be useful to specify what in particular is not clear.

for example, what is $\overrightarrow{B} $? hwo do you combine the $x$s to get the $z$? your question is well-posed for you because you know what you want to do, it's a confirmation bias

I've added some more text to describe B. Ultimately, B just represents some weights that are used to train a neural network to look like a nonlinear function f(x, A). This nonlinear function is close to, but not exactly a neuron.

In your first example, i.e. $f_{nl}(x\cdot A-\epsilon Ax^2)$, since $Ax^2$ is continuous in the parameters $A$ you can implement this with common ML tools like tensorflow and pytorch, even with multiple neurons and in a batch fashion directly. It would be something like:
h1 = matmul(X, A)
h2 = matmul(A, square(X))
out = activation(h1 - epsilon * h2)

where epsilon I think is an hyper-parameter, i.e. something that is constant and you don't want to learn with back-prop.
So if you know the mathematical form of the novel "neuron" (or better said a layer) you want to implement, assuming also the function is differentiable, you can implement and train it easily.
But the second case, when you talk about a $f$ whose form is unknown, can be more difficult. If you have access to both inputs and output pairs, you can approximate $f$ the usual way by training a DNN regressor model: for example minimizing an $l_2$-loss or something more specific. Also, what you have described sounds like Neural ODE: I don't know much about it, but it's related with differential equations and, indeed, neural networks.

thanks for the answer. The former example you give of code in pytorch is not what I'm looking for -- as I understand that this is a very simple thing. The ladder is what I'm asking about, but I'm looking for something a bit more comprehensive about it -- but a DNN regressor model does point me in the right direction I think.


text:
Is it redunant to add more layers to a neural network with same number of neurons as the previous layer?
Lets say I have a neural network with three layers and the last layer has 3 outputs.
If I added additional layer of 3 neurons to the end of the network, would that be a more powerful neural network?
Here's an example picture of what I mean:

(where the circles are the neurons in the network, L1 is layer1. Also assume that all of the neurons are fully-connected to the neurons of the previous layer.)
Typically neural networks that I see the amount of neurons per network typically decrease as there are layers. But is it theoretically possible to a really powerful network by signiciantly increasing the depth of the network?
For example consider the following case:

Does stacking extra layers on the end of a neural network allow it to learn more complex tasks? My guess would be that stacking extra layers only helps if you are distilling the information down to less neurons per layer. But with more advanced neural networks like GPT-3, what has been said to me is what is important is the total amount of tunable parameters, and that the particular structure or depth of the network isn't important (when holding the total number of tunable parameters constant).
So phrasing this another way: If, for example, I wanted to create a neural network at the scale of a GPT-3 level, (ignoring the intricacies of the transformer architecture and just assuming that it is a basic neural network), is it possible that instead of adding additional neurons per layer, I can simply add more layers of the same neuron size to scale-up the total weight parameters?

Think again. Where does "the total amount of tunable parameters" come from? From "the amount of nodes or layers". It does not matter how big or small the network is.

@lpounng, sorry but I dont understand what you mean here. Of course a larger network (one with more parameters) is neccessary for more complicated problems -- so I don't follow.

Referring to this line, "...what is important is the total amount of tunable parameters, and that the amount of nodes or layers is not as important".

I know what line you are referring to, but don't understand your question.

It was not a question, but a statement. "the total amount of tunable parameters" is proportional to "the amount of nodes or layers", regardless of the scale of model. So the statement "what is important is the total amount of tunable parameters, and that the amount of nodes or layers is not as important" does not make sense.

I edited that sentence to make it more clear.

A deeper network will have more capacity, regardless of whether new layers have the same number of neurons as the previous layers, fewer, or more.
More capacity means that in theory the network can learn more complex functions. It does not mean that the network will then be better on any given task.
One important thing to note: This is only the case if you have a non linear activation function. If you do not have a non linear activation function between layers, then you are correct that each added layer, despite adding more parameters, does not add capacity to the network.
For further reference, I recommend reading this chapter of the deep learning book: https://www.deeplearningbook.org/contents/mlp.html
which contains all the information above (perhaps with more technical jargon) and more.
For a greater understanding of how adding layers affects capacity,  this article will be useful: https://en.wikipedia.org/wiki/Universal_approximation_theorem

"If you do not have a non linear activation function" - is that simply saying you have a linear activation function?

@eddiewould Nice reversal of the double negative :) But it also could mean that you lack an activation function entirely.


text:
What will happen if I concatenate one-hot-encoded categorical features along with continuous numerical features?
Here is one row from my data:
H  7.042 5.781 5.399  -9.118   5.488  7.470

The first column is a categorical class. The rest of them are continuous numerical features.
I encoded the classes using one-hot-encoding labels and concatenated them with the numerical features list:
    1   0  1  7.042 5.781 5.399  -9.118   5.488  7.470

Then I used this list for training.

Is this a valid technique?
Am I achieving anything really useful here?

My supervisor says that I made a mistake by using labels as features. I am trying to understand what I did wrong. In my view, the mistake seems to be: when I am using labels as features, I must provide those labels as input whenever I want to use the trained model. Therefore, this trained model is practically useless. Am I correct?

Cross posted in stats.SE

when you say 'features' do you refer specifically to continuous values as opposed to one hot encoded ones?

@EdoardoGuerriero, Yes. see the edit.

Your approach is totally valid (especially considering you only have a single categorical feature).
If you're training a neural network though make sure to standardize your numerical features in order to be in the range [0, 1].
Also, in general, and again only if you're training a neural network, you might keep in mind that there's also another valid concatenation approach, happening not at the input layer level but in a hidden layer. This is common in natural language processing, where the text features can be quite larger in number than the numerical ones. By processing separately numerical and text features we can ensure that both are mapped to a fixed size dense layer, so when the concatenation occurs, we give same importance to both types of features regardless of their respective amounts. Another advantage of this approach is that when using one hot encoding the resulting features are sparse, and by keeping them separated you can treat them accordingly (for example using a loss function for sparse features only on the one hot feature block).

OP's post on stats.SE clarifies in a comment that the intention is to use the label to predict the label. https://stats.stackexchange.com/questions/565623/what-will-happen-if-i-concatenate-one-hot-encoded-class-along-with-features#comment1172091_565623 In other words, the proposed procedure is entirely tautological. I think this answer assumes the typical case, where a person encodes categorical features and uses them to predict the class label.


text:
Why is the validation accuracy lower in case of CNN?
I fed the same set of 1.4 million data to two different models:

MLP
CNN model

In both cases, I used the same parameters and hyperparameters.
The CNN is showing comparatively lower accuracy (80%) than that of the MLP (82%).
Why?
And, also, what does this experiment tell us?
Edit:

Is the data images, video or audio, or other grid-based signals (same signal repeated at multiple locations with a meaningful distance metric between them) in your case?

It is protein C-alpha distances data that has 3 classes (helix, strand, coil) and n-number of features, where n is an even number.

In fact, what is it, what was your expectation of the relative performance of the two models, and why?

I thought CNN would be more efficient and thereby would demonstrate better test/validation accuracy.

Is the data images, video or audio, or other grid-based signal (same signal repeated at multiple locations with a meaningful distance metric between them) in your case? In fact what is it, what was your expectation of relative performance of the two models, and why?

@NeilSlater, see the edit.

Thanks for the edit. Sorry, I still don't understand your description of the data. A protein could be represented by a 1d signal of identical features (e.g. an array of amino acids), but could be anything else. I need more. E.g. If I said my data was "car data" you would have no clue whether it was pictures of cars, measurements of dimensions, engine stats, accident stats etc. Please give something more useful to work with. Also, while I am asking, how have you "balanced" between the models in terms of number of parameters (perhaps give the basic number of neurons etc).

@NeilSlater, C-alpha distances.

So you are doing something like this: https://www.researchgate.net/publication/253393184_Using_Distances_Between_alpha-Carbons_to_Predict_Protein_Structure ?

@NeilSlater, Yes...

To get a full understanding of your problem, one would like to know what approximately the $n$-features are.
Whether, it is about the geometrical structure, protein is described by a graph, where vertices correspond to atoms and edges to bonds within them - I would consider use of GraphNN, there is some research, that has demonstrated the success of GraphNN for protein prediction:

Structure-based protein function prediction using graph convolutional networks
Protein Interface Prediction using Graph Convolutional Networks

In case, your data is some general form tabular data with features of different nature and form - like some continuous features, binary features, categorical features, whatever, there is no notion of proximity between different features.
The efficiency of CNN is based a lot on the ability of them to have a notion of locality, aggregate the information from the neighborhood, and construct the hierarchy of low-level (for several neighboring pixels) features and global (that understand the image or a signal as a whole) features.
Two neighboring pixes from the cat's ear have a notion of proximity and relatedness, whereas the red color and square shape do not have.
For unstructured tabular data I would recommend to start from some tree ensembling approach:

Random forest
Gradient boosting

EDIT
Seems like what you receive as input is a matrix of pairwise distances $\rho(i, j)$ between all possible amino residues - Protein contact map.
I would think about it as a weighted adjacency matrix. However, there is a rather special structure,
Therefore, there is no need for generic GNN, most likely.
In the literature, the recent research solves this problem with the help of CNN:

Improved protein structure prediction using predicted interresidue orientations
A fully open-source framework for deep learning protein real-valued distances

Probably, you should tune some hyperparameters, or introduce residual connections, if there are no such at the moment.

What do you mean by residual connection?

@user366312 for deep neural networks - with the number of layers more then 10, say, networks become very hard to optimize and accuracy saturates at rather high loss (train and test). I was a breakthrough, when Resnets were introduced, where after each layer, without downsampling or change of number of channels, one adds the input of the stack of layers to the output. Concatenation is possible as well. This combats the problems with optimization.


text:
How can I approach this problem of producing a 3-bit binary string given a sequence of letters?
Suppose, I have the following data-set:
... ...
... ...
AABBB  7.027  5.338  5.335  8.122  5.537  6.408
ABBBA  5.338  5.335  5.659  5.537  5.241  7.043
BBBAA  5.335  5.659  6.954  5.241  8.470  8.474
BBAAA  5.659  6.954  5.954  8.470  9.266  9.334
BAACA  6.954  5.954  6.117  9.266  9.243 12.200
AABAA  5.954  6.117  6.180  9.243  8.688 11.842
ACAAA  6.117  6.180  5.393  8.688  5.073  7.722
ABAAC  6.180  5.393  6.795  5.073  8.719  7.854
BAACC  5.393  6.795  5.796  8.719  9.196  9.705
... ...
... ...

Apparently, the feature values represent a string pattern comprising of only three letters A, B, and C.
I have to design a neural network that would be able to detect these patterns and spit out a binary representation of these strings where the letters should be encoded in 3-bit binary(one-hot encoding).
My first question is, What kind of problem is it and why?
My next question is, How should I approach this problem to solve it?

Is the length of the strings consistent? If they aren't use an RNN. Otherwise, use crinix's answer

@Recessive, Is it a Multi-label Classification problem? Or, is it a multinomial logistic regression problem?

@Recessive, the length of the string is fixed to 5.

If you're trying to predict the string pattern, given the numerical feature and assuming your string pattern is fixed sized, you can one-hot encode each letter then combine them (into an array that is no longer one-hot).
So AABBC would look like:
[1,0,0,1,0,0,0,1,0,0,1,0,0,0,1] <- Use this for training
[A,B,C,A,B,C,A,B,C,A,B,C,A,B,C]
[A,_,_,A,_,_,_,B,_,_,B,_,_,_,C]
AABBC

where each group of triplets represent a single integer.
Then you can train a network with cross-entropy.
This is the problem formulation of multi-task learning where you predict multiple things simultaneously.
Needless to say, it is classification.

Is it a Multi-label Classification problem? Or, is it a multinomial logistic regression problem?

It is not multinomial logistic regression or multi-label classification, which are the same in their essence, where there are more than 2 classes to predict. It can be said to be multi-multinomial logistic regression. Its proper name is multi-task learning.

Is multi-task learning synonymous to multi-output learning?

@user366312 I can't find a definition for multi-output learning anywhere, but maybe you mean multi-instance learning? In which case, no, they are not equivalent. I think what may be confusing you is one small detail - you should split each character classifier into it's own cost function. So on the final layer, you'll have 5 classifiers, each for characters 1,2,3,4 and 5 in the sequence. Then you can encode each classifier 1 hot and use a simple softmax cross-entropy set up.

@Recessive, Scikit-learn 84:Supervised Learning 62: Multiclass & Multioutput