<p>Answering your edited question: any learning algorithm (more specifically: supervised classification) algorithm <em>always</em> (either implicitly or explicitly) works with a well-defined class of <em>hypotheses</em> or mappings from examples to labels. In the case of the perceptron in my previous answer, it works with linear separators (and hence will not consider hypotheses involving square roots). A standard no-free-lunch theorem shows that an algorithm whose hypothesis space is too rich will no be able to learn (it will overfit).</p>

CC BY-SA 3.0
2017-01-18T12:11:45.563Z
2017-01-18T12:11:45.563Z
Aryeh
<p>A perceptron
<a href="https://en.wikipedia.org/wiki/Perceptron" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Perceptron</a>
or SVM
<a href="https://en.wikipedia.org/wiki/Support_vector_machine" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Support_vector_machine</a>
learning algorithm would find a separating hyperplane -- which in your case, would be something like $w=(0,1,0,0,1)$ with bias $b=-1$.</p>

CC BY-SA 3.0
2017-01-18T08:08:42.223Z
2017-01-18T08:08:42.223Z
Aryeh
<p>I am truly fascinated by algorithms learning on their own with a little help from humans and as a newbie in this field (with programming experience mainly in C/C++), seek your help to obtain the big picture before delving into specifics. I am not a researcher, but broadly want to know what is happening under the hood. A nudge in the right direction is all I need.</p>

<p>What I have learnt so far can be summarized as:</p>

<ul>
<li>There is both supervised and unsupervised learning.</li>
<li>Supervised learning is typically used to solve regression and classification </li>
<li>Unsupervised learning is used to solve clustering related problems</li>
<li>In supervised learning, we provide training data that are labelled</li>
<li>Each training data example can comprise multiple features.</li>
<li>An ML algorithm reads the labelled data, possibly finds patterns and "deduces" a general rule that maps input to output (e.g. mapping between a feature vector and a label)</li>
<li>If now an unlabeled feature vector was given, the algorithm uses the "deduced" rule and classifies the input into a label class.</li>
<li>I also understand that there are quite a few classification algorithms like SVM, Nearest neighbor, Decision Trees etc.</li>
<li>Each of these algorithms have different criteria that leads them to different "deduced" rules on basis of which, they perform classification.</li>
</ul>

<p>So the concepts am so desirous to understand at a broad level are:</p>

<ul>
<li>How does an algorithm determine inter-relation ship among features?</li>
<li>How does it know what features are to be considered and what could possibly be disregarded?</li>
<li>What does a deduced rule physically look like?</li>
<li>Is there a standard suite of rules to choose from?</li>
<li>How does the algorithm arrive at such a rule?  </li>
</ul>

<p>For example, if I had a dataset as the following:</p>

<pre><code>0 1 0 0 1 1   +
1 1 1 0 1 1   +
1 1 0 1 1 0   +
1 0 1 0 1 1   -
1 0 1 1 1 1   -
0 1 1 0 0 0   -
</code></pre>

<p>In the above example, the label is + if the second and fifth element were 1. How will an algorithm deduce this? </p>

<p>I shall be truly grateful for any clarification you provide.</p>

<p>EDIT</p>

<p>Based on Jan's recommendation, my question boils down to the following:</p>

<p>Assume x1, x2 ... xn are attributes/features. Assume h(y) is the mapping function and it exists. How does the algorithm decide that h(y) should be (x1 + x2 + x9) /5 and not h(y) = Sq Root(x1^2 + X9^3) + k</p>

<p>Given the size of the dataset, the possibilities are endless, isn't it?</p>

<p>h(y) could either be a direct classification or some value that could lead to a class subsequently.  </p>

CC BY-SA 3.0
2017-01-18T07:24:54.103Z
I think this question may be too broad to attract a satisfactory answer.
kodlu
CC BY-SA 3.0
2017-01-18T07:31:47.717Z
You are entitled to your opinion. But the main reason why people come to these forums is because they need the spark/insight/intuition which classic literature does not provide. Satisfactory answer or not is for the OP to decide and not you!
Raj
CC BY-SA 3.0
2017-01-18T08:29:26.140Z
I agree this question is too broad. Also there are several questions here, please ask only one definitely answerable question per post.
Jan Johannsen
<machine-learning>
Machine Learning: How ML algorithms build classification rules
Raj
# Machine Learning: How ML algorithms build classification rules

(Asked by: username_0 on 2017-01-18)

I am truly fascinated by algorithms learning on their own with a little help from humans and as a newbie in this field (with programming experience mainly in C/C++), seek your help to obtain the big picture before delving into specifics. I am not a researcher, but broadly want to know what is happening under the hood. A nudge in the right direction is all I need.

What I have learnt so far can be summarized as:

* There is both supervised and unsupervised learning.
* Supervised learning is typically used to solve regression and classification
* Unsupervised learning is used to solve clustering related problems
* In supervised learning, we provide training data that are labelled
* Each training data example can comprise multiple features.
* An ML algorithm reads the labelled data, possibly finds patterns and "deduces" a general rule that maps input to output (e.g. mapping between a feature vector and a label)
* If now an unlabeled feature vector was given, the algorithm uses the "deduced" rule and classifies the input into a label class.
* I also understand that there are quite a few classification algorithms like SVM, Nearest neighbor, Decision Trees etc.
* Each of these algorithms have different criteria that leads them to different "deduced" rules on basis of which, they perform classification.

So the concepts am so desirous to understand at a broad level are:

* How does an algorithm determine inter-relation ship among features?
* How does it know what features are to be considered and what could possibly be disregarded?
* What does a deduced rule physically look like?
* Is there a standard suite of rules to choose from?
* How does the algorithm arrive at such a rule?

For example, if I had a dataset as the following:

```
0 1 0 0 1 1   +
1 1 1 0 1 1   +
1 1 0 1 1 0   +
1 0 1 0 1 1   -
1 0 1 1 1 1   -
0 1 1 0 0 0   -

```

In the above example, the label is + if the second and fifth element were 1. How will an algorithm deduce this?

I shall be truly grateful for any clarification you provide.

EDIT

Based on Jan's recommendation, my question boils down to the following:

Assume x1, x2 ... xn are attributes/features. Assume h(y) is the mapping function and it exists. How does the algorithm decide that h(y) should be (x1 + x2 + x9) /5 and not h(y) = Sq Root(x1^2 + X9^3) + k

Given the size of the dataset, the possibilities are endless, isn't it?

h(y) could either be a direct classification or some value that could lead to a class subsequently.

**Tags:** machine-learning

> **Comments:**
> **username_1**: I think this question may be too broad to attract a satisfactory answer.  
> **username_0**: You are entitled to your opinion. But the main reason why people come to these forums is because they need the spark/insight/intuition which classic literature does not provide. Satisfactory answer or not is for the OP to decide and not you!  
> **username_2**: I agree this question is too broad. Also there are several questions here, please ask only one definitely answerable question per post.  

---

## 2 answers

### Answer by username_3 on 2017-01-18. Score: 0 | Accepted answer

Answering your edited question: any learning algorithm (more specifically: supervised classification) algorithm *always* (either implicitly or explicitly) works with a well-defined class of *hypotheses* or mappings from examples to labels. In the case of the perceptron in my previous answer, it works with linear separators (and hence will not consider hypotheses involving square roots). A standard no-free-lunch theorem shows that an algorithm whose hypothesis space is too rich will no be able to learn (it will overfit).


---

### Answer by username_3 on 2017-01-18. Score: 0

A perceptron
<https://en.wikipedia.org/wiki/Perceptron>
or SVM
<https://en.wikipedia.org/wiki/Support_vector_machine>
learning algorithm would find a separating hyperplane -- which in your case, would be something like $w=(0,1,0,0,1)$ with bias $b=-1$.


---

