Pseudo-polynomial time algorithms Consider the following algorithm:

Given a natural number as input, say $N$, the algorithm runs a loop (in which the algorithm does $O(1)$ time operations) $N$ times. Now, by definition of time complexity, we want to measure the running time as a function of input size, so the complexity of this algorithm is $O(N)=O(2^{\lg N})$.  I have always felt a little uneasy that this simple algorithm is actually has an exponential running time.

More examples of such problems include integer factorization, knapsack (and a lot of other related papers Google throws up) which all have a running time that is pseudo-polynomial in the size of the input(also called as weakly NP-complete problems). Is there any survey/book that treats these problems in detail? Are there any interesting aspects of these problems (as in, they have some properties that characterize them distinctly from strongly NP-complete problems) that are well-known?
I think this somehow “uneasy feeling” comes from the fact that human beings consider natural numbers only with a semantic—if I see 2001, e.g., I immediately think of 9/11 or Stanley Kubrick or something, that is, you interpret the number. In terms of inputs for an algorithm, however, it is merely a string of characters, and could just as well be such nonsense as ¨*^! or anything. What we as algorithm analyzers care about is only the length of the string, and absolutely not what it contains or represents. You really have to get this into your mind because otherwise you could e.g. say that $PRIMES$ is obviously in $\mathsf{P}$ as you can just try to divide a number $x$ by all numbers between 1 and $x,$ and this algorithm has obviously polynomial runtime. I hope you see the pitfall.