Etydhv

I know that, from compactness theorem, one can prove that there are models of first order arithmetic in which there is some "number" which is not a successor of zero, in the sense that it is strictly bigger than any successor of zero (i.e. any element of the model obtained by applying the successor function to zero finitely many times).

From the axiom of infinity, it follows that every model of ZFC must contain an element which one can think of as "the natural numbers", in the sense that it is a model of Peano Axioms. Peano Axioms are a second-order theory, since the principle of induction is a second order axiom, and from the principle of induction it follows that the Peano Axioms have a unique model in ZFC, so that we can call this model among the others of first-order arithmetic, the "standard" model of arithmetic.

But picking one model of ZFC, how do we know what's really inside its standard model of arithmetic? How do we know there is nothing else than the successors of zero? After all, every model of ZFC thinks that his natural numbers are the standard ones, so one can use compactness to produce a model of ZFC which has non-standard natural numbers from an external point of view, and in which there will be a standard natural numbers object containing elements bigger than any successor of zero.

So, if one wants to do mathematics inside a model of a first order theory of sets, how can one know that he is able to pick a model in which the natural numbers are not non-standard?

This, in fact, cannot be proven, even in $ZFC+Con(ZFC)$. This is because $ZFC$ proves the following statement:

If we have a model $M$ of $ZFC$ whose natural numbers are standard, then $M$ satisfies $ZFC+Con(ZFC)$.

Indeed, $Con(ZFC)$ is an arithmetic statement. Since we are assuming that $ZFC$ has a model $M$, and hence that $ZFC$ is consistent, $mathbb NvDash Con(ZFC)$, and since $mathbb N^Mcongmathbb N$, $mathbb N^MvDash Con(ZFC)$, and hence $MvDash Con(ZFC)$.

Now if $ZFC$ could prove "if there is a model of $ZFC$, then there is a model of $ZFC$ with standard $mathbb N$", then we would get that $ZFC+Con(ZFC)$ proves that $ZFC+Con(ZFC)$ has a model, contradicting Godel's second incompleteness theorem.

Therefore, we cannot conclude, from existence of a model, existence of a model with standard $mathbb N$.

Thought it might be worth mentioning that this reasoning is under the assumption that $ZFC+Con(ZFC)$ is consistent. In the other case, $ZFC$ proves that $ZFC$ has no models, so the implication I discuss holds vacuously.