Without loss of generality, assume that $z = \min\{x, y, z\}$. The inequality can be rewritten as
$$4xy \le 3 + (x+y-z)^2.$$
Since $x+y-z \ge 2\sqrt{xy-z} > 0$, we have that $(x+y-z)^2 \ge (2\sqrt{xy-z})^2$. So, it is sufficient to prove that $3 + (2\sqrt{xy-z})^2 \ge 4xy$. From the problem condition $xy = \frac{1}{z}$, hence, we need to prove the following inequality
$$3 + \left( \frac{2}{\sqrt{z}} - z \right)^2 \ge \frac{4}{z}.$$
Expanding the brackets and multiplying by $z$, we obtain the inequality: $z^2 + 3 \ge 4\sqrt{z}$, which follows from the AM-GM inequality:
$$z^2 + 1 + 1 + 1 \ge 4\sqrt{z^2 \cdot 1 \cdot 1 \cdot 1} = 4\sqrt{z}.$$

We can rewrite our inequality in the form
$$2xy + 2yz + 2zx \le 3 + x^2 + y^2 + z^2.$$
We will use the Schur's inequality $(u, v, w \ge 0)$:
$$u^3 + v^3 + w^3 + 3uvw \ge u^2v + v^2u + w^2u + w^2v + w^2u + w^2v + w^2u,$$
and the AM-GM inequality for two numbers. With $x = a^3$, $y = b^3$, $z = c^3$, we have
$$$$
$$\begin{aligned} x^2 + y^2 + z^2 + 3 &= a^6 + b^6 + c^6 + 3a^2b^2c^2 \ &\ge (a^4b^2 + a^2b^4) + (b^4c^2 + b^2c^4) + (c^4a^2 + c^2a^4) \\ &\ge 2a^3b^3 + 2b^3c^3 + 2c^3a^3 = 2xy + 2yz + 2xz. \end{aligned}$$