Based on the hypothesis that the (non-reversible) arrow of time is intrinsic in any system, no matter how small, the consequences are discussed. Within the framework of local quantum physics it is shown how such a semi-group action of time can consistently be extended to that of the group of spacetime translations in Minkowski space. In presence of massless excitations, however, there arise ambiguities in the theoretical extensions of the time translations to the past. The corresponding loss of quantum information on states upon time is determined. Finally, it is explained how the description of operations in classical terms combined with constraints imposed by the arrow of time leads to a quantum theoretical framework. These results suggest that the arrow of time is fundamental in nature and not merely a consequence of statistical effects on which the Second Law is based.