A self-contained introduction to finite dimensional vector spaces, matrices, systems of linear equations, spectral analysis on euclidean and hermitian spaces, affine euclidean geometry, quadratic forms and conic sections. The mathematical formalism is motivated and introduced by problems from physics, notably mechanics (including celestial) and electro-magnetism, with more than two hundreds examples and solved exercises.Topics include: The group of orthogonal transformations on euclidean spaces, in particular rotations, with Euler angles and angular velocity. The rigid body with its inertia matrix. The unitary group. Lie algebras and exponential map. The Dirac's bra-ket formalism. Spectral theory for self-adjoint endomorphisms on euclidean and hermitian spaces. The Minkowski spacetime from special relativity and the Maxwell equations. Conic sections with the use of eccentricity and Keplerian motions. An appendix collects basic algebraic notions like group, ring and field; and complex numbers and integers modulo a prime number.The book will be useful to students taking a physics or engineer degree for a basic education as well as for students who wish to be competent in the subject and who may want to pursue a post-graduate qualification.