Reservoir operation is a multi-objective optimization problem, and is traditionally solved with dynamic programming (DP) and stochastic dynamic programming (SDP) algorithms. The thesis presents novel algorithms for optimal reservoir operation, named nested DP (nDP), nested SDP (nSDP), nested reinforcement learning (nRL) and their multi-objective (MO) variants, correspondingly MOnDP, MOnSDP and MOnRL. The idea is to include a nested optimization algorithm into each state transition, which reduces the initial problem dimension and alleviates the curse of dimensionality. These algorithms can solve multi-objective optimization problems, without significantly increasing the algorithm complexity or the computational expenses. It can additionally handle dense and irregular variable discretization. All algorithms are coded in Java and were tested on the case study of the Knezevo reservoir in the Republic of Macedonia. Nested optimization algorithms are embedded in a cloud application platform for water resources modeling and optimization. The platform is available 24/7, accessible from everywhere, scalable, distributed, interoperable, and it creates a real-time multiuser collaboration platform. This thesis contributes with new and more powerful algorithms for an optimal reservoir operation and cloud application platform. All source codes are available for public use and can be used by researchers and practitioners to further advance the mentioned areas.