编程实现优化算法，并3D可视化

1. 函数3D可视化

分别画出和的3D图

NNDL实验 优化算法3D轨迹 鱼书例题3D版_优化算法3d展示-CSDN博客

2.加入优化算法，画出轨迹

分别画出和的3D轨迹图

从轨迹、速度等多个角度讲解各个算法优缺点

NNDL实验 优化算法3D轨迹 pytorch版-CSDN博客

3.复现CS231经典动画

从轨迹、速度等多个角度讲解各个算法优缺点

Animations that may help your intuitions about the learning process dynamics. 

Left: Contours of a loss surface and time evolution of different optimization algorithms. Notice the "overshooting" behavior of momentum-based methods, which make the optimization look like a ball rolling down the hill. 

Right: A visualization of a saddle point in the optimization landscape, where the curvature along different dimension has different signs (one dimension curves up and another down). Notice that SGD has a very hard time breaking symmetry and gets stuck on the top. Conversely, algorithms such as RMSprop will see very low gradients in the saddle direction. Due to the denominator term in the RMSprop update, this will increase the effective learning rate along this direction, helping RMSProp proceed. 

NNDL实验 优化算法3D轨迹 复现cs231经典动画_HBU_David的博客-CSDN博客