🎬个人简介：一个全栈工程师的升级之路！
📋个人专栏：pytorch深度学习
🎀CSDN主页 发狂的小花
🌄人生秘诀：学习的本质就是极致重复!
视频来自【b站刘二大人】
目录
1 Linear Regression
2 Dataloader 数据读取机制
3 代码
1 Linear Regression
使用Pytorch实现，步骤如下：
PyTorch Fashion(风格)
- prepare dataset
- design model using Class ，前向传播，计算y_pred
- Construct loss and optimizer，计算loss，Optimizer 更新w
- Training cycle (forward,backward,update)
2 Dataloader 数据读取机制
- Pytorch数据读取机制
一文搞懂Pytorch数据读取机制！_pytorch的batch读取数据-CSDN博客
- 小批量数据读取
import torch
import torch.utils.data as Data
BATCH_SIZE = 3
x_data = torch.tensor([[1.0],[2.0],[3.0],[4.0],[5.0],[6.0],[7.0],[8.0],[9.0]])
y_data = torch.tensor([[2.0],[4.0],[6.0],[8.0],[10.0],[12.0],[14.0],[16.0],[18.0]])
dataset = Data.TensorDataset(x_data,y_data)
loader = Data.DataLoader(
dataset=dataset,
batch_size=BATCH_SIZE,
shuffle=True,
num_workers=0
)
for epoch in range(3):
for step, (batch_x, batch_y) in enumerate(loader):
print('epoch', epoch,
'| step:', step,
'| batch_x', batch_x,
'| batch_y:', batch_y)
3 代码
import torch
import torch.utils.data as Data
import matplotlib.pyplot as plt
# prepare dataset
BATCH_SIZE = 3
epoch_list = []
loss_list = []
x_data = torch.tensor([[1.0],[2.0],[3.0],[4.0],[5.0],[6.0],[7.0],[8.0],[9.0]])
y_data = torch.tensor([[2.0],[4.0],[6.0],[8.0],[10.0],[12.0],[14.0],[16.0],[18.0]])
dataset = Data.TensorDataset(x_data,y_data)
loader = Data.DataLoader(
dataset=dataset,
batch_size=BATCH_SIZE,
shuffle=True,
num_workers=0
)
#design model using class
"""
our model class should be inherit from nn.Module, which is base class for all neural network modules.
member methods __init__() and forward() have to be implemented
class nn.linear contain two member Tensors: weight and bias
class nn.Linear has implemented the magic method __call__(),which enable the instance of the class can
be called just like a function.Normally the forward() will be called
"""
class LinearModel(torch.nn.Module):
def __init__(self):
super(LinearModel, self).__init__()
# (1,1)是指输入x和输出y的特征维度，这里数据集中的x和y的特征都是1维的
# 该线性层需要学习的参数是w和b 获取w/b的方式分别是~linear.weight/linear.bias
self.linear = torch.nn.Linear(1, 1)
def forward(self, x):
y_pred = self.linear(x)
return y_pred
model = LinearModel()
# construct loss and optimizer
# criterion = torch.nn.MSELoss(size_average = False)
criterion = torch.nn.MSELoss(reduction = 'sum')
optimizer = torch.optim.SGD(model.parameters(), lr = 0.01)
# training cycle forward, backward, update
for epoch in range(1000):
for iteration, (batch_x, batch_y) in enumerate(loader):
y_pred = model(batch_x) # forward
loss = criterion(y_pred, batch_y) # backward
# print("epoch: ",epoch, " iteration: ",iteration," loss: ",loss.item())
optimizer.zero_grad() # the grad computer by .backward() will be accumulated. so before backward, remember set the grad to zero
loss.backward() # backward: autograd，自动计算梯度
optimizer.step() # update 参数，即更新w和b的值
print("epoch: ",epoch, " loss: ",loss.item())
epoch_list.append(epoch)
loss_list.append(loss.data.item())
if (loss.data.item() < 1e-7):
print("Epoch: ",epoch+1,"loss is: ",loss.data.item(),"(w,b): ","(",model.linear.weight.item(),",",model.linear.bias.item(),")")
break
print('w = ', model.linear.weight.item())
print('b = ', model.linear.bias.item())
x_test = torch.tensor([[10.0]])
y_test = model(x_test)
print('y_pred = ', y_test.data)
plt.plot(epoch_list,loss_list)
plt.title("SGD")
plt.xlabel("epoch")
plt.ylabel("loss")
plt.savefig("./data/pytorch4.png")
- 几种不同的优化器对应的结果：
Pytorch优化器全总结（三）牛顿法、BFGS、L-BFGS 含代码
pytorch LBFGS_lbfgs优化器-CSDN博客
scg.step() missing 1 required positiona-CSDN博客
- LFBGS 代码
import torch
import torch.utils.data as Data
import matplotlib.pyplot as plt
# prepare dataset
BATCH_SIZE = 3
epoch_list = []
loss_list = []
x_data = torch.tensor([[1.0],[2.0],[3.0],[4.0],[5.0],[6.0],[7.0],[8.0],[9.0]])
y_data = torch.tensor([[2.0],[4.0],[6.0],[8.0],[10.0],[12.0],[14.0],[16.0],[18.0]])
dataset = Data.TensorDataset(x_data,y_data)
loader = Data.DataLoader(
dataset=dataset,
batch_size=BATCH_SIZE,
shuffle=True,
num_workers=0
)
#design model using class
"""
our model class should be inherit from nn.Module, which is base class for all neural network modules.
member methods __init__() and forward() have to be implemented
class nn.linear contain two member Tensors: weight and bias
class nn.Linear has implemented the magic method __call__(),which enable the instance of the class can
be called just like a function.Normally the forward() will be called
"""
class LinearModel(torch.nn.Module):
def __init__(self):
super(LinearModel, self).__init__()
# (1,1)是指输入x和输出y的特征维度，这里数据集中的x和y的特征都是1维的
# 该线性层需要学习的参数是w和b 获取w/b的方式分别是~linear.weight/linear.bias
self.linear = torch.nn.Linear(1, 1)
def forward(self, x):
y_pred = self.linear(x)
return y_pred
model = LinearModel()
# construct loss and optimizer
# criterion = torch.nn.MSELoss(size_average = False)
criterion = torch.nn.MSELoss(reduction = 'sum')
optimizer = torch.optim.LBFGS(model.parameters(), lr = 0.1) # model.parameters()自动完成参数的初始化操作，这个地方我可能理解错了
loss = torch.Tensor([1000.])
# training cycle forward, backward, update
for epoch in range(1000):
for iteration, (batch_x, batch_y) in enumerate(loader):
def closure():
y_pred = model(batch_x) # forward
loss = criterion(y_pred, batch_y) # backward
# print("epoch: ",epoch, " iteration: ",iteration," loss: ",loss.item())
optimizer.zero_grad() # the grad computer by .backward() will be accumulated. so before backward, remember set the grad to zero
loss.backward() # backward: autograd，自动计算梯度
return loss
loss = closure()
optimizer.step(closure) # update 参数，即更新w和b的值
print("epoch: ",epoch, " loss: ",loss.item())
epoch_list.append(epoch)
loss_list.append(loss.data.item())
if (loss.data.item() < 1e-7):
print("Epoch: ",epoch+1,"loss is: ",loss.data.item(),"(w,b): ","(",model.linear.weight.item(),",",model.linear.bias.item(),")")
break
print('w = ', model.linear.weight.item())
print('b = ', model.linear.bias.item())
x_test = torch.tensor([[10.0]])
y_test = model(x_test)
print('y_pred = ', y_test.data)
plt.plot(epoch_list,loss_list)
plt.title("LBFGS(lr = 0.1)")
plt.xlabel("epoch")
plt.ylabel("loss")
plt.savefig("./data/pytorch4.png")
- Rprop:
Rprop 优化方法(弹性反向传播)，适用于 full-batch，不适用于 mini-batch，因而在 mini-batch 大行其道的时代里，很少见到。
优点：它可以自动调节学习率，不需要人为调节
缺点：仍依赖于人工设置一个全局学习率,随着迭代次数增多，学习率会越来越小，最终会趋近于0
结果：修改学习率和epoch均不能使其表现良好，无法满足1e-7精度条件下收敛
🌈我的分享也就到此结束啦🌈
如果我的分享也能对你有帮助，那就太好了！
若有不足，还请大家多多指正，我们一起学习交流！
📢未来的富豪们：点赞👍→收藏⭐→关注🔍，如果能评论下就太惊喜了！
感谢大家的观看和支持！最后，☺祝愿大家每天有钱赚！！！欢迎关注、关注！
