东北大学Python

目前金属矿开采，爆破还是主要的破岩方式，为了保证巷道采场的安全，需要对爆破震动进行监测，获取的监测数据如附件，第1列数据为震动的序号，第2、3、4列为x,y,z三个方向的震动速度，往往由于各种因素如环境、采集设备故障等导致采集的数据存在缺失、异常、冗余等现象，这会降低系统分析和预测能力，因此，如何将获取“脏数据”进行科学合理的清洗是亟待解决的问题。一般说来，脏数据主要存在异常、缺失和冗余这三类，如某一行的数据不是4列，或者x,y,z缺失或者格式为字符不是数字等。对数据进行清洗后，请计算出爆破振动的合速度，建立新的文件，保存在第5列，在文件开头保存程序运行时的系统时间和程序开发者姓名，并绘制出合速度的波形图,绘制波形使用turtle库或matplotlib库。

X(S),X(cm/s),Y(cm/s),Z(cm/s)
-2023.2048,-0.040775947,-0.026564745,-0.01956255
-0.2046,-0.040775947,-0.026564745,-0.01956255
-0.2044,-0.043995101,-0.025502155,-0.023680981
-0.2042,-0.045068152,-0.027627335,-0.023680981
-0.204,-0.046141203,-0.032940284,-0.020592158
-0.204,-0.046141203,-0.032940284,-0.020592158,ttytt
110.204,-0.046141203,-0.032940284,-0.020592158
-0.204,,-0.032940284,-0.020592158
-0.204,dog,-0.032940284,-0.020592158
-0.2038,-0.048287306,-0.035065463,-0.024710589
-0.2036,-0.045068152,-0.031877694,-0.023680981
-0.2034,-0.046141203,-0.024439565,-0.018532942
-0.2032,-0.035410691,-0.023376976,-0.022651373
-0.203,-0.035410691,-0.022314386,-0.01956255
-0.2028,-0.035410691,-0.021251796,-0.020592158
-0.2026,-0.032191537,-0.021251796,-0.01956255
-0.2024,-0.033264589,-0.018064027,-0.021621766

·

·

·

实现代码如下：

import csv
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import datetime


with open("test.txt", 'w') as f:
    f.write('ID:20220000,Name:XXX,time:%s\n' % datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'))

out = open('data.csv', 'w+', newline='', encoding='utf-8')
csv_writer = csv.writer(out, dialect='excel')

f = open("data.txt", "r")
flag = 0
for line in f.readlines():
    lst = list(line.strip('\n').split(','))  # 将字符串转为列表，从而可以按单元格写入csv
    for item in lst:
        if item == 'X(S)' or item == 'X(cm/s)' or item == 'Y(cm/s)' or item == 'Z(cm/s)':
            flag = 4
            break
        elif item.islower():
            flag = 0
            break
        elif item.find('.') < 3 and item.find('.') > 0:
            flag = flag + 1
            continue

    if flag == 4:
        csv_writer.writerow(lst)
    flag = 0

# 读取数据文件
reader = pd.read_csv('data.csv', header=[0])
X = reader.iloc[:, 1]
Y = reader.iloc[:, 2]
Z = reader.iloc[:, 3]
T = reader.iloc[:, 0]
# 计算合速度
V = np.sqrt(X ** 2 + Y ** 2 + Z ** 2) * (X / abs(X))
# 将V存入第五列
reader['V'] = V
reader.to_csv('data.csv', index=False)
# 生成结果表单
out = open('result.csv', 'w+', newline='', encoding='utf-8')
csv_writer2 = csv.writer(out, dialect='excel')
f1 = open("test.txt")
for line in f1.readlines():
    lst = list(line.strip('\n').split(','))
    csv_writer2.writerow(lst)
f2 = open("data.csv")
for line in f2.readlines():
    lst = list(line.strip('\n').split(','))
    csv_writer2.writerow(lst)
# 绘制波形图
plt.figure(num=1, figsize=(8, 4))
plt.plot(T, V, color='green')
plt.xlabel('time')
plt.ylabel('velocity')
plt.title('Blasting vibration composite velocity waveform graph')
plt.grid()
plt.show()

效果：