整理一下平时经常会使用的cdo命令
如何来更改netcdf数据中的变量名呢？
假设我现在有一个sst月平均数据,希望将里面的变量名称sst修改为sst_new
netcdf oisst_monthly {
dimensions:
lat = 180 ;
lon = 360 ;
time = UNLIMITED ; // (476 currently)
nbnds = 2 ;
variables:
float lat(lat) ;
lat:units = "degrees_north" ;
lat:long_name = "Latitude" ;
lat:actual_range = 89.5f, -89.5f ;
lat:standard_name = "latitude" ;
lat:axis = "Y" ;
lat:coordinate_defines = "center" ;
float lon(lon) ;
lon:units = "degrees_east" ;
lon:long_name = "Longitude" ;
lon:actual_range = 0.5f, 359.5f ;
lon:standard_name = "longitude" ;
lon:axis = "X" ;
lon:coordinate_defines = "center" ;
short sst(time, lat, lon) ;
sst:long_name = "Monthly Mean of Sea Surface Temperature" ;
sst:unpacked_valid_range = -5.f, 40.f ;
sst:actual_range = -1.8f, 35.56862f ;
sst:units = "degC" ;
sst:add_offset = 0.f ;
sst:scale_factor = 0.01f ;
sst:missing_value = 32767s ;
sst:precision = 2s ;
sst:least_significant_digit = 2s ;
sst:var_desc = "Sea Surface Temperature" ;
sst:dataset = "NOAA Optimum Interpolation (OI) SST V2" ;
sst:level_desc = "Surface" ;
sst:statistic = "Mean" ;
sst:parent_stat = "Weekly Mean" ;
sst:standard_name = "sea_surface_temperature" ;
sst:cell_methods = "time: mean (monthly from weekly values interpolated to daily)" ;
sst:valid_range = -500s, 4000s ;
使用nco
ncrename -v sst,sst_new oisst_monthly.nc
修改结果,修改成功了
ncdump -h oisst_monthly.nc
netcdf oisst_monthly {
dimensions:
lat = 180 ;
lon = 360 ;
time = UNLIMITED ; // (476 currently)
nbnds = 2 ;
variables:
float lat(lat) ;
lat:units = "degrees_north" ;
lat:long_name = "Latitude" ;
lat:actual_range = 89.5f, -89.5f ;
lat:standard_name = "latitude" ;
lat:axis = "Y" ;
lat:coordinate_defines = "center" ;
float lon(lon) ;
lon:units = "degrees_east" ;
lon:long_name = "Longitude" ;
lon:actual_range = 0.5f, 359.5f ;
lon:standard_name = "longitude" ;
lon:axis = "X" ;
lon:coordinate_defines = "center" ;
short sst_new(time, lat, lon) ;
sst_new:long_name = "Monthly Mean of Sea Surface Temperature" ;
sst_new:unpacked_valid_range = -5.f, 40.f ;
sst_new:actual_range = -1.8f, 35.56862f ;
sst_new:units = "degC" ;
sst_new:add_offset = 0.f ;
sst_new:scale_factor = 0.01f ;
sst_new:missing_value = 32767s ;
sst_new:precision = 2s ;
sst_new:least_significant_digit = 2s ;
sst_new:var_desc = "Sea Surface Temperature" ;
sst_new:dataset = "NOAA Optimum Interpolation (OI) SST V2" ;
sst_new:level_desc = "Surface" ;
sst_new:statistic = "Mean" ;
sst_new:parent_stat = "Weekly Mean" ;
sst_new:standard_name = "sea_surface_temperature" ;
sst_new:cell_methods = "time: mean (monthly from weekly values interpolated to daily)" ;
sst_new:valid_range = -500s, 4000s ;
当然，使用nco还可以修改属性的名称；下面是一个将 netcdf 文件中的 miss_value和 _FillValue 更改为零的示例:
ncatted -O -a missing_value, MITVAR,o,f,0 dasilva94_monthly_sst.nc
ncatted -O -a _FillValue, MITVAR,o,f,0 dasilva94_monthly_sst.nc
其中，MITVAR是变量名，o 表示覆盖; f 表示浮点类型; 0是新值。
使用cdo
在刚刚修改为sst_new的基础上，使用cdo将其修改为sst_renew
cdo chname,sst_new,sst_renew oisst_monthly.nc oisst_monthly_renew.nc
修改结果
ncdump -h oisst_monthly_renew.nc ✔ anaconda3 95% hdd 16:49:39
netcdf oisst_monthly_renew {
dimensions:
time = UNLIMITED ; // (476 currently)
bnds = 2 ;
lon = 360 ;
lat = 180 ;
variables:
double time(time) ;
time:standard_name = "time" ;
time:long_name = "Time" ;
time:bounds = "time_bnds" ;
time:units = "days since 1800-1-1 00:00:00" ;
time:calendar = "standard" ;
time:axis = "T" ;
double time_bnds(time, bnds) ;
float lon(lon) ;
lon:standard_name = "longitude" ;
lon:long_name = "Longitude" ;
lon:units = "degrees_east" ;
lon:axis = "X" ;
float lat(lat) ;
lat:standard_name = "latitude" ;
lat:long_name = "Latitude" ;
lat:units = "degrees_north" ;
lat:axis = "Y" ;
short sst_renew(time, lat, lon) ;
sst_renew:standard_name = "sea_surface_temperature" ;
sst_renew:long_name = "Monthly Mean of Sea Surface Temperature" ;
sst_renew:units = "degC" ;
sst_renew:add_offset = 0.f ;
sst_renew:scale_factor = 0.01f ;
sst_renew:_FillValue = 32767s ;
sst_renew:missing_value = 32767s ;
sst_renew:unpacked_valid_range = -5.f, 40.f ;
sst_renew:actual_range = -1.8f, 35.56862f ;
sst_renew:precision = 2s ;
sst_renew:least_significant_digit = 2s ;
sst_renew:var_desc = "Sea Surface Temperature" ;
sst_renew:dataset = "NOAA Optimum Interpolation (OI) SST V2" ;
sst_renew:level_desc = "Surface" ;
sst_renew:statistic = "Mean" ;
sst_renew:parent_stat = "Weekly Mean" ;
sst_renew:cell_methods = "time: mean (monthly from weekly values interpolated to daily)" ;
sst_renew:institution = "NCEP" ;
- 经过测试，对于变量名称的修改，cdo的速度要明显快于nco(在我这个测试个例里面);当然，nco的修改结果还是在原本的数据里面，而cdo的修改结果保留在一个新的nc文件里面
如何使用wget从ftp或者网页直接下载数据呢？
- 通过写一个
shell脚本来实现
Index of /data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr
- https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/
一个完整的数据为：https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/202404/oisst-avhrr-v02r01.20240401.nc
可以发现其链接的组成很有规则，下载一个文件时，我们可以直接wget https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/202404/oisst-avhrr-v02r01.20240401.nc
但是如果下载多个文件呢，就可以根据它有规则命名的属性来进行循环迭代下载
#!/bin/bash
# 循环遍历每个年份
for year in {2021..2021}; do
# 循环遍历每个月份
for month in {01..12}; do
# 循环遍历每一天
for day in {01..31}; do
# 构建文件名
filename="oisst-avhrr-v02r01.${year}${month}${day}.nc"
# 构建下载链接
url="https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/${year}${month}/${filename}"
# 使用 wget 下载文件
wget -o ./get -c -nH -c -r -A nc "$url"
done
done
done
如何对于多个netcdf文件进行求和计算呢？
cdo enssum file1 file2 out.nc
或者
cdo enssum *_sst.nc out.nc
如何对于netcdf数据进行插值呢？
- 守恒插值 (适用于降水数据)
cdo remapcon,r180x90 pr.nc pr_regrid_2x2.nc
- 双线性插值
cdo remapbic,r144x73 in.nc out_interp.nc
如何修改数据calendar的类型呢？
cdo setcalendar,standard ua_day_EC-Earth3_data.nc ua_day_EC-Earth3_data_stand.nc
如何合并多个netcdf文件呢?
将多个nc数据按照时间维度进行合并，屡试不爽的方法。当然最好是保证要合并的数据都在同一个文件夹内。
cdo mergetime GPM*.nc GPM-2004-05-07.nc
- 也可以通过python基于xr.concat()进行合并，当时还是cdo一行命令好使呀。虽然合并后的数据可能会偏大。
如何对于netcdf进行时间滤波呢？
- 对于数据是月平均来说，如果要进行10个月的低通滤波，可以使用以下命令：
cdo lowpass,0.1 sst.anomaly.nc out.nc
- 对于数据是日平均来说，如果要进行30-60天的带通滤波
cdo bandpass,365/60,365/30 sst.day.ano.nc out.nc
注意数据中不能存在nan值或者缺测值
如何处理数据中存在的nan值或者缺测值？
- 将正常的数据转换为Nan值
cdo setmissval,nan input.nc output.nc
- 将Nan值转化为其他值
cdo setmissval,0 input.nc output.nc
https://dwang.wordpress.com/
https://code.mpimet.mpg.de/boards/2/topics/4057
https://code.mpimet.mpg.de/boards/2/topics/11084
python & cdo ：https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo%7Brbpy%7D
cdo使用手册：http://www.idris.fr/media/ada/cdo.pdf
