Initializing the WRF model with ERA5 (pressure level)

sci-tech

Oct 3, 2019
2 minute read

Note

This is the tutorial of using WRF with ERA5 pressure level (38 vertical levels) data.

If you prefer model level (138 vertical levels), please check another tutorial.

I recommend the pressure level data, because the download speed is much faster.

If you don’t have cdsapi, please check this official tutorial.

Required Fields

WRF requires a number of fields as input.
3D Data (e.g. data on pressure levels)
Temperature
U and V components of Wind
Geopotential Height
Relative Humidity (the code can calculate RH if Specific Humidity is available;this is controlled in the Vtable)
2D Data
Surface Pressure
Mean Sea Level Pressure
Skin Temperature/SST
2-meter Temperature
2-meter Relative or Specific Humidity
10-meter U and V components of wind
Soil data (temperature and moisture) and soil height
If any masked field is ingested, then a LANDSEA field is recommended
Water equivalent snow depth (SNOW) is a nice field to have, but not required.
SEAICE is good to have for a high latitude winter case, but it is not required.

Single level data (GetERA5-sl.py)

import cdsapi

c = cdsapi.Client()

c.retrieve(
    'reanalysis-era5-single-levels',
    {
        'product_type':'reanalysis',
        'format':'grib',
        'variable':[
            '10m_u_component_of_wind','10m_v_component_of_wind','2m_dewpoint_temperature',
            '2m_temperature','land_sea_mask','mean_sea_level_pressure',
            'sea_ice_cover','sea_surface_temperature','skin_temperature',
            'snow_depth','soil_temperature_level_1','soil_temperature_level_2',
            'soil_temperature_level_3','soil_temperature_level_4','surface_pressure',
            'volumetric_soil_water_layer_1','volumetric_soil_water_layer_2','volumetric_soil_water_layer_3',
            'volumetric_soil_water_layer_4'
        ],
        'date':'DATE1/DATE2',
        'area':'Nort/West/Sout/East',
        'time':'00/to/23/by/1',
    },
    'ERA5-DATE1-DATE2-sl.grib')

Pressure level data (GetERA5-pl.py)

import cdsapi

c = cdsapi.Client()

c.retrieve(
    'reanalysis-era5-pressure-levels',
    {
        'product_type':'reanalysis',
        'format':'grib',
        'pressure_level':[
            '1','2','3',
            '5','7','10',
            '20','30','50',
            '70','100','125',
            '150','175','200',
            '225','250','300',
            '350','400','450',
            '500','550','600',
            '650','700','750',
            '775','800','825',
            '850','875','900',
            '925','950','975',
            '1000'
        ],
        'date':'DATE1/DATE2',
        'area':'Nort/West/Sout/East',
        'time':'00/to/23/by/1',
        'variable':[
            'geopotential','relative_humidity','specific_humidity',
            'temperature','u_component_of_wind','v_component_of_wind'
        ]
    },
    'ERA5-DATE1-DATE2-pl.grib')

Automation script (Download)

You can automate the script rather than change elements one by one.

#!/bin/bash -l

CODEDIR=/nuist/u/home/yinyan/xin/scratch/data/ERA5/code
DATADIR=/nuist/u/home/yinyan/xin/scratch/data/ERA5/data

# Set your python environment
export PATH=~/xin/work/anaconda3/bin:$PATH
source activate root
cd $CODEDIR

DATE1=20170419
DATE2=20170420
YY1=`echo $DATE1 | cut -c1-4`
MM1=`echo $DATE1 | cut -c5-6`
DD1=`echo $DATE1 | cut -c7-8`
YY2=`echo $DATE2 | cut -c1-4`
MM2=`echo $DATE2 | cut -c5-6`
DD2=`echo $DATE2 | cut -c7-8`
Nort=60
West=80
Sout=15
East=150

sed -e "s/DATE1/${DATE1}/g;s/DATE2/${DATE2}/g;s/Nort/${Nort}/g;s/West/${West}/g;s/Sout/${Sout}/g;s/East/${East}/g;" GetERA5-sl.py > GetERA5-${DATE1}-${DATE2}-sl.py

python GetERA5-${DATE1}-${DATE2}-sl.py

sed -e "s/DATE1/${DATE1}/g;s/DATE2/${DATE2}/g;s/Nort/${Nort}/g;s/West/${West}/g;s/Sout/${Sout}/g;s/East/${East}/g;" GetERA5-pl.py > GetERA5-${DATE1}-${DATE2}-pl.py

python GetERA5-${DATE1}-${DATE2}-pl.py

mkdir -p ${DATADIR}/$YY1

mv ERA5-${DATE1}-${DATE2}-sl.grib ERA5-${DATE1}-${DATE2}-pl.grib ${DATADIR}/$YY1/


exit 0

geogrid

Run geogrid.exe as usual.

If you’re not familiar with WRF, please check the online tutorial in detail.

ungrib

Run ungrib.exe by using the Vtable.ERA-interim.pl table:

$ ln -s ./ungrib/Variable_Tables/Vtable.ERA-interim.pl Vtable

metgrid

Run metgrid.exe as usual.

References

Version control

Version	Action	Time
1.0	Init	2019-10-03
1.1	Add support of selecting area	2019-10-04
1.2	Update the method of assigning ’time'	2020-03-27

ERA5

WRF-Chem

Hugo+Staticman+Travis

IELTS Notes

Say something

Thank you

Your comment has been submitted and will be published once it has been approved.

OOPS!

Your comment has not been submitted. Please go back and try again. Thank You!

If this error persists, please open an issue by clicking here.

Comments (50)

Sam
Tuesday, Mar 24, 2020

Do I need to merge both dataset before running ungrib? If not what name I should give to each? GRIBFILE.AAA and GRIBFILE.AAB?

Xin Zhang
In reply to Sam
Tuesday, Mar 24, 2020

Hi, Sam!

You don’t have to merge both datasets.

You can just use ./link_grib.csh ERF5* to link all files.

Then ungrib.exe will read all GRIBFILE.*.

Xin

Zhenning Li
Thursday, Mar 26, 2020

Hi Xin,
Thanks for your great post. Very helpful! Just one question, in your downloading py script, the specific time ranges from 00Z to 12Z, I am just wondering why 13-23Z were not selected? Thanks
Zhenning

Xin Zhang
In reply to Zhenning Li
Thursday, Mar 26, 2020

Hi Zhenning,

Thanks!

That’s just an example which downloads 12 hours data.

You can choose any time interval you like, even unequally spaced intervals.

Xin

Xin Zhang
In reply to Zhenning Li
Friday, Mar 27, 2020

Hi Zhenning,

You can use a simpler method to choose the time like this:

'time':'00/to/23/by/1',

I’ve updated that part in the blog. Thank you for your question ;)

Xin

Zhenning Li
In reply to Zhenning Li
Saturday, May 1, 2021

Thanks! Sorry I just noted you reply. Good suggestion!

Regards, Zhenning

Prerita Agarwal
Tuesday, Apr 7, 2020

Hi Xin,
What is the exact difference between using model and pressure level data?
Also, does linking the grib files to folder like ./link_grib.csh ../DATA/ERA5 links both pressure and single-level data and does not need to be done separately?
Thanks, Prerita

Xin Zhang
In reply to Prerita Agarwal
Tuesday, Apr 7, 2020

Hi Prerita,

A1:

The pressure levels are interpolated from model levels. You can check the ERA5 document in detail. Also, the definition of 137 model levels is here and the diagram.

A2: Yes, you can link both kinds of data. That’s easier to do ungrib.exe and metgrid.exe.

Cheers, Xin

Yasmin
Sunday, Apr 12, 2020

Hi Xin, With the older dataset (Era-Interim), between the steps ungrib.exe and metgrid.exe, you had to apply the calc_ecmwf_p.exe (https://metclim.ucd.ie/2018/03/wrfv3-9-1-with-era-interim-data-on-orr2/). Is that not necessary anymore?
Thank you.

Xin Zhang
In reply to Yasmin
Sunday, Apr 12, 2020

Hi Yasmin,

That step is related to the ERA5 model level data. For the pressure level data, you don’t have to calculate data manually again.

Of course, I’ve written another tutorial for the model level data. Feel free to check that ;)

Xin

Yasmin
In reply to Yasmin
Thursday, Apr 16, 2020

Ahhh.. I see Xin, Thank you for the elucidation. Which one do you recommend? Model or Pressure level data?

I am downloading the pressure data + surface data. Hopefully I can run successfully the WRF. Let’s see. Yasmin

Xin Zhang
In reply to Yasmin
Sunday, Apr 19, 2020

Hi Yasmin,

In my opinion, the pressure level data is enough. But, if you want the higher vertical resolution, such as >100 levels, or focus on upper atmosphere, you can try the model data.

Looking forward to your test ;)

Xin

Yasmin
Sunday, Apr 19, 2020

Hi again Xin, So I just downloaded my data and performed a test, however I got the error relate to the Pressure levels, I guess I need to use the calc_ecmwf_p.exe Do you know where I can find the ecmwf_coeffs table for 38 pressure levels?
WARNING: Couldn’t open file PRES:2019-11-02_00 for input. Processing 2019-11-02_06 FILE PRES

Xin Zhang
In reply to Yasmin
Monday, Apr 20, 2020

Hi Yasmin,

I’ve used the ERA5 pressure level data several times and it always works well. So, I suppose there’s something wrong in your step of ./link_grib.csh or ungrib.exe. You need to check files (GRIBFILE.AAA ……) and logs (ungrib.log) related to these two command. If these steps are successful, then you should see the FILE:** files which cover the simulation time duration.

The most possible cause is the namelist.wps, you don’t need PRES. Here’s an example:

&ungrib
 out_format                         = 'WPS',
 prefix                             = 'FILE',
 /

&metgrid
 fg_name                            = 'FILE',
 io_form_metgrid                    = 2,
 /

Xin

Yasmin
Monday, Apr 20, 2020

Hey Xin, You were right, The problem was in the namelist.wps, which I didnt remove the fg_nam = ‘PRES’ (I’ve copied it from past simulations that I did using ERA-Interim). My simulations are already running. Thank you very much. Hopefully this dataset will provide results as good as the Era-Interim =)

Xin Zhang
In reply to Yasmin
Tuesday, Apr 21, 2020

Great! Looking forward to your new results ;)

Xin

Guilherme
Tuesday, Apr 21, 2020

Hi Xin, After i run the command “calc_ecmwf_p.exe”, the following error appears:
ERROR: Error opening ecmwf_coeffs Reading from FILE at time 2016-10-17_00 Found PSFC field in FILE:2016-10-17_00 WARNING: Either TT or SPECHUMD not found. No RH will be computed… … … … Any idea? Thank you!

Xin Zhang
In reply to Guilherme
Tuesday, Apr 21, 2020

Hi,

Note that the step of calc_ecmwf_p.exe is necessary for model level data, not pressure data.

If you’re using model level data, you need to create ecmwf_coeffs by yourself. Please check my another tutorial.

Xin

Stefan Rahimi
Friday, Jun 26, 2020

Dear Xin,
Thank you for this wonderful tutorial. I just have one question.
Where are you downloading these data from? ERA5 data are on the CISL RDA (https://rda.ucar.edu/datasets/ds630.0/), but the data are not in discrete time slices as required by ungrib.exe. I am planning to downscale ERA5 for a 40-year period, so I am trying to find grb files that are date-time specific…
It looks like the above scripts nicely subset the data and pull relevant variables into new grb files?
Thanks, -Stefan Rahimi, UCLA

Xin Zhang
In reply to Stefan Rahimi
Saturday, Jun 27, 2020

Dear Stefan,

The resource is from CDS which contains ERA5 Pressure level data and Single level data.

Since you need 40-year data, you can choose either method: using the above scripts or just clicking the boxes on the CDS web page.

Regards, Xin

bara
Friday, Jun 26, 2020

Hi Xin Thank you for your nice tutorial. I have followed all steps ,but I did not get the verticale levels for 4-dimensional variables like relative humidity (RH) and temperature (TT)) in my met file ( met_em.d01.2018-06-28_18:00:00.nc example)
How to solve this problem?
bara

Xin Zhang
In reply to bara
Saturday, Jun 27, 2020

Hi Bara,

That’s strange …

Could you check whether RH and TT exist in the pressure level grib files you have downloaded?

BTW, it;s better to make sure there’s no error in the log files (ungrib.log and metgrid.log).

Xin

Bara
In reply to bara
Saturday, Jun 27, 2020

Hi Xin! Thank you for your quick reply I checked the downloaded files but they contain pressure levels for UU, TT …. But the problem in coming from ungrib.exe and metgrid.exe. Files created by these both executables have not pressure levels. Do you have any idea please

Xin Zhang
In reply to bara
Sunday, Jun 28, 2020

Hi Bara,

Could you send the log files to my [email](mailto: xinzhang1215@gmail.com)? (If the generated files aren’t big, you can send them too) So, I can check the details …

Xin

Bara
Tuesday, Jun 30, 2020

Hi Xin, I really appreciated your collaboration. Finally, I have solved the problem by adding the following code to the 38 pressure levels:
echo ‘write “[centre][dataDate][dataType][levelType][step].grib[edition]”;’ > split.rule grib_filter split.rule ERA5-${DATE1}-${DATE2}-sfc.grb grib_set -s deletePV=1,edition=1 ERA5-${DATE1}-${DATE2}-ml.grb ERA5-${DATE1}-${DATE2}-ml.grib1 grib_filter split.rule ERA5-${DATE1}-${DATE2}-ml.grib1
This code is the last step for bash program for your 138 model levels tutorial.
Bara

Xin Zhang
In reply to Bara
Tuesday, Jun 30, 2020

Ha, nice!

I thought that step was optional for pressure level data and never used that before …

Anyway, I’m glad you solved it ;)

Xin

Chi
Friday, Jul 3, 2020

Hi Xin, thanks for your tutorial, and I have some questions. I’m ungribbing ERA-Interim data(pressure level), can I use SH, 2m T and 2m dewpoint T to calculate RH by adding SPECHUMD to the Vtable.ECMWF? I noticed this RH data are different from ungribbing it directly with ERA-Interim RH data in met files, what could be the problem?
Chi

Xin Zhang
In reply to Chi
Friday, Jul 3, 2020

Hi Chi,

This post is for ERA5 dataset.

If you’re using ERA-Interim data, please check my another post.

Xin

Bart Brashers
Sunday, Jul 5, 2020

Would it be beneficial to add ‘geopotential’ to the GetERA5-sl.py part of the code? That would download GRIB code=129, level code=1 (surface), which WRF uses to construct SOILGEO and SOILHGT. [url]https://confluence.ecmwf.int/pages/viewpage.action?pageId=82870405#ERA5:datadocumentation-Table1[/url] says it’s available.
Back when I first stated using ERA-Interim for WRF I constructed an invariant SOILGEO file, so why not for ERA5?

Xin Zhang
In reply to Bart Brashers
Monday, Jul 6, 2020

Thanks, Bart. Nice question!

I can’t find the soil height variable in the ERA5 dataset …

Have you found it on the CDS website?

Regards,

Xin

Du Xin-guan
Monday, Jul 6, 2020

Dear Xin,
When i do ungrib.exe, I can ungrib pressure-level ERA5 data, but I can not ungrib surface ERA5 data. It was said that ‘was expecting a Grib1 file , but this is a Grib2 file. It is possible this is because your GRIBFILE.XXX files are not all of the same type. '
I will appreciate your any suggestions.
Du

Jason
Friday, Aug 7, 2020

Hi Xin! your tutorial did help me a lot! I used WRFdomainWizard to run WRF, and I found that, in the namelist.wps, the levels seem to be strange:
&mod_levs press_pa = 201300 , 200100 , 100000 , 95000 , 90000 , 85000 , 80000 , 75000 , 70000 , 65000 , 60000 , 55000 , 50000 , 45000 , 40000 , 35000 , 30000 , 25000 , 20000 , 15000 , 10000 , 5000 , 1000
If I don’t use WRFdomainWizard to run WRF, the original namelist.wps doesn’t have content about &mod_levs. Should I set this before run WPS? If I should set this, what array should I input (pressure data)?

Sumitra
Friday, Aug 7, 2020

HI Xin! I followed the steps you mentioned in this post, now I am getting error while running the command ./real.exe. ————– FATAL CALLED ————— FATAL CALLED FROM FILE: LINE: 3388 mismatch_landmask_ivgtyp
Thank you

pyadav
In reply to Sumitra
Monday, Aug 31, 2020

hi xin this error i am also getting ‘mismatch_landmask_ivgtyp’, while real run please reply

Josué Arellano Palacios
Wednesday, Sep 9, 2020

Hi Xin! How are you? first thanks for the tutorial, so I have followed all steps and when I do wrf.exe I get this:
FATAL CALLED FROM FILE: LINE: 242 program wrf: error opening wrfrst_d01_2001-10-01_00:00:00 for reading
What could have gone wrong?

Xin Zhang
In reply to Josué Arellano Palacios
Wednesday, Sep 9, 2020

Hi Josué,

This is caused by the restart option in the namelist.input file.

If you don’t want restart, you can set it to False. Otherwise, you have to set the suitable restart_interval for the restart.

You can check the official WRF USER GUIDE in detail.

Regards,

Xin

Josué Arellano Palacios
In reply to Josué Arellano Palacios
Friday, Sep 18, 2020

Thanks for your help, it was very useful! Your tutorial is great!

Best,

Josué

He Ming
Tuesday, Oct 20, 2020

hi Xin~ i have meet some troubles when i wrong WRF3.8. i followed all of the steps to download ERA5 data and it wroks. But it only runs several seconds and then stopped after i submit the job. i check .err file and found: mpirun has exited due to process rank 13 with PID 60264 on node h2ocn16 exiting improperly. There are two reasons this could occur:
this process did not call “init” before exiting, but others in the job did. This can cause a job to hang indefinitely while it waits for all processes to call “init”. By rule, if one process calls “init”, then ALL processes must call “init” prior to termination.
this process called “init”, but exited without calling “finalize”. By rule, all processes that call “init” MUST call “finalize” prior to exiting or it will be considered an “abnormal termination”
This may have caused other processes in the application to be terminated by signals sent by mpirun (as reported here).
how can fix this problem ? thanks !

Xin Zhang
In reply to He Ming
Thursday, Oct 22, 2020

Hi, He Ming,

You can check the detailed log (rsl.error.****) related to that specific node (node h2ocn16).

That should contain the information you want.

Anyway, it seems not the problem of ERA5 data, but the setting of WRF.

Xin

Divyansh Chug
Thursday, Oct 22, 2020

Thank you so much Xin Zhang for documenting this process!

Xin Zhang
In reply to Divyansh Chug
Sunday, Oct 25, 2020

You’re welcome, Divyansh Chug.

I’m glad that this can help you ;)

joseph
Friday, Nov 27, 2020

Do you have python script to download monthly averaged reanalysis? Thanks.

Xin Zhang
In reply to joseph
Friday, Nov 27, 2020

I haven’t used the monthly data before.

But, you can generate the official script on the CDS page and then write your own based on that.

Regards, Xin

Peng
Friday, Dec 25, 2020

Hi Xin, Thank you very much for this wonderful tutorial! Does ERA5 provide a forecast field data with which I can do a forecast rather than a simulation? I will set the forecast field data as the boundary forcing data.

Xin Zhang
In reply to Peng
Saturday, Dec 26, 2020

Hi Peng,

You can check the free ERA5 products here.

I suppose the real-time forecast data isn’t freely available.

Regards,

Xin

Peng
In reply to Peng
Saturday, Dec 26, 2020

Thank you very much for your reply.

Kind regards, Peng

peng
Thursday, Dec 31, 2020

Hi Xin,
Sorry to bother you again. Thanks very much for your wonderful tutorial to enable me to run WRF successfully with ERA5 reanalysis data. But error happened when I used the forecast field data as the boundary forcing condition. Error is as follows when running the ungrib.exe,
“Unknown Data Representation Type, ksec2(4)= 50 "
Do you have any idea to solve this problem?
Happy new year! Peng

Vinayak
Sunday, Apr 11, 2021

Hi Xin, I came across your very informative blog recently. I wanted to know whether we can use ERA-Interim Subsets to force WRF. When I tried forcing the subset from rda ucar website. metgrid.exe throws “Gaussian Latittude Computation Error”. Other blogs are suggesting to download global data which is humongous and takes a lot of time. I want simulations only on my study region. Can you suggest any alternative or detailed tutorial to use ERA-INterim subsets with WRF??

Xin Zhang
In reply to Vinayak
Tuesday, Apr 13, 2021

Hi Vinayak,

You can check my other tutorial about ERA-Interim. Hope that can help you.

Xin

Rocio
Tuesday, May 11, 2021

Hi! I’m having this error after I run the metgrid.exe. Do you know what could be the reason?
ARNING: Field PRES has missing values at level 200100 at (i,j)=(1,1) WARNING: Field PMSL has missing values at level 200100 at (i,j)=(1,1) WARNING: Field PSFC has missing values at level 200100 at (i,j)=(1,1)
Regards, Rocío