rdwd use cases

Berry Boessenkool, berry-b@gmx.de

2019-05-30

Vignette Rmd source code
main rdwd vignette
Interactive map vignette

Rainfall intensity depends on temperature

Clausius-Clapeyron scaling holds even for very high temperatures, we just don’t have enough data yet to have observed the expected extreme rainfall intensities.

Code (with a much older version of rdwd, might not run out of the box any more): https://github.com/brry/prectemp/blob/master/Code_analysis.R
Publication: http://www.nat-hazards-earth-syst-sci-discuss.net/nhess-2016-183

TOC

Get all hourly rainfall data 2014:2016

Step 1: get the URLS of data to be downloaded

## [1] 5930
## [1] 2142

Step 2: download the data

If some downloads fail (mostly because you’ll get kicked off the FTP server), you can just run the same code again and only the missing files will be downloaded.

If you really want to download 2k historical (large!) datasets, you definitely want to set sleep to a much higher value.

For speed, we’ll only work with the first 3 urls.

Step 3: read the data

2k large datasets probably is way too much for memory, so we’ll use a custom reading function. It will only select the relevant time section and rainfall column. The latter will be named with the id extracted from the filename.

##             Par               Kurz          Einheit
## NSH_TAG NSH_TAG    Schneehoehe_Neu               cm
## RS           RS Niederschlagshoehe               mm
## RSF         RSF  Niederschlagsform numerischer Code
## SH_TAG   SH_TAG        Schneehoehe               cm
## 'data.frame':    1090 obs. of  2 variables:
##  $ date : Date, format: "2014-01-02" "2014-01-03" ...
##  $ 00006: num  1.8 0.4 2.3 0.7 0.2 0 0 8.3 0 4 ...

Now let’s apply this to all our files and merge the result.

## 'data.frame':    1094 obs. of  4 variables:
##  $ date : Date, format: "2014-01-02" "2014-01-03" ...
##  $ 00006: num  1.8 0.4 2.3 0.7 0.2 0 0 8.3 0 4 ...
##  $ 00015: num  1.2 0.2 1.5 1.5 0 0 0 5.1 0.3 0.6 ...
##  $ 00019: num  3.3 0.4 2.9 0 0.2 0.1 0 6.3 0.2 3.1 ...
##       date                00006            00015            00019       
##  Min.   :2014-01-02   Min.   : 0.000   Min.   : 0.000   Min.   : 0.000  
##  1st Qu.:2014-10-02   1st Qu.: 0.000   1st Qu.: 0.000   1st Qu.: 0.000  
##  Median :2015-07-02   Median : 0.100   Median : 0.000   Median : 0.100  
##  Mean   :2015-07-02   Mean   : 2.142   Mean   : 1.647   Mean   : 2.152  
##  3rd Qu.:2016-03-31   3rd Qu.: 1.900   3rd Qu.: 1.500   3rd Qu.: 1.900  
##  Max.   :2016-12-30   Max.   :68.400   Max.   :54.000   Max.   :53.600  
##                       NA's   :5

TOC

step 4: visual data checks

Let’s see the locations of our stations in an interactive map.

For a static map with scaleBar, OSMscale works nicely but currently still has a Java dependency, see https://github.com/brry/OSMscale#installation

TOC

plot all rainfall values around a given point

1. Find meteo stations around a given point

##   Stations_id von_datum bis_datum Stationshoehe geoBreite geoLaenge
## 1          NA        NA        NA            NA  49.21178  9.812475
## 2        2848  19310101  20190527           458  49.25750  9.859400
## 3        5988  20180523  20190527           415  49.16380  9.916200
## 4        2787  20050101  20190527           354  49.24280  9.678600
## 5        2787  19410101  20190527           354  49.24280  9.678600
## 6        5206  20081103  20161111           396  49.11750  9.897200
##                  Stationsname         Bundesland    res           var
## 1 Braunsbach catchment center               <NA>      z          <NA>
## 2         Langenburg-Atzenrod Baden-Wuerttemberg  daily   more_precip
## 3                    Ilshofen Baden-Wuerttemberg  daily   more_precip
## 4         Kupferzell-Rechbach Baden-Wuerttemberg hourly precipitation
## 5         Kupferzell-Rechbach Baden-Wuerttemberg  daily   more_precip
## 6       Vellberg-Kleinaltdorf Baden-Wuerttemberg hourly precipitation
##          per hasfile      dist
## 1       <NA>      NA  0.000000
## 2 historical    TRUE  6.119540
## 3 historical    TRUE  9.235422
## 4 historical    TRUE 10.315219
## 5 historical    TRUE 10.315219
## 6 historical    TRUE 12.159814

Interactive map of just the meteo station locations:

TOC

2. Download and process data

Download and process data for the stations, get the rainfall sums of a particular day (Braunsbach flood May 2016):

## Warning in readDWD.data(file[i], fread = fread[i],
## varnames = varnames[i], : File contains no rows: C:/
## Dropbox/Rpack/rdwd/localtests/CreateVignettes/DWDdata/
## daily_more_precip_historical_tageswerte_RR_05988_20180523_20181231_hist.zip
## Warning in readDWD.data(file[i], fread = fread[i],
## varnames = varnames[i], : File contains no rows: C:/
## Dropbox/Rpack/rdwd/localtests/CreateVignettes/DWDdata/
## daily_more_precip_historical_tageswerte_RR_05989_20180523_20181231_hist.zip
## Warning in readDWD.data(file[i], fread = fread[i],
## varnames = varnames[i], : File contains no rows: C:/
## Dropbox/Rpack/rdwd/localtests/CreateVignettes/DWDdata/
## daily_more_precip_historical_tageswerte_RR_07233_20060915_20181231_hist.zip
## Warning in readDWD.data(file[i], fread = fread[i],
## varnames = varnames[i], : File contains no rows: C:/
## Dropbox/Rpack/rdwd/localtests/CreateVignettes/DWDdata/
## daily_more_precip_historical_tageswerte_RR_07491_20061125_20181231_hist.zip
## Warning in readDWD.data(file[i], fread = fread[i],
## varnames = varnames[i], : File contains no rows: C:/
## Dropbox/Rpack/rdwd/localtests/CreateVignettes/DWDdata/
## daily_more_precip_historical_tageswerte_RR_15011_20140501_20181231_hist.zip
## Warning in readDWD.data(file[i], fread = fread[i],
## varnames = varnames[i], : File contains no rows: C:/
## Dropbox/Rpack/rdwd/localtests/CreateVignettes/DWDdata/
## daily_more_precip_historical_tageswerte_RR_07231_20060915_20161109_hist.zip
## Warning in readDWD.data(file[i], fread = fread[i],
## varnames = varnames[i], : File contains no rows: C:/
## Dropbox/Rpack/rdwd/localtests/CreateVignettes/DWDdata/
## daily_more_precip_historical_tageswerte_RR_07495_20070114_20181231_hist.zip
##   Stations_id precsum Stationshoehe geoBreite geoLaenge
## 1        2848   105.0           458   49.2575    9.8594
## 2        5988      NA           415   49.1638    9.9162
## 3        2787    72.0           354   49.2428    9.6786
## 4        5206    82.2           396   49.1175    9.8972
## 5        2575    94.0           426   49.1804    9.9800
## 6        3416    82.5           294   49.3423    9.8073
##                     Stationsname
## 1            Langenburg-Atzenrod
## 2                       Ilshofen
## 3            Kupferzell-Rechbach
## 4          Vellberg-Kleinaltdorf
## 5 Kirchberg/Jagst-Herboldshausen
## 6                Mulfingen/Jagst

TOC

3. Plot rainfall sum on map

For a quick look without a map, this works:

But it’s nicer to have an actual map. If OSMscale installation fails, go to https://github.com/brry/OSMscale#installation

TOC

map climate data to Landkreise

Shapefile of Landkreis districts:
https://public.opendatasoft.com/explore/dataset/landkreise-in-germany/export/ (file size 4 MB, unzipped 10 MB)

a) find available meteo stations for each district

## although coordinates are longitude/latitude, st_intersects assumes that they are planar

https://gis.stackexchange.com/a/318629/36710

b) Average data per district

Running analysis for a few selected districts only to reduce computation time.
Monthly rainfall average per Landkreis.

## Warning: No rainfall data available for Landkreis 300: Offenbach
##   MESS_DATUM Hohenlohekreis Bodenseekreis Offenbach    Berlin
## 1 2017-10-15          56.25          48.4        NA 84.183333
## 2 2017-11-15         103.40          96.7        NA 69.616667
## 3 2017-12-15          90.85          94.4        NA 39.866667
## 4 2018-01-15         124.65         114.9        NA 67.816667
## 5 2018-02-15          24.75          53.3        NA  3.183333
## 6 2018-03-15          46.75          51.1        NA 47.700000

TOC

Any feedback on this package (or this vignette) is very welcome via github or !