Introduction to Cache

Eliot J. B. McIntire

January 28 2019

1 Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

1.1 Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

1.2 Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmpQuowAF/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   2.301   0.193   2.499
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   2.457   0.197   2.810
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.091   0.003   0.094
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.027   0.000   0.027
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

1.3 Caching examples

1.3.1 Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 16)  # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 6) # different

1.4 Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 0243ae58075d99364077224d53b7c96c         format
##  2: 0243ae58075d99364077224d53b7c96c           name
##  3: 0243ae58075d99364077224d53b7c96c          class
##  4: 0243ae58075d99364077224d53b7c96c           date
##  5: 0243ae58075d99364077224d53b7c96c        cacheId
##  6: 0243ae58075d99364077224d53b7c96c     objectName
##  7: 0243ae58075d99364077224d53b7c96c       function
##  8: 0243ae58075d99364077224d53b7c96c    object.size
##  9: 0243ae58075d99364077224d53b7c96c       accessed
## 10: 0243ae58075d99364077224d53b7c96c otherFunctions
## 11: 0243ae58075d99364077224d53b7c96c otherFunctions
## 12: 0243ae58075d99364077224d53b7c96c otherFunctions
## 13: 0243ae58075d99364077224d53b7c96c otherFunctions
## 14: 0243ae58075d99364077224d53b7c96c otherFunctions
## 15: 0243ae58075d99364077224d53b7c96c otherFunctions
## 16: 0243ae58075d99364077224d53b7c96c otherFunctions
## 17: 0243ae58075d99364077224d53b7c96c otherFunctions
## 18: 0243ae58075d99364077224d53b7c96c otherFunctions
## 19: 0243ae58075d99364077224d53b7c96c otherFunctions
## 20: 0243ae58075d99364077224d53b7c96c      preDigest
## 21: 0243ae58075d99364077224d53b7c96c      preDigest
## 22: d2a8bd53f23c9f193b2f448c9e1b15b4         format
## 23: d2a8bd53f23c9f193b2f448c9e1b15b4           name
## 24: d2a8bd53f23c9f193b2f448c9e1b15b4          class
## 25: d2a8bd53f23c9f193b2f448c9e1b15b4           date
## 26: d2a8bd53f23c9f193b2f448c9e1b15b4        cacheId
## 27: d2a8bd53f23c9f193b2f448c9e1b15b4     objectName
## 28: d2a8bd53f23c9f193b2f448c9e1b15b4       function
## 29: d2a8bd53f23c9f193b2f448c9e1b15b4    object.size
## 30: d2a8bd53f23c9f193b2f448c9e1b15b4       accessed
## 31: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 32: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 33: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 34: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 35: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 36: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 37: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 38: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 39: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 40: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 41: d2a8bd53f23c9f193b2f448c9e1b15b4      preDigest
## 42: d2a8bd53f23c9f193b2f448c9e1b15b4      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:05
##  2: 0243ae58075d99364077224d53b7c96c 2019-01-28 11:00:05
##  3:                          numeric 2019-01-28 11:00:05
##  4:              2019-01-28 11:00:05 2019-01-28 11:00:05
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:05
##  6:                                a 2019-01-28 11:00:05
##  7:                            rnorm 2019-01-28 11:00:05
##  8:                             1008 2019-01-28 11:00:05
##  9:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 10:                 vweave_rmarkdown 2019-01-28 11:00:05
## 11:                     process_file 2019-01-28 11:00:05
## 12:                    process_group 2019-01-28 11:00:05
## 13:              process_group.block 2019-01-28 11:00:05
## 14:                       call_block 2019-01-28 11:00:05
## 15:                       block_exec 2019-01-28 11:00:05
## 16:                           in_dir 2019-01-28 11:00:05
## 17:                        timing_fn 2019-01-28 11:00:05
## 18:                           handle 2019-01-28 11:00:05
## 19:                      withVisible 2019-01-28 11:00:05
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:05
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:05
## 22:                              rda 2019-01-28 11:00:05
## 23: d2a8bd53f23c9f193b2f448c9e1b15b4 2019-01-28 11:00:05
## 24:                          numeric 2019-01-28 11:00:06
## 25:              2019-01-28 11:00:05 2019-01-28 11:00:06
## 26:                 3aef38d1fc02aee5 2019-01-28 11:00:06
## 27:                                b 2019-01-28 11:00:06
## 28:                            runif 2019-01-28 11:00:06
## 29:                             1008 2019-01-28 11:00:06
## 30:              2019-01-28 11:00:05 2019-01-28 11:00:06
## 31:                 vweave_rmarkdown 2019-01-28 11:00:06
## 32:                     process_file 2019-01-28 11:00:06
## 33:                    process_group 2019-01-28 11:00:06
## 34:              process_group.block 2019-01-28 11:00:06
## 35:                       call_block 2019-01-28 11:00:06
## 36:                       block_exec 2019-01-28 11:00:06
## 37:                           in_dir 2019-01-28 11:00:06
## 38:                        timing_fn 2019-01-28 11:00:06
## 39:                           handle 2019-01-28 11:00:06
## 40:                      withVisible 2019-01-28 11:00:06
## 41:               n:7eef4eae85fd9229 2019-01-28 11:00:06
## 42:            .FUN:881ec847b7161f3c 2019-01-28 11:00:06
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 0243ae58075d99364077224d53b7c96c         format
##  2: 0243ae58075d99364077224d53b7c96c           name
##  3: 0243ae58075d99364077224d53b7c96c          class
##  4: 0243ae58075d99364077224d53b7c96c           date
##  5: 0243ae58075d99364077224d53b7c96c        cacheId
##  6: 0243ae58075d99364077224d53b7c96c     objectName
##  7: 0243ae58075d99364077224d53b7c96c       function
##  8: 0243ae58075d99364077224d53b7c96c    object.size
##  9: 0243ae58075d99364077224d53b7c96c       accessed
## 10: 0243ae58075d99364077224d53b7c96c otherFunctions
## 11: 0243ae58075d99364077224d53b7c96c otherFunctions
## 12: 0243ae58075d99364077224d53b7c96c otherFunctions
## 13: 0243ae58075d99364077224d53b7c96c otherFunctions
## 14: 0243ae58075d99364077224d53b7c96c otherFunctions
## 15: 0243ae58075d99364077224d53b7c96c otherFunctions
## 16: 0243ae58075d99364077224d53b7c96c otherFunctions
## 17: 0243ae58075d99364077224d53b7c96c otherFunctions
## 18: 0243ae58075d99364077224d53b7c96c otherFunctions
## 19: 0243ae58075d99364077224d53b7c96c otherFunctions
## 20: 0243ae58075d99364077224d53b7c96c      preDigest
## 21: 0243ae58075d99364077224d53b7c96c      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:05
##  2: 0243ae58075d99364077224d53b7c96c 2019-01-28 11:00:05
##  3:                          numeric 2019-01-28 11:00:05
##  4:              2019-01-28 11:00:05 2019-01-28 11:00:05
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:05
##  6:                                a 2019-01-28 11:00:05
##  7:                            rnorm 2019-01-28 11:00:05
##  8:                             1008 2019-01-28 11:00:05
##  9:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 10:                 vweave_rmarkdown 2019-01-28 11:00:05
## 11:                     process_file 2019-01-28 11:00:05
## 12:                    process_group 2019-01-28 11:00:05
## 13:              process_group.block 2019-01-28 11:00:05
## 14:                       call_block 2019-01-28 11:00:05
## 15:                       block_exec 2019-01-28 11:00:05
## 16:                           in_dir 2019-01-28 11:00:05
## 17:                        timing_fn 2019-01-28 11:00:05
## 18:                           handle 2019-01-28 11:00:05
## 19:                      withVisible 2019-01-28 11:00:05
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:05
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:05
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: d2a8bd53f23c9f193b2f448c9e1b15b4         format
##  2: d2a8bd53f23c9f193b2f448c9e1b15b4           name
##  3: d2a8bd53f23c9f193b2f448c9e1b15b4          class
##  4: d2a8bd53f23c9f193b2f448c9e1b15b4           date
##  5: d2a8bd53f23c9f193b2f448c9e1b15b4        cacheId
##  6: d2a8bd53f23c9f193b2f448c9e1b15b4     objectName
##  7: d2a8bd53f23c9f193b2f448c9e1b15b4       function
##  8: d2a8bd53f23c9f193b2f448c9e1b15b4    object.size
##  9: d2a8bd53f23c9f193b2f448c9e1b15b4       accessed
## 10: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 11: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 12: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 13: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 14: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 15: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 16: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 17: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 18: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 19: d2a8bd53f23c9f193b2f448c9e1b15b4 otherFunctions
## 20: d2a8bd53f23c9f193b2f448c9e1b15b4      preDigest
## 21: d2a8bd53f23c9f193b2f448c9e1b15b4      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:05
##  2: d2a8bd53f23c9f193b2f448c9e1b15b4 2019-01-28 11:00:05
##  3:                          numeric 2019-01-28 11:00:06
##  4:              2019-01-28 11:00:05 2019-01-28 11:00:06
##  5:                 3aef38d1fc02aee5 2019-01-28 11:00:06
##  6:                                b 2019-01-28 11:00:06
##  7:                            runif 2019-01-28 11:00:06
##  8:                             1008 2019-01-28 11:00:06
##  9:              2019-01-28 11:00:05 2019-01-28 11:00:06
## 10:                 vweave_rmarkdown 2019-01-28 11:00:06
## 11:                     process_file 2019-01-28 11:00:06
## 12:                    process_group 2019-01-28 11:00:06
## 13:              process_group.block 2019-01-28 11:00:06
## 14:                       call_block 2019-01-28 11:00:06
## 15:                       block_exec 2019-01-28 11:00:06
## 16:                           in_dir 2019-01-28 11:00:06
## 17:                        timing_fn 2019-01-28 11:00:06
## 18:                           handle 2019-01-28 11:00:06
## 19:                      withVisible 2019-01-28 11:00:06
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:06
## 21:            .FUN:881ec847b7161f3c 2019-01-28 11:00:06
##                             tagValue         createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##                             artifact         tagKey
##  1: 0243ae58075d99364077224d53b7c96c         format
##  2: 0243ae58075d99364077224d53b7c96c           name
##  3: 0243ae58075d99364077224d53b7c96c          class
##  4: 0243ae58075d99364077224d53b7c96c           date
##  5: 0243ae58075d99364077224d53b7c96c        cacheId
##  6: 0243ae58075d99364077224d53b7c96c     objectName
##  7: 0243ae58075d99364077224d53b7c96c       function
##  8: 0243ae58075d99364077224d53b7c96c    object.size
##  9: 0243ae58075d99364077224d53b7c96c       accessed
## 10: 0243ae58075d99364077224d53b7c96c otherFunctions
## 11: 0243ae58075d99364077224d53b7c96c otherFunctions
## 12: 0243ae58075d99364077224d53b7c96c otherFunctions
## 13: 0243ae58075d99364077224d53b7c96c otherFunctions
## 14: 0243ae58075d99364077224d53b7c96c otherFunctions
## 15: 0243ae58075d99364077224d53b7c96c otherFunctions
## 16: 0243ae58075d99364077224d53b7c96c otherFunctions
## 17: 0243ae58075d99364077224d53b7c96c otherFunctions
## 18: 0243ae58075d99364077224d53b7c96c otherFunctions
## 19: 0243ae58075d99364077224d53b7c96c otherFunctions
## 20: 0243ae58075d99364077224d53b7c96c      preDigest
## 21: 0243ae58075d99364077224d53b7c96c      preDigest
## 22: 3359459fba6fc1cba12846ef314ae4c2         format
## 23: 3359459fba6fc1cba12846ef314ae4c2           name
## 24: 3359459fba6fc1cba12846ef314ae4c2          class
## 25: 3359459fba6fc1cba12846ef314ae4c2           date
## 26: 3359459fba6fc1cba12846ef314ae4c2        cacheId
## 27: 3359459fba6fc1cba12846ef314ae4c2       function
## 28: 3359459fba6fc1cba12846ef314ae4c2    object.size
## 29: 3359459fba6fc1cba12846ef314ae4c2       accessed
## 30: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 31: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 32: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 33: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 34: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 35: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 36: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 37: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 38: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 39: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 40: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 41: 3359459fba6fc1cba12846ef314ae4c2 otherFunctions
## 42: 3359459fba6fc1cba12846ef314ae4c2      preDigest
## 43: 3359459fba6fc1cba12846ef314ae4c2      preDigest
## 44: 3359459fba6fc1cba12846ef314ae4c2      preDigest
## 45: 4668c420748e5459c55223e926a0483b         format
## 46: 4668c420748e5459c55223e926a0483b           name
## 47: 4668c420748e5459c55223e926a0483b          class
## 48: 4668c420748e5459c55223e926a0483b           date
## 49: 4668c420748e5459c55223e926a0483b        cacheId
## 50: 4668c420748e5459c55223e926a0483b       function
## 51: 4668c420748e5459c55223e926a0483b    object.size
## 52: 4668c420748e5459c55223e926a0483b       accessed
## 53: 4668c420748e5459c55223e926a0483b otherFunctions
## 54: 4668c420748e5459c55223e926a0483b otherFunctions
## 55: 4668c420748e5459c55223e926a0483b otherFunctions
## 56: 4668c420748e5459c55223e926a0483b otherFunctions
## 57: 4668c420748e5459c55223e926a0483b otherFunctions
## 58: 4668c420748e5459c55223e926a0483b otherFunctions
## 59: 4668c420748e5459c55223e926a0483b otherFunctions
## 60: 4668c420748e5459c55223e926a0483b otherFunctions
## 61: 4668c420748e5459c55223e926a0483b otherFunctions
## 62: 4668c420748e5459c55223e926a0483b otherFunctions
## 63: 4668c420748e5459c55223e926a0483b      preDigest
## 64: 4668c420748e5459c55223e926a0483b      preDigest
## 65: 4668c420748e5459c55223e926a0483b      preDigest
## 66: 4668c420748e5459c55223e926a0483b       accessed
## 67: 4668c420748e5459c55223e926a0483b       accessed
## 68: 4668c420748e5459c55223e926a0483b       accessed
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:05
##  2: 0243ae58075d99364077224d53b7c96c 2019-01-28 11:00:05
##  3:                          numeric 2019-01-28 11:00:05
##  4:              2019-01-28 11:00:05 2019-01-28 11:00:05
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:05
##  6:                                a 2019-01-28 11:00:05
##  7:                            rnorm 2019-01-28 11:00:05
##  8:                             1008 2019-01-28 11:00:05
##  9:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 10:                 vweave_rmarkdown 2019-01-28 11:00:05
## 11:                     process_file 2019-01-28 11:00:05
## 12:                    process_group 2019-01-28 11:00:05
## 13:              process_group.block 2019-01-28 11:00:05
## 14:                       call_block 2019-01-28 11:00:05
## 15:                       block_exec 2019-01-28 11:00:05
## 16:                           in_dir 2019-01-28 11:00:05
## 17:                        timing_fn 2019-01-28 11:00:05
## 18:                           handle 2019-01-28 11:00:05
## 19:                      withVisible 2019-01-28 11:00:05
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:05
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:05
## 22:                              rda 2019-01-28 11:00:05
## 23: 3359459fba6fc1cba12846ef314ae4c2 2019-01-28 11:00:05
## 24:                          numeric 2019-01-28 11:00:05
## 25:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 26:                 c16f0e2c57596069 2019-01-28 11:00:05
## 27:              rnorm pipe sequence 2019-01-28 11:00:05
## 28:                             1104 2019-01-28 11:00:05
## 29:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 30:                 vweave_rmarkdown 2019-01-28 11:00:05
## 31:                     process_file 2019-01-28 11:00:05
## 32:                    process_group 2019-01-28 11:00:05
## 33:              process_group.block 2019-01-28 11:00:05
## 34:                       call_block 2019-01-28 11:00:05
## 35:                       block_exec 2019-01-28 11:00:05
## 36:                           in_dir 2019-01-28 11:00:05
## 37:                        timing_fn 2019-01-28 11:00:05
## 38:                           handle 2019-01-28 11:00:05
## 39:                      withVisible 2019-01-28 11:00:05
## 40:                              %C% 2019-01-28 11:00:05
## 41:                          do.call 2019-01-28 11:00:05
## 42:               n:c5775c3b366fb719 2019-01-28 11:00:05
## 43:            mean:152602b8ff81e5bb 2019-01-28 11:00:05
## 44:            .FUN:4f604aa46882b368 2019-01-28 11:00:05
## 45:                              rda 2019-01-28 11:00:05
## 46: 4668c420748e5459c55223e926a0483b 2019-01-28 11:00:05
## 47:                          numeric 2019-01-28 11:00:05
## 48:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 49:                 f11fb1a2880f8060 2019-01-28 11:00:05
## 50:                            rnorm 2019-01-28 11:00:05
## 51:                             1104 2019-01-28 11:00:05
## 52:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 53:                 vweave_rmarkdown 2019-01-28 11:00:05
## 54:                     process_file 2019-01-28 11:00:05
## 55:                    process_group 2019-01-28 11:00:05
## 56:              process_group.block 2019-01-28 11:00:05
## 57:                       call_block 2019-01-28 11:00:05
## 58:                       block_exec 2019-01-28 11:00:05
## 59:                           in_dir 2019-01-28 11:00:05
## 60:                        timing_fn 2019-01-28 11:00:05
## 61:                           handle 2019-01-28 11:00:05
## 62:                      withVisible 2019-01-28 11:00:05
## 63:               n:c5775c3b366fb719 2019-01-28 11:00:05
## 64:            mean:15620f138033a66c 2019-01-28 11:00:05
## 65:            .FUN:4f604aa46882b368 2019-01-28 11:00:05
## 66:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 67:              2019-01-28 11:00:05 2019-01-28 11:00:05
## 68:              2019-01-28 11:00:05 2019-01-28 11:00:05
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.5 Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate
clearCache(tmpDir, ask = FALSE)

1.6 Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 41b03820fbc805b1f29f8b66ab0693cf         format
##  2: 41b03820fbc805b1f29f8b66ab0693cf           name
##  3: 41b03820fbc805b1f29f8b66ab0693cf          class
##  4: 41b03820fbc805b1f29f8b66ab0693cf           date
##  5: 41b03820fbc805b1f29f8b66ab0693cf        cacheId
##  6: 41b03820fbc805b1f29f8b66ab0693cf     objectName
##  7: 41b03820fbc805b1f29f8b66ab0693cf       function
##  8: 41b03820fbc805b1f29f8b66ab0693cf    object.size
##  9: 41b03820fbc805b1f29f8b66ab0693cf       accessed
## 10: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 11: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 12: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 13: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 14: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 15: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 16: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 17: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 18: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 19: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 20: 41b03820fbc805b1f29f8b66ab0693cf      preDigest
## 21: 41b03820fbc805b1f29f8b66ab0693cf      preDigest
## 22: b585bc8994f5bf2762ec1d51f5a50651         format
## 23: b585bc8994f5bf2762ec1d51f5a50651           name
## 24: b585bc8994f5bf2762ec1d51f5a50651          class
## 25: b585bc8994f5bf2762ec1d51f5a50651           date
## 26: b585bc8994f5bf2762ec1d51f5a50651        cacheId
## 27: b585bc8994f5bf2762ec1d51f5a50651     objectName
## 28: b585bc8994f5bf2762ec1d51f5a50651       function
## 29: b585bc8994f5bf2762ec1d51f5a50651    object.size
## 30: b585bc8994f5bf2762ec1d51f5a50651       accessed
## 31: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 32: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 33: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 34: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 35: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 36: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 37: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 38: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 39: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 40: b585bc8994f5bf2762ec1d51f5a50651 otherFunctions
## 41: b585bc8994f5bf2762ec1d51f5a50651      preDigest
## 42: b585bc8994f5bf2762ec1d51f5a50651      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:06
##  2: 41b03820fbc805b1f29f8b66ab0693cf 2019-01-28 11:00:06
##  3:                          numeric 2019-01-28 11:00:06
##  4:              2019-01-28 11:00:06 2019-01-28 11:00:06
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:06
##  6:                                a 2019-01-28 11:00:06
##  7:                            rnorm 2019-01-28 11:00:06
##  8:                             1008 2019-01-28 11:00:06
##  9:              2019-01-28 11:00:06 2019-01-28 11:00:06
## 10:                 vweave_rmarkdown 2019-01-28 11:00:06
## 11:                     process_file 2019-01-28 11:00:06
## 12:                    process_group 2019-01-28 11:00:06
## 13:              process_group.block 2019-01-28 11:00:06
## 14:                       call_block 2019-01-28 11:00:06
## 15:                       block_exec 2019-01-28 11:00:06
## 16:                           in_dir 2019-01-28 11:00:06
## 17:                        timing_fn 2019-01-28 11:00:06
## 18:                           handle 2019-01-28 11:00:06
## 19:                      withVisible 2019-01-28 11:00:06
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:06
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:06
## 22:                              rda 2019-01-28 11:00:06
## 23: b585bc8994f5bf2762ec1d51f5a50651 2019-01-28 11:00:06
## 24:                          numeric 2019-01-28 11:00:06
## 25:              2019-01-28 11:00:06 2019-01-28 11:00:06
## 26:                 3aef38d1fc02aee5 2019-01-28 11:00:06
## 27:                                b 2019-01-28 11:00:06
## 28:                            runif 2019-01-28 11:00:06
## 29:                             1008 2019-01-28 11:00:06
## 30:              2019-01-28 11:00:06 2019-01-28 11:00:06
## 31:                 vweave_rmarkdown 2019-01-28 11:00:06
## 32:                     process_file 2019-01-28 11:00:06
## 33:                    process_group 2019-01-28 11:00:06
## 34:              process_group.block 2019-01-28 11:00:06
## 35:                       call_block 2019-01-28 11:00:06
## 36:                       block_exec 2019-01-28 11:00:06
## 37:                           in_dir 2019-01-28 11:00:06
## 38:                        timing_fn 2019-01-28 11:00:06
## 39:                           handle 2019-01-28 11:00:06
## 40:                      withVisible 2019-01-28 11:00:06
## 41:               n:7eef4eae85fd9229 2019-01-28 11:00:06
## 42:            .FUN:881ec847b7161f3c 2019-01-28 11:00:06
##                             tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 41b03820fbc805b1f29f8b66ab0693cf         format
##  2: 41b03820fbc805b1f29f8b66ab0693cf           name
##  3: 41b03820fbc805b1f29f8b66ab0693cf          class
##  4: 41b03820fbc805b1f29f8b66ab0693cf           date
##  5: 41b03820fbc805b1f29f8b66ab0693cf        cacheId
##  6: 41b03820fbc805b1f29f8b66ab0693cf     objectName
##  7: 41b03820fbc805b1f29f8b66ab0693cf       function
##  8: 41b03820fbc805b1f29f8b66ab0693cf    object.size
##  9: 41b03820fbc805b1f29f8b66ab0693cf       accessed
## 10: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 11: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 12: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 13: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 14: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 15: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 16: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 17: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 18: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 19: 41b03820fbc805b1f29f8b66ab0693cf otherFunctions
## 20: 41b03820fbc805b1f29f8b66ab0693cf      preDigest
## 21: 41b03820fbc805b1f29f8b66ab0693cf      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:06
##  2: 41b03820fbc805b1f29f8b66ab0693cf 2019-01-28 11:00:06
##  3:                          numeric 2019-01-28 11:00:06
##  4:              2019-01-28 11:00:06 2019-01-28 11:00:06
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:06
##  6:                                a 2019-01-28 11:00:06
##  7:                            rnorm 2019-01-28 11:00:06
##  8:                             1008 2019-01-28 11:00:06
##  9:              2019-01-28 11:00:06 2019-01-28 11:00:06
## 10:                 vweave_rmarkdown 2019-01-28 11:00:06
## 11:                     process_file 2019-01-28 11:00:06
## 12:                    process_group 2019-01-28 11:00:06
## 13:              process_group.block 2019-01-28 11:00:06
## 14:                       call_block 2019-01-28 11:00:06
## 15:                       block_exec 2019-01-28 11:00:06
## 16:                           in_dir 2019-01-28 11:00:06
## 17:                        timing_fn 2019-01-28 11:00:06
## 18:                           handle 2019-01-28 11:00:06
## 19:                      withVisible 2019-01-28 11:00:06
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:06
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:06
##                             tagValue         createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

1.7 Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 4f2eec101161fba7326da197d793b4a6         format
##  2: 4f2eec101161fba7326da197d793b4a6           name
##  3: 4f2eec101161fba7326da197d793b4a6          class
##  4: 4f2eec101161fba7326da197d793b4a6           date
##  5: 4f2eec101161fba7326da197d793b4a6        cacheId
##  6: 4f2eec101161fba7326da197d793b4a6     objectName
##  7: 4f2eec101161fba7326da197d793b4a6       function
##  8: 4f2eec101161fba7326da197d793b4a6    object.size
##  9: 4f2eec101161fba7326da197d793b4a6       accessed
## 10: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 11: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 12: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 13: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 14: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 15: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 16: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 17: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 18: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 19: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 20: 4f2eec101161fba7326da197d793b4a6      preDigest
## 21: 4f2eec101161fba7326da197d793b4a6      preDigest
## 22: aefa8fdeb9be6b1f06243d5536fc0b8c         format
## 23: aefa8fdeb9be6b1f06243d5536fc0b8c           name
## 24: aefa8fdeb9be6b1f06243d5536fc0b8c          class
## 25: aefa8fdeb9be6b1f06243d5536fc0b8c           date
## 26: aefa8fdeb9be6b1f06243d5536fc0b8c        cacheId
## 27: aefa8fdeb9be6b1f06243d5536fc0b8c     objectName
## 28: aefa8fdeb9be6b1f06243d5536fc0b8c       function
## 29: aefa8fdeb9be6b1f06243d5536fc0b8c    object.size
## 30: aefa8fdeb9be6b1f06243d5536fc0b8c       accessed
## 31: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 32: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 33: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 34: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 35: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 36: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 37: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 38: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 39: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 40: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 41: aefa8fdeb9be6b1f06243d5536fc0b8c      preDigest
## 42: aefa8fdeb9be6b1f06243d5536fc0b8c      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:07
##  2: 4f2eec101161fba7326da197d793b4a6 2019-01-28 11:00:07
##  3:                          numeric 2019-01-28 11:00:07
##  4:              2019-01-28 11:00:07 2019-01-28 11:00:07
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:07
##  6:                                b 2019-01-28 11:00:07
##  7:                            rnorm 2019-01-28 11:00:07
##  8:                             1008 2019-01-28 11:00:07
##  9:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 10:                 vweave_rmarkdown 2019-01-28 11:00:07
## 11:                     process_file 2019-01-28 11:00:07
## 12:                    process_group 2019-01-28 11:00:07
## 13:              process_group.block 2019-01-28 11:00:07
## 14:                       call_block 2019-01-28 11:00:07
## 15:                       block_exec 2019-01-28 11:00:07
## 16:                           in_dir 2019-01-28 11:00:07
## 17:                        timing_fn 2019-01-28 11:00:07
## 18:                           handle 2019-01-28 11:00:07
## 19:                      withVisible 2019-01-28 11:00:07
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:07
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:07
## 22:                              rda 2019-01-28 11:00:07
## 23: aefa8fdeb9be6b1f06243d5536fc0b8c 2019-01-28 11:00:07
## 24:                          numeric 2019-01-28 11:00:07
## 25:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 26:                 3aef38d1fc02aee5 2019-01-28 11:00:07
## 27:                                a 2019-01-28 11:00:07
## 28:                            runif 2019-01-28 11:00:07
## 29:                             1008 2019-01-28 11:00:07
## 30:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 31:                 vweave_rmarkdown 2019-01-28 11:00:07
## 32:                     process_file 2019-01-28 11:00:07
## 33:                    process_group 2019-01-28 11:00:07
## 34:              process_group.block 2019-01-28 11:00:07
## 35:                       call_block 2019-01-28 11:00:07
## 36:                       block_exec 2019-01-28 11:00:07
## 37:                           in_dir 2019-01-28 11:00:07
## 38:                        timing_fn 2019-01-28 11:00:07
## 39:                           handle 2019-01-28 11:00:07
## 40:                      withVisible 2019-01-28 11:00:07
## 41:               n:7eef4eae85fd9229 2019-01-28 11:00:07
## 42:            .FUN:881ec847b7161f3c 2019-01-28 11:00:07
##                             tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 4f2eec101161fba7326da197d793b4a6         format
##  2: 4f2eec101161fba7326da197d793b4a6           name
##  3: 4f2eec101161fba7326da197d793b4a6          class
##  4: 4f2eec101161fba7326da197d793b4a6           date
##  5: 4f2eec101161fba7326da197d793b4a6        cacheId
##  6: 4f2eec101161fba7326da197d793b4a6     objectName
##  7: 4f2eec101161fba7326da197d793b4a6       function
##  8: 4f2eec101161fba7326da197d793b4a6    object.size
##  9: 4f2eec101161fba7326da197d793b4a6       accessed
## 10: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 11: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 12: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 13: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 14: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 15: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 16: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 17: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 18: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 19: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 20: 4f2eec101161fba7326da197d793b4a6      preDigest
## 21: 4f2eec101161fba7326da197d793b4a6      preDigest
## 22: aefa8fdeb9be6b1f06243d5536fc0b8c         format
## 23: aefa8fdeb9be6b1f06243d5536fc0b8c           name
## 24: aefa8fdeb9be6b1f06243d5536fc0b8c          class
## 25: aefa8fdeb9be6b1f06243d5536fc0b8c           date
## 26: aefa8fdeb9be6b1f06243d5536fc0b8c        cacheId
## 27: aefa8fdeb9be6b1f06243d5536fc0b8c     objectName
## 28: aefa8fdeb9be6b1f06243d5536fc0b8c       function
## 29: aefa8fdeb9be6b1f06243d5536fc0b8c    object.size
## 30: aefa8fdeb9be6b1f06243d5536fc0b8c       accessed
## 31: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 32: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 33: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 34: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 35: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 36: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 37: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 38: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 39: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 40: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 41: aefa8fdeb9be6b1f06243d5536fc0b8c      preDigest
## 42: aefa8fdeb9be6b1f06243d5536fc0b8c      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:07
##  2: 4f2eec101161fba7326da197d793b4a6 2019-01-28 11:00:07
##  3:                          numeric 2019-01-28 11:00:07
##  4:              2019-01-28 11:00:07 2019-01-28 11:00:07
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:07
##  6:                                b 2019-01-28 11:00:07
##  7:                            rnorm 2019-01-28 11:00:07
##  8:                             1008 2019-01-28 11:00:07
##  9:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 10:                 vweave_rmarkdown 2019-01-28 11:00:07
## 11:                     process_file 2019-01-28 11:00:07
## 12:                    process_group 2019-01-28 11:00:07
## 13:              process_group.block 2019-01-28 11:00:07
## 14:                       call_block 2019-01-28 11:00:07
## 15:                       block_exec 2019-01-28 11:00:07
## 16:                           in_dir 2019-01-28 11:00:07
## 17:                        timing_fn 2019-01-28 11:00:07
## 18:                           handle 2019-01-28 11:00:07
## 19:                      withVisible 2019-01-28 11:00:07
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:07
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:07
## 22:                              rda 2019-01-28 11:00:07
## 23: aefa8fdeb9be6b1f06243d5536fc0b8c 2019-01-28 11:00:07
## 24:                          numeric 2019-01-28 11:00:07
## 25:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 26:                 3aef38d1fc02aee5 2019-01-28 11:00:07
## 27:                                a 2019-01-28 11:00:07
## 28:                            runif 2019-01-28 11:00:07
## 29:                             1008 2019-01-28 11:00:07
## 30:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 31:                 vweave_rmarkdown 2019-01-28 11:00:07
## 32:                     process_file 2019-01-28 11:00:07
## 33:                    process_group 2019-01-28 11:00:07
## 34:              process_group.block 2019-01-28 11:00:07
## 35:                       call_block 2019-01-28 11:00:07
## 36:                       block_exec 2019-01-28 11:00:07
## 37:                           in_dir 2019-01-28 11:00:07
## 38:                        timing_fn 2019-01-28 11:00:07
## 39:                           handle 2019-01-28 11:00:07
## 40:                      withVisible 2019-01-28 11:00:07
## 41:               n:7eef4eae85fd9229 2019-01-28 11:00:07
## 42:            .FUN:881ec847b7161f3c 2019-01-28 11:00:07
##                             tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 4f2eec101161fba7326da197d793b4a6         format
##  2: 4f2eec101161fba7326da197d793b4a6           name
##  3: 4f2eec101161fba7326da197d793b4a6          class
##  4: 4f2eec101161fba7326da197d793b4a6           date
##  5: 4f2eec101161fba7326da197d793b4a6        cacheId
##  6: 4f2eec101161fba7326da197d793b4a6     objectName
##  7: 4f2eec101161fba7326da197d793b4a6       function
##  8: 4f2eec101161fba7326da197d793b4a6    object.size
##  9: 4f2eec101161fba7326da197d793b4a6       accessed
## 10: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 11: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 12: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 13: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 14: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 15: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 16: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 17: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 18: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 19: 4f2eec101161fba7326da197d793b4a6 otherFunctions
## 20: 4f2eec101161fba7326da197d793b4a6      preDigest
## 21: 4f2eec101161fba7326da197d793b4a6      preDigest
## 22: aefa8fdeb9be6b1f06243d5536fc0b8c         format
## 23: aefa8fdeb9be6b1f06243d5536fc0b8c           name
## 24: aefa8fdeb9be6b1f06243d5536fc0b8c          class
## 25: aefa8fdeb9be6b1f06243d5536fc0b8c           date
## 26: aefa8fdeb9be6b1f06243d5536fc0b8c        cacheId
## 27: aefa8fdeb9be6b1f06243d5536fc0b8c     objectName
## 28: aefa8fdeb9be6b1f06243d5536fc0b8c       function
## 29: aefa8fdeb9be6b1f06243d5536fc0b8c    object.size
## 30: aefa8fdeb9be6b1f06243d5536fc0b8c       accessed
## 31: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 32: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 33: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 34: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 35: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 36: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 37: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 38: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 39: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 40: aefa8fdeb9be6b1f06243d5536fc0b8c otherFunctions
## 41: aefa8fdeb9be6b1f06243d5536fc0b8c      preDigest
## 42: aefa8fdeb9be6b1f06243d5536fc0b8c      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:07
##  2: 4f2eec101161fba7326da197d793b4a6 2019-01-28 11:00:07
##  3:                          numeric 2019-01-28 11:00:07
##  4:              2019-01-28 11:00:07 2019-01-28 11:00:07
##  5:                 f7bee22047b8d0c1 2019-01-28 11:00:07
##  6:                                b 2019-01-28 11:00:07
##  7:                            rnorm 2019-01-28 11:00:07
##  8:                             1008 2019-01-28 11:00:07
##  9:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 10:                 vweave_rmarkdown 2019-01-28 11:00:07
## 11:                     process_file 2019-01-28 11:00:07
## 12:                    process_group 2019-01-28 11:00:07
## 13:              process_group.block 2019-01-28 11:00:07
## 14:                       call_block 2019-01-28 11:00:07
## 15:                       block_exec 2019-01-28 11:00:07
## 16:                           in_dir 2019-01-28 11:00:07
## 17:                        timing_fn 2019-01-28 11:00:07
## 18:                           handle 2019-01-28 11:00:07
## 19:                      withVisible 2019-01-28 11:00:07
## 20:               n:7eef4eae85fd9229 2019-01-28 11:00:07
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:07
## 22:                              rda 2019-01-28 11:00:07
## 23: aefa8fdeb9be6b1f06243d5536fc0b8c 2019-01-28 11:00:07
## 24:                          numeric 2019-01-28 11:00:07
## 25:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 26:                 3aef38d1fc02aee5 2019-01-28 11:00:07
## 27:                                a 2019-01-28 11:00:07
## 28:                            runif 2019-01-28 11:00:07
## 29:                             1008 2019-01-28 11:00:07
## 30:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 31:                 vweave_rmarkdown 2019-01-28 11:00:07
## 32:                     process_file 2019-01-28 11:00:07
## 33:                    process_group 2019-01-28 11:00:07
## 34:              process_group.block 2019-01-28 11:00:07
## 35:                       call_block 2019-01-28 11:00:07
## 36:                       block_exec 2019-01-28 11:00:07
## 37:                           in_dir 2019-01-28 11:00:07
## 38:                        timing_fn 2019-01-28 11:00:07
## 39:                           handle 2019-01-28 11:00:07
## 40:                      withVisible 2019-01-28 11:00:07
## 41:               n:7eef4eae85fd9229 2019-01-28 11:00:07
## 42:            .FUN:881ec847b7161f3c 2019-01-28 11:00:07
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.8 Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

1.9 Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1] -1.4258265 -1.4687567  0.9123924
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 4571083d4dbddfba7362b1236be88944         format
##  2: 4571083d4dbddfba7362b1236be88944           name
##  3: 4571083d4dbddfba7362b1236be88944          class
##  4: 4571083d4dbddfba7362b1236be88944           date
##  5: 4571083d4dbddfba7362b1236be88944        cacheId
##  6: 4571083d4dbddfba7362b1236be88944       function
##  7: 4571083d4dbddfba7362b1236be88944    object.size
##  8: 4571083d4dbddfba7362b1236be88944       accessed
##  9: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 10: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 11: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 12: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 13: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 14: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 15: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 16: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 17: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 18: 4571083d4dbddfba7362b1236be88944 otherFunctions
## 19: 4571083d4dbddfba7362b1236be88944      preDigest
## 20: 4571083d4dbddfba7362b1236be88944      preDigest
## 21: 4571083d4dbddfba7362b1236be88944      preDigest
## 22: 8a4e81fc565841d5cbdd42362704d996         format
## 23: 8a4e81fc565841d5cbdd42362704d996           name
## 24: 8a4e81fc565841d5cbdd42362704d996          class
## 25: 8a4e81fc565841d5cbdd42362704d996           date
## 26: 8a4e81fc565841d5cbdd42362704d996        cacheId
## 27: 8a4e81fc565841d5cbdd42362704d996       function
## 28: 8a4e81fc565841d5cbdd42362704d996    object.size
## 29: 8a4e81fc565841d5cbdd42362704d996       accessed
## 30: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 31: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 32: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 33: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 34: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 35: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 36: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 37: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 38: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 39: 8a4e81fc565841d5cbdd42362704d996 otherFunctions
## 40: 8a4e81fc565841d5cbdd42362704d996      preDigest
## 41: 8a4e81fc565841d5cbdd42362704d996      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:07
##  2: 4571083d4dbddfba7362b1236be88944 2019-01-28 11:00:07
##  3:                          numeric 2019-01-28 11:00:07
##  4:              2019-01-28 11:00:07 2019-01-28 11:00:07
##  5:                 4ac2b7c0f42d1e46 2019-01-28 11:00:07
##  6:                            rnorm 2019-01-28 11:00:07
##  7:                             1008 2019-01-28 11:00:07
##  8:              2019-01-28 11:00:07 2019-01-28 11:00:07
##  9:                 vweave_rmarkdown 2019-01-28 11:00:07
## 10:                     process_file 2019-01-28 11:00:07
## 11:                    process_group 2019-01-28 11:00:07
## 12:              process_group.block 2019-01-28 11:00:07
## 13:                       call_block 2019-01-28 11:00:07
## 14:                       block_exec 2019-01-28 11:00:07
## 15:                           in_dir 2019-01-28 11:00:07
## 16:                        timing_fn 2019-01-28 11:00:07
## 17:                           handle 2019-01-28 11:00:07
## 18:                      withVisible 2019-01-28 11:00:07
## 19:               n:7f12988bd88a0fb8 2019-01-28 11:00:07
## 20:            mean:22413394efd9f6a3 2019-01-28 11:00:07
## 21:            .FUN:4f604aa46882b368 2019-01-28 11:00:07
## 22:                              rda 2019-01-28 11:00:07
## 23: 8a4e81fc565841d5cbdd42362704d996 2019-01-28 11:00:07
## 24:                          numeric 2019-01-28 11:00:07
## 25:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 26:                 a7af5367c13aba8f 2019-01-28 11:00:07
## 27:                            outer 2019-01-28 11:00:07
## 28:                             1008 2019-01-28 11:00:07
## 29:              2019-01-28 11:00:07 2019-01-28 11:00:07
## 30:                 vweave_rmarkdown 2019-01-28 11:00:07
## 31:                     process_file 2019-01-28 11:00:07
## 32:                    process_group 2019-01-28 11:00:07
## 33:              process_group.block 2019-01-28 11:00:07
## 34:                       call_block 2019-01-28 11:00:07
## 35:                       block_exec 2019-01-28 11:00:07
## 36:                           in_dir 2019-01-28 11:00:07
## 37:                        timing_fn 2019-01-28 11:00:07
## 38:                           handle 2019-01-28 11:00:08
## 39:                      withVisible 2019-01-28 11:00:08
## 40:               n:82dc709f2b91918a 2019-01-28 11:00:08
## 41:            .FUN:892a6afc47a63a90 2019-01-28 11:00:08
##                             tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: e4746767371ee7e6b38f9647f9b73482         format
##  2: e4746767371ee7e6b38f9647f9b73482           name
##  3: e4746767371ee7e6b38f9647f9b73482          class
##  4: e4746767371ee7e6b38f9647f9b73482           date
##  5: e4746767371ee7e6b38f9647f9b73482        cacheId
##  6: e4746767371ee7e6b38f9647f9b73482       function
##  7: e4746767371ee7e6b38f9647f9b73482    object.size
##  8: e4746767371ee7e6b38f9647f9b73482       accessed
##  9: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 10: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 11: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 12: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 13: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 14: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 15: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 16: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 17: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 18: e4746767371ee7e6b38f9647f9b73482 otherFunctions
## 19: e4746767371ee7e6b38f9647f9b73482      preDigest
## 20: e4746767371ee7e6b38f9647f9b73482      preDigest
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:07
##  2: e4746767371ee7e6b38f9647f9b73482 2019-01-28 11:00:07
##  3:                          numeric 2019-01-28 11:00:07
##  4:              2019-01-28 11:00:07 2019-01-28 11:00:07
##  5:                 33ceb4fb525fd08f 2019-01-28 11:00:07
##  6:                            inner 2019-01-28 11:00:07
##  7:                             1008 2019-01-28 11:00:07
##  8:              2019-01-28 11:00:07 2019-01-28 11:00:07
##  9:                 vweave_rmarkdown 2019-01-28 11:00:07
## 10:                     process_file 2019-01-28 11:00:07
## 11:                    process_group 2019-01-28 11:00:07
## 12:              process_group.block 2019-01-28 11:00:07
## 13:                       call_block 2019-01-28 11:00:07
## 14:                       block_exec 2019-01-28 11:00:07
## 15:                           in_dir 2019-01-28 11:00:07
## 16:                        timing_fn 2019-01-28 11:00:07
## 17:                           handle 2019-01-28 11:00:07
## 18:                      withVisible 2019-01-28 11:00:07
## 19:            mean:22413394efd9f6a3 2019-01-28 11:00:07
## 20:            .FUN:87e2c30917a34d25 2019-01-28 11:00:07
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##                             artifact         tagKey
##  1: 55ca5ec58674d9e2119c6c41b84d2d46         format
##  2: 55ca5ec58674d9e2119c6c41b84d2d46           name
##  3: 55ca5ec58674d9e2119c6c41b84d2d46          class
##  4: 55ca5ec58674d9e2119c6c41b84d2d46           date
##  5: 55ca5ec58674d9e2119c6c41b84d2d46        cacheId
##  6: 55ca5ec58674d9e2119c6c41b84d2d46       outerTag
##  7: 55ca5ec58674d9e2119c6c41b84d2d46       function
##  8: 55ca5ec58674d9e2119c6c41b84d2d46    object.size
##  9: 55ca5ec58674d9e2119c6c41b84d2d46       accessed
## 10: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 11: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 12: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 13: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 14: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 15: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 16: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 17: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 18: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 19: 55ca5ec58674d9e2119c6c41b84d2d46 otherFunctions
## 20: 55ca5ec58674d9e2119c6c41b84d2d46      preDigest
## 21: 55ca5ec58674d9e2119c6c41b84d2d46      preDigest
## 22: 8cce866a994487d14cae2bbf5e5fff3e         format
## 23: 8cce866a994487d14cae2bbf5e5fff3e           name
## 24: 8cce866a994487d14cae2bbf5e5fff3e          class
## 25: 8cce866a994487d14cae2bbf5e5fff3e           date
## 26: 8cce866a994487d14cae2bbf5e5fff3e        cacheId
## 27: 8cce866a994487d14cae2bbf5e5fff3e       outerTag
## 28: 8cce866a994487d14cae2bbf5e5fff3e       function
## 29: 8cce866a994487d14cae2bbf5e5fff3e    object.size
## 30: 8cce866a994487d14cae2bbf5e5fff3e       accessed
## 31: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 32: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 33: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 34: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 35: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 36: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 37: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 38: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 39: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 40: 8cce866a994487d14cae2bbf5e5fff3e otherFunctions
## 41: 8cce866a994487d14cae2bbf5e5fff3e      preDigest
## 42: 8cce866a994487d14cae2bbf5e5fff3e      preDigest
## 43: d799f879bf686af1fba8aea59b51953f         format
## 44: d799f879bf686af1fba8aea59b51953f           name
## 45: d799f879bf686af1fba8aea59b51953f          class
## 46: d799f879bf686af1fba8aea59b51953f           date
## 47: d799f879bf686af1fba8aea59b51953f        cacheId
## 48: d799f879bf686af1fba8aea59b51953f       innerTag
## 49: d799f879bf686af1fba8aea59b51953f       outerTag
## 50: d799f879bf686af1fba8aea59b51953f       function
## 51: d799f879bf686af1fba8aea59b51953f    object.size
## 52: d799f879bf686af1fba8aea59b51953f       accessed
## 53: d799f879bf686af1fba8aea59b51953f otherFunctions
## 54: d799f879bf686af1fba8aea59b51953f otherFunctions
## 55: d799f879bf686af1fba8aea59b51953f otherFunctions
## 56: d799f879bf686af1fba8aea59b51953f otherFunctions
## 57: d799f879bf686af1fba8aea59b51953f otherFunctions
## 58: d799f879bf686af1fba8aea59b51953f otherFunctions
## 59: d799f879bf686af1fba8aea59b51953f otherFunctions
## 60: d799f879bf686af1fba8aea59b51953f otherFunctions
## 61: d799f879bf686af1fba8aea59b51953f otherFunctions
## 62: d799f879bf686af1fba8aea59b51953f otherFunctions
## 63: d799f879bf686af1fba8aea59b51953f      preDigest
## 64: d799f879bf686af1fba8aea59b51953f      preDigest
## 65: d799f879bf686af1fba8aea59b51953f      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-01-28 11:00:08
##  2: 55ca5ec58674d9e2119c6c41b84d2d46 2019-01-28 11:00:08
##  3:                          numeric 2019-01-28 11:00:08
##  4:              2019-01-28 11:00:08 2019-01-28 11:00:08
##  5:                 b06af03d5a73dc7d 2019-01-28 11:00:08
##  6:                         outerTag 2019-01-28 11:00:08
##  7:                            inner 2019-01-28 11:00:08
##  8:                             1008 2019-01-28 11:00:08
##  9:              2019-01-28 11:00:08 2019-01-28 11:00:08
## 10:                 vweave_rmarkdown 2019-01-28 11:00:08
## 11:                     process_file 2019-01-28 11:00:08
## 12:                    process_group 2019-01-28 11:00:08
## 13:              process_group.block 2019-01-28 11:00:08
## 14:                       call_block 2019-01-28 11:00:08
## 15:                       block_exec 2019-01-28 11:00:08
## 16:                           in_dir 2019-01-28 11:00:08
## 17:                        timing_fn 2019-01-28 11:00:08
## 18:                           handle 2019-01-28 11:00:08
## 19:                      withVisible 2019-01-28 11:00:08
## 20:            mean:22413394efd9f6a3 2019-01-28 11:00:08
## 21:            .FUN:7ad10bc1ae528d8c 2019-01-28 11:00:08
## 22:                              rda 2019-01-28 11:00:08
## 23: 8cce866a994487d14cae2bbf5e5fff3e 2019-01-28 11:00:08
## 24:                          numeric 2019-01-28 11:00:08
## 25:              2019-01-28 11:00:08 2019-01-28 11:00:08
## 26:                 88a34e1d033329e5 2019-01-28 11:00:08
## 27:                         outerTag 2019-01-28 11:00:08
## 28:                            outer 2019-01-28 11:00:08
## 29:                             1008 2019-01-28 11:00:08
## 30:              2019-01-28 11:00:08 2019-01-28 11:00:08
## 31:                 vweave_rmarkdown 2019-01-28 11:00:08
## 32:                     process_file 2019-01-28 11:00:08
## 33:                    process_group 2019-01-28 11:00:08
## 34:              process_group.block 2019-01-28 11:00:08
## 35:                       call_block 2019-01-28 11:00:08
## 36:                       block_exec 2019-01-28 11:00:08
## 37:                           in_dir 2019-01-28 11:00:08
## 38:                        timing_fn 2019-01-28 11:00:08
## 39:                           handle 2019-01-28 11:00:08
## 40:                      withVisible 2019-01-28 11:00:08
## 41:               n:82dc709f2b91918a 2019-01-28 11:00:08
## 42:            .FUN:5f06fb5fbffe9e3b 2019-01-28 11:00:08
## 43:                              rda 2019-01-28 11:00:08
## 44: d799f879bf686af1fba8aea59b51953f 2019-01-28 11:00:08
## 45:                          numeric 2019-01-28 11:00:08
## 46:              2019-01-28 11:00:08 2019-01-28 11:00:08
## 47:                 4ac2b7c0f42d1e46 2019-01-28 11:00:08
## 48:                         innerTag 2019-01-28 11:00:08
## 49:                         outerTag 2019-01-28 11:00:08
## 50:                            rnorm 2019-01-28 11:00:08
## 51:                             1008 2019-01-28 11:00:08
## 52:              2019-01-28 11:00:08 2019-01-28 11:00:08
## 53:                 vweave_rmarkdown 2019-01-28 11:00:08
## 54:                     process_file 2019-01-28 11:00:08
## 55:                    process_group 2019-01-28 11:00:08
## 56:              process_group.block 2019-01-28 11:00:08
## 57:                       call_block 2019-01-28 11:00:08
## 58:                       block_exec 2019-01-28 11:00:08
## 59:                           in_dir 2019-01-28 11:00:08
## 60:                        timing_fn 2019-01-28 11:00:08
## 61:                           handle 2019-01-28 11:00:08
## 62:                      withVisible 2019-01-28 11:00:08
## 63:               n:7f12988bd88a0fb8 2019-01-28 11:00:08
## 64:            mean:22413394efd9f6a3 2019-01-28 11:00:08
## 65:            .FUN:4f604aa46882b368 2019-01-28 11:00:08
##                             tagValue         createdDate

1.10 cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

1.11 Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

2 Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

2.1 Nested Caching

2.1.0.1 Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.