Introduction to Cache

Eliot J. B. McIntire

July 08 2019

1 Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

1.1 Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

1.2 Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)
## Registered S3 method overwritten by 'R.oo':
##   method        from       
##   throw.default R.methodsS3
tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmptuRuQo/reproducible_examples/Cache"
ras <- raster(extent(0, 1000, 0, 1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   2.629   0.235   2.925
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   3.612   0.207   4.025
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.306   0.012   0.319
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.036   0.004   0.039
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

1.3 Caching examples

1.3.1 Basic use

library(raster)
library(magrittr)
## 
## Attaching package: 'magrittr'
## The following object is masked from 'package:raster':
## 
##     extract
try(clearCache(tmpDir, ask = FALSE), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 16)  # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- Cache(cacheRepo = tmpDir) %C% rnorm(10, 6) # different

1.4 Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 78c191d8dec3b97f26fe462521cda3f3         format
##  2: 78c191d8dec3b97f26fe462521cda3f3           name
##  3: 78c191d8dec3b97f26fe462521cda3f3          class
##  4: 78c191d8dec3b97f26fe462521cda3f3           date
##  5: 78c191d8dec3b97f26fe462521cda3f3        cacheId
##  6: 78c191d8dec3b97f26fe462521cda3f3     objectName
##  7: 78c191d8dec3b97f26fe462521cda3f3       function
##  8: 78c191d8dec3b97f26fe462521cda3f3    object.size
##  9: 78c191d8dec3b97f26fe462521cda3f3       accessed
## 10: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 11: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 12: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 13: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 14: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 15: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 16: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 17: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 18: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 19: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 20: 78c191d8dec3b97f26fe462521cda3f3      preDigest
## 21: 78c191d8dec3b97f26fe462521cda3f3      preDigest
## 22: a94147d8c2f54bae8b134184c2ac86a2         format
## 23: a94147d8c2f54bae8b134184c2ac86a2           name
## 24: a94147d8c2f54bae8b134184c2ac86a2          class
## 25: a94147d8c2f54bae8b134184c2ac86a2           date
## 26: a94147d8c2f54bae8b134184c2ac86a2        cacheId
## 27: a94147d8c2f54bae8b134184c2ac86a2     objectName
## 28: a94147d8c2f54bae8b134184c2ac86a2       function
## 29: a94147d8c2f54bae8b134184c2ac86a2    object.size
## 30: a94147d8c2f54bae8b134184c2ac86a2       accessed
## 31: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 32: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 33: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 34: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 35: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 36: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 37: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 38: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 39: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 40: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 41: a94147d8c2f54bae8b134184c2ac86a2      preDigest
## 42: a94147d8c2f54bae8b134184c2ac86a2      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:54
##  2: 78c191d8dec3b97f26fe462521cda3f3 2019-07-08 19:59:54
##  3:                          numeric 2019-07-08 19:59:54
##  4:              2019-07-08 19:59:54 2019-07-08 19:59:54
##  5:                 f7bee22047b8d0c1 2019-07-08 19:59:54
##  6:                                a 2019-07-08 19:59:54
##  7:                            rnorm 2019-07-08 19:59:54
##  8:                             1008 2019-07-08 19:59:54
##  9:              2019-07-08 19:59:54 2019-07-08 19:59:54
## 10:                 vweave_rmarkdown 2019-07-08 19:59:54
## 11:                     process_file 2019-07-08 19:59:54
## 12:                    process_group 2019-07-08 19:59:54
## 13:              process_group.block 2019-07-08 19:59:54
## 14:                       call_block 2019-07-08 19:59:54
## 15:                       block_exec 2019-07-08 19:59:54
## 16:                           in_dir 2019-07-08 19:59:54
## 17:                        timing_fn 2019-07-08 19:59:54
## 18:                           handle 2019-07-08 19:59:54
## 19:                      withVisible 2019-07-08 19:59:54
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:54
## 21:            .FUN:4f604aa46882b368 2019-07-08 19:59:54
## 22:                              rda 2019-07-08 19:59:54
## 23: a94147d8c2f54bae8b134184c2ac86a2 2019-07-08 19:59:54
## 24:                          numeric 2019-07-08 19:59:54
## 25:              2019-07-08 19:59:54 2019-07-08 19:59:54
## 26:                 3aef38d1fc02aee5 2019-07-08 19:59:54
## 27:                                b 2019-07-08 19:59:54
## 28:                            runif 2019-07-08 19:59:54
## 29:                             1008 2019-07-08 19:59:54
## 30:              2019-07-08 19:59:54 2019-07-08 19:59:54
## 31:                 vweave_rmarkdown 2019-07-08 19:59:54
## 32:                     process_file 2019-07-08 19:59:54
## 33:                    process_group 2019-07-08 19:59:54
## 34:              process_group.block 2019-07-08 19:59:54
## 35:                       call_block 2019-07-08 19:59:54
## 36:                       block_exec 2019-07-08 19:59:54
## 37:                           in_dir 2019-07-08 19:59:54
## 38:                        timing_fn 2019-07-08 19:59:54
## 39:                           handle 2019-07-08 19:59:54
## 40:                      withVisible 2019-07-08 19:59:54
## 41:               n:7eef4eae85fd9229 2019-07-08 19:59:54
## 42:            .FUN:881ec847b7161f3c 2019-07-08 19:59:54
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 78c191d8dec3b97f26fe462521cda3f3         format
##  2: 78c191d8dec3b97f26fe462521cda3f3           name
##  3: 78c191d8dec3b97f26fe462521cda3f3          class
##  4: 78c191d8dec3b97f26fe462521cda3f3           date
##  5: 78c191d8dec3b97f26fe462521cda3f3        cacheId
##  6: 78c191d8dec3b97f26fe462521cda3f3     objectName
##  7: 78c191d8dec3b97f26fe462521cda3f3       function
##  8: 78c191d8dec3b97f26fe462521cda3f3    object.size
##  9: 78c191d8dec3b97f26fe462521cda3f3       accessed
## 10: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 11: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 12: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 13: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 14: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 15: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 16: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 17: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 18: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 19: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 20: 78c191d8dec3b97f26fe462521cda3f3      preDigest
## 21: 78c191d8dec3b97f26fe462521cda3f3      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:54
##  2: 78c191d8dec3b97f26fe462521cda3f3 2019-07-08 19:59:54
##  3:                          numeric 2019-07-08 19:59:54
##  4:              2019-07-08 19:59:54 2019-07-08 19:59:54
##  5:                 f7bee22047b8d0c1 2019-07-08 19:59:54
##  6:                                a 2019-07-08 19:59:54
##  7:                            rnorm 2019-07-08 19:59:54
##  8:                             1008 2019-07-08 19:59:54
##  9:              2019-07-08 19:59:54 2019-07-08 19:59:54
## 10:                 vweave_rmarkdown 2019-07-08 19:59:54
## 11:                     process_file 2019-07-08 19:59:54
## 12:                    process_group 2019-07-08 19:59:54
## 13:              process_group.block 2019-07-08 19:59:54
## 14:                       call_block 2019-07-08 19:59:54
## 15:                       block_exec 2019-07-08 19:59:54
## 16:                           in_dir 2019-07-08 19:59:54
## 17:                        timing_fn 2019-07-08 19:59:54
## 18:                           handle 2019-07-08 19:59:54
## 19:                      withVisible 2019-07-08 19:59:54
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:54
## 21:            .FUN:4f604aa46882b368 2019-07-08 19:59:54
##                             tagValue         createdDate
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: a94147d8c2f54bae8b134184c2ac86a2         format
##  2: a94147d8c2f54bae8b134184c2ac86a2           name
##  3: a94147d8c2f54bae8b134184c2ac86a2          class
##  4: a94147d8c2f54bae8b134184c2ac86a2           date
##  5: a94147d8c2f54bae8b134184c2ac86a2        cacheId
##  6: a94147d8c2f54bae8b134184c2ac86a2     objectName
##  7: a94147d8c2f54bae8b134184c2ac86a2       function
##  8: a94147d8c2f54bae8b134184c2ac86a2    object.size
##  9: a94147d8c2f54bae8b134184c2ac86a2       accessed
## 10: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 11: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 12: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 13: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 14: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 15: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 16: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 17: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 18: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 19: a94147d8c2f54bae8b134184c2ac86a2 otherFunctions
## 20: a94147d8c2f54bae8b134184c2ac86a2      preDigest
## 21: a94147d8c2f54bae8b134184c2ac86a2      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:54
##  2: a94147d8c2f54bae8b134184c2ac86a2 2019-07-08 19:59:54
##  3:                          numeric 2019-07-08 19:59:54
##  4:              2019-07-08 19:59:54 2019-07-08 19:59:54
##  5:                 3aef38d1fc02aee5 2019-07-08 19:59:54
##  6:                                b 2019-07-08 19:59:54
##  7:                            runif 2019-07-08 19:59:54
##  8:                             1008 2019-07-08 19:59:54
##  9:              2019-07-08 19:59:54 2019-07-08 19:59:54
## 10:                 vweave_rmarkdown 2019-07-08 19:59:54
## 11:                     process_file 2019-07-08 19:59:54
## 12:                    process_group 2019-07-08 19:59:54
## 13:              process_group.block 2019-07-08 19:59:54
## 14:                       call_block 2019-07-08 19:59:54
## 15:                       block_exec 2019-07-08 19:59:54
## 16:                           in_dir 2019-07-08 19:59:54
## 17:                        timing_fn 2019-07-08 19:59:54
## 18:                           handle 2019-07-08 19:59:54
## 19:                      withVisible 2019-07-08 19:59:54
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:54
## 21:            .FUN:881ec847b7161f3c 2019-07-08 19:59:54
##                             tagValue         createdDate
clearCache(tmpDir, userTags = c("runif"), ask = FALSE) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 1 Kb
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 804 bytes
##   Selected objects (not including Rasters): 804 bytes
##                             artifact         tagKey
##  1: 0aa301f9ed377479c7f8328ecc41b33a         format
##  2: 0aa301f9ed377479c7f8328ecc41b33a           name
##  3: 0aa301f9ed377479c7f8328ecc41b33a          class
##  4: 0aa301f9ed377479c7f8328ecc41b33a           date
##  5: 0aa301f9ed377479c7f8328ecc41b33a        cacheId
##  6: 0aa301f9ed377479c7f8328ecc41b33a       function
##  7: 0aa301f9ed377479c7f8328ecc41b33a    object.size
##  8: 0aa301f9ed377479c7f8328ecc41b33a       accessed
##  9: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 10: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 11: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 12: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 13: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 14: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 15: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 16: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 17: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 18: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 19: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 20: 0aa301f9ed377479c7f8328ecc41b33a otherFunctions
## 21: 0aa301f9ed377479c7f8328ecc41b33a      preDigest
## 22: 0aa301f9ed377479c7f8328ecc41b33a      preDigest
## 23: 0aa301f9ed377479c7f8328ecc41b33a      preDigest
## 24: 78c191d8dec3b97f26fe462521cda3f3         format
## 25: 78c191d8dec3b97f26fe462521cda3f3           name
## 26: 78c191d8dec3b97f26fe462521cda3f3          class
## 27: 78c191d8dec3b97f26fe462521cda3f3           date
## 28: 78c191d8dec3b97f26fe462521cda3f3        cacheId
## 29: 78c191d8dec3b97f26fe462521cda3f3     objectName
## 30: 78c191d8dec3b97f26fe462521cda3f3       function
## 31: 78c191d8dec3b97f26fe462521cda3f3    object.size
## 32: 78c191d8dec3b97f26fe462521cda3f3       accessed
## 33: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 34: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 35: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 36: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 37: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 38: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 39: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 40: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 41: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 42: 78c191d8dec3b97f26fe462521cda3f3 otherFunctions
## 43: 78c191d8dec3b97f26fe462521cda3f3      preDigest
## 44: 78c191d8dec3b97f26fe462521cda3f3      preDigest
## 45: c679b3770edf4f535ecc97f20a556ee3         format
## 46: c679b3770edf4f535ecc97f20a556ee3           name
## 47: c679b3770edf4f535ecc97f20a556ee3          class
## 48: c679b3770edf4f535ecc97f20a556ee3           date
## 49: c679b3770edf4f535ecc97f20a556ee3        cacheId
## 50: c679b3770edf4f535ecc97f20a556ee3       function
## 51: c679b3770edf4f535ecc97f20a556ee3    object.size
## 52: c679b3770edf4f535ecc97f20a556ee3       accessed
## 53: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 54: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 55: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 56: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 57: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 58: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 59: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 60: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 61: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 62: c679b3770edf4f535ecc97f20a556ee3 otherFunctions
## 63: c679b3770edf4f535ecc97f20a556ee3      preDigest
## 64: c679b3770edf4f535ecc97f20a556ee3      preDigest
## 65: c679b3770edf4f535ecc97f20a556ee3      preDigest
## 66: c679b3770edf4f535ecc97f20a556ee3       accessed
## 67: c679b3770edf4f535ecc97f20a556ee3       accessed
## 68: c679b3770edf4f535ecc97f20a556ee3       accessed
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:53
##  2: 0aa301f9ed377479c7f8328ecc41b33a 2019-07-08 19:59:53
##  3:                          numeric 2019-07-08 19:59:53
##  4:              2019-07-08 19:59:53 2019-07-08 19:59:53
##  5:                 c16f0e2c57596069 2019-07-08 19:59:53
##  6:              rnorm pipe sequence 2019-07-08 19:59:53
##  7:                             1104 2019-07-08 19:59:53
##  8:              2019-07-08 19:59:53 2019-07-08 19:59:53
##  9:                 vweave_rmarkdown 2019-07-08 19:59:53
## 10:                     process_file 2019-07-08 19:59:53
## 11:                    process_group 2019-07-08 19:59:53
## 12:              process_group.block 2019-07-08 19:59:53
## 13:                       call_block 2019-07-08 19:59:53
## 14:                       block_exec 2019-07-08 19:59:53
## 15:                           in_dir 2019-07-08 19:59:53
## 16:                        timing_fn 2019-07-08 19:59:53
## 17:                           handle 2019-07-08 19:59:53
## 18:                      withVisible 2019-07-08 19:59:53
## 19:                              %C% 2019-07-08 19:59:53
## 20:                          do.call 2019-07-08 19:59:53
## 21:               n:c5775c3b366fb719 2019-07-08 19:59:53
## 22:            mean:152602b8ff81e5bb 2019-07-08 19:59:53
## 23:            .FUN:4f604aa46882b368 2019-07-08 19:59:53
## 24:                              rda 2019-07-08 19:59:54
## 25: 78c191d8dec3b97f26fe462521cda3f3 2019-07-08 19:59:54
## 26:                          numeric 2019-07-08 19:59:54
## 27:              2019-07-08 19:59:54 2019-07-08 19:59:54
## 28:                 f7bee22047b8d0c1 2019-07-08 19:59:54
## 29:                                a 2019-07-08 19:59:54
## 30:                            rnorm 2019-07-08 19:59:54
## 31:                             1008 2019-07-08 19:59:54
## 32:              2019-07-08 19:59:54 2019-07-08 19:59:54
## 33:                 vweave_rmarkdown 2019-07-08 19:59:54
## 34:                     process_file 2019-07-08 19:59:54
## 35:                    process_group 2019-07-08 19:59:54
## 36:              process_group.block 2019-07-08 19:59:54
## 37:                       call_block 2019-07-08 19:59:54
## 38:                       block_exec 2019-07-08 19:59:54
## 39:                           in_dir 2019-07-08 19:59:54
## 40:                        timing_fn 2019-07-08 19:59:54
## 41:                           handle 2019-07-08 19:59:54
## 42:                      withVisible 2019-07-08 19:59:54
## 43:               n:7eef4eae85fd9229 2019-07-08 19:59:54
## 44:            .FUN:4f604aa46882b368 2019-07-08 19:59:54
## 45:                              rda 2019-07-08 19:59:53
## 46: c679b3770edf4f535ecc97f20a556ee3 2019-07-08 19:59:53
## 47:                          numeric 2019-07-08 19:59:53
## 48:              2019-07-08 19:59:53 2019-07-08 19:59:53
## 49:                 f11fb1a2880f8060 2019-07-08 19:59:53
## 50:                            rnorm 2019-07-08 19:59:53
## 51:                             1104 2019-07-08 19:59:53
## 52:              2019-07-08 19:59:53 2019-07-08 19:59:53
## 53:                 vweave_rmarkdown 2019-07-08 19:59:53
## 54:                     process_file 2019-07-08 19:59:53
## 55:                    process_group 2019-07-08 19:59:53
## 56:              process_group.block 2019-07-08 19:59:53
## 57:                       call_block 2019-07-08 19:59:53
## 58:                       block_exec 2019-07-08 19:59:53
## 59:                           in_dir 2019-07-08 19:59:53
## 60:                        timing_fn 2019-07-08 19:59:53
## 61:                           handle 2019-07-08 19:59:53
## 62:                      withVisible 2019-07-08 19:59:53
## 63:               n:c5775c3b366fb719 2019-07-08 19:59:53
## 64:            mean:15620f138033a66c 2019-07-08 19:59:53
## 65:            .FUN:4f604aa46882b368 2019-07-08 19:59:53
## 66:              2019-07-08 19:59:53 2019-07-08 19:59:53
## 67:              2019-07-08 19:59:53 2019-07-08 19:59:53
## 68:              2019-07-08 19:59:53 2019-07-08 19:59:53
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.5 Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove, ask = FALSE) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 3 cols): md5hash,name,createdDate
clearCache(tmpDir, ask = FALSE)

1.6 Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay, ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 3262f9dc02b5fba2ad991c7219430e79         format
##  2: 3262f9dc02b5fba2ad991c7219430e79           name
##  3: 3262f9dc02b5fba2ad991c7219430e79          class
##  4: 3262f9dc02b5fba2ad991c7219430e79           date
##  5: 3262f9dc02b5fba2ad991c7219430e79        cacheId
##  6: 3262f9dc02b5fba2ad991c7219430e79     objectName
##  7: 3262f9dc02b5fba2ad991c7219430e79       function
##  8: 3262f9dc02b5fba2ad991c7219430e79    object.size
##  9: 3262f9dc02b5fba2ad991c7219430e79       accessed
## 10: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 11: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 12: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 13: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 14: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 15: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 16: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 17: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 18: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 19: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 20: 3262f9dc02b5fba2ad991c7219430e79      preDigest
## 21: 3262f9dc02b5fba2ad991c7219430e79      preDigest
## 22: 39a699fbc97b792c2e3793781657578a         format
## 23: 39a699fbc97b792c2e3793781657578a           name
## 24: 39a699fbc97b792c2e3793781657578a          class
## 25: 39a699fbc97b792c2e3793781657578a           date
## 26: 39a699fbc97b792c2e3793781657578a        cacheId
## 27: 39a699fbc97b792c2e3793781657578a     objectName
## 28: 39a699fbc97b792c2e3793781657578a       function
## 29: 39a699fbc97b792c2e3793781657578a    object.size
## 30: 39a699fbc97b792c2e3793781657578a       accessed
## 31: 39a699fbc97b792c2e3793781657578a otherFunctions
## 32: 39a699fbc97b792c2e3793781657578a otherFunctions
## 33: 39a699fbc97b792c2e3793781657578a otherFunctions
## 34: 39a699fbc97b792c2e3793781657578a otherFunctions
## 35: 39a699fbc97b792c2e3793781657578a otherFunctions
## 36: 39a699fbc97b792c2e3793781657578a otherFunctions
## 37: 39a699fbc97b792c2e3793781657578a otherFunctions
## 38: 39a699fbc97b792c2e3793781657578a otherFunctions
## 39: 39a699fbc97b792c2e3793781657578a otherFunctions
## 40: 39a699fbc97b792c2e3793781657578a otherFunctions
## 41: 39a699fbc97b792c2e3793781657578a      preDigest
## 42: 39a699fbc97b792c2e3793781657578a      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:55
##  2: 3262f9dc02b5fba2ad991c7219430e79 2019-07-08 19:59:55
##  3:                          numeric 2019-07-08 19:59:55
##  4:              2019-07-08 19:59:55 2019-07-08 19:59:55
##  5:                 f7bee22047b8d0c1 2019-07-08 19:59:55
##  6:                                a 2019-07-08 19:59:55
##  7:                            rnorm 2019-07-08 19:59:55
##  8:                             1008 2019-07-08 19:59:55
##  9:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 10:                 vweave_rmarkdown 2019-07-08 19:59:55
## 11:                     process_file 2019-07-08 19:59:55
## 12:                    process_group 2019-07-08 19:59:55
## 13:              process_group.block 2019-07-08 19:59:55
## 14:                       call_block 2019-07-08 19:59:55
## 15:                       block_exec 2019-07-08 19:59:55
## 16:                           in_dir 2019-07-08 19:59:55
## 17:                        timing_fn 2019-07-08 19:59:55
## 18:                           handle 2019-07-08 19:59:55
## 19:                      withVisible 2019-07-08 19:59:55
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 21:            .FUN:4f604aa46882b368 2019-07-08 19:59:55
## 22:                              rda 2019-07-08 19:59:55
## 23: 39a699fbc97b792c2e3793781657578a 2019-07-08 19:59:55
## 24:                          numeric 2019-07-08 19:59:55
## 25:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 26:                 3aef38d1fc02aee5 2019-07-08 19:59:55
## 27:                                b 2019-07-08 19:59:55
## 28:                            runif 2019-07-08 19:59:55
## 29:                             1008 2019-07-08 19:59:55
## 30:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 31:                 vweave_rmarkdown 2019-07-08 19:59:55
## 32:                     process_file 2019-07-08 19:59:55
## 33:                    process_group 2019-07-08 19:59:55
## 34:              process_group.block 2019-07-08 19:59:55
## 35:                       call_block 2019-07-08 19:59:55
## 36:                       block_exec 2019-07-08 19:59:55
## 37:                           in_dir 2019-07-08 19:59:55
## 38:                        timing_fn 2019-07-08 19:59:55
## 39:                           handle 2019-07-08 19:59:55
## 40:                      withVisible 2019-07-08 19:59:55
## 41:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 42:            .FUN:881ec847b7161f3c 2019-07-08 19:59:55
##                             tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: 3262f9dc02b5fba2ad991c7219430e79         format
##  2: 3262f9dc02b5fba2ad991c7219430e79           name
##  3: 3262f9dc02b5fba2ad991c7219430e79          class
##  4: 3262f9dc02b5fba2ad991c7219430e79           date
##  5: 3262f9dc02b5fba2ad991c7219430e79        cacheId
##  6: 3262f9dc02b5fba2ad991c7219430e79     objectName
##  7: 3262f9dc02b5fba2ad991c7219430e79       function
##  8: 3262f9dc02b5fba2ad991c7219430e79    object.size
##  9: 3262f9dc02b5fba2ad991c7219430e79       accessed
## 10: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 11: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 12: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 13: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 14: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 15: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 16: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 17: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 18: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 19: 3262f9dc02b5fba2ad991c7219430e79 otherFunctions
## 20: 3262f9dc02b5fba2ad991c7219430e79      preDigest
## 21: 3262f9dc02b5fba2ad991c7219430e79      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:55
##  2: 3262f9dc02b5fba2ad991c7219430e79 2019-07-08 19:59:55
##  3:                          numeric 2019-07-08 19:59:55
##  4:              2019-07-08 19:59:55 2019-07-08 19:59:55
##  5:                 f7bee22047b8d0c1 2019-07-08 19:59:55
##  6:                                a 2019-07-08 19:59:55
##  7:                            rnorm 2019-07-08 19:59:55
##  8:                             1008 2019-07-08 19:59:55
##  9:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 10:                 vweave_rmarkdown 2019-07-08 19:59:55
## 11:                     process_file 2019-07-08 19:59:55
## 12:                    process_group 2019-07-08 19:59:55
## 13:              process_group.block 2019-07-08 19:59:55
## 14:                       call_block 2019-07-08 19:59:55
## 15:                       block_exec 2019-07-08 19:59:55
## 16:                           in_dir 2019-07-08 19:59:55
## 17:                        timing_fn 2019-07-08 19:59:55
## 18:                           handle 2019-07-08 19:59:55
## 19:                      withVisible 2019-07-08 19:59:55
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 21:            .FUN:4f604aa46882b368 2019-07-08 19:59:55
##                             tagValue         createdDate
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 0 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 3 cols): md5hash,name,createdDate

1.7 Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 77f28ba78ebb7dc186a3e14ba41aeff7         format
##  2: 77f28ba78ebb7dc186a3e14ba41aeff7           name
##  3: 77f28ba78ebb7dc186a3e14ba41aeff7          class
##  4: 77f28ba78ebb7dc186a3e14ba41aeff7           date
##  5: 77f28ba78ebb7dc186a3e14ba41aeff7        cacheId
##  6: 77f28ba78ebb7dc186a3e14ba41aeff7     objectName
##  7: 77f28ba78ebb7dc186a3e14ba41aeff7       function
##  8: 77f28ba78ebb7dc186a3e14ba41aeff7    object.size
##  9: 77f28ba78ebb7dc186a3e14ba41aeff7       accessed
## 10: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 11: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 12: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 13: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 14: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 15: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 16: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 17: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 18: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 19: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 20: 77f28ba78ebb7dc186a3e14ba41aeff7      preDigest
## 21: 77f28ba78ebb7dc186a3e14ba41aeff7      preDigest
## 22: f16511288cc20c4dd9cc04339f0d72f6         format
## 23: f16511288cc20c4dd9cc04339f0d72f6           name
## 24: f16511288cc20c4dd9cc04339f0d72f6          class
## 25: f16511288cc20c4dd9cc04339f0d72f6           date
## 26: f16511288cc20c4dd9cc04339f0d72f6        cacheId
## 27: f16511288cc20c4dd9cc04339f0d72f6     objectName
## 28: f16511288cc20c4dd9cc04339f0d72f6       function
## 29: f16511288cc20c4dd9cc04339f0d72f6    object.size
## 30: f16511288cc20c4dd9cc04339f0d72f6       accessed
## 31: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 32: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 33: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 34: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 35: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 36: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 37: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 38: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 39: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 40: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 41: f16511288cc20c4dd9cc04339f0d72f6      preDigest
## 42: f16511288cc20c4dd9cc04339f0d72f6      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:55
##  2: 77f28ba78ebb7dc186a3e14ba41aeff7 2019-07-08 19:59:55
##  3:                          numeric 2019-07-08 19:59:55
##  4:              2019-07-08 19:59:55 2019-07-08 19:59:55
##  5:                 f7bee22047b8d0c1 2019-07-08 19:59:55
##  6:                                b 2019-07-08 19:59:55
##  7:                            rnorm 2019-07-08 19:59:55
##  8:                             1008 2019-07-08 19:59:55
##  9:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 10:                 vweave_rmarkdown 2019-07-08 19:59:55
## 11:                     process_file 2019-07-08 19:59:55
## 12:                    process_group 2019-07-08 19:59:55
## 13:              process_group.block 2019-07-08 19:59:55
## 14:                       call_block 2019-07-08 19:59:55
## 15:                       block_exec 2019-07-08 19:59:55
## 16:                           in_dir 2019-07-08 19:59:55
## 17:                        timing_fn 2019-07-08 19:59:55
## 18:                           handle 2019-07-08 19:59:55
## 19:                      withVisible 2019-07-08 19:59:55
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 21:            .FUN:4f604aa46882b368 2019-07-08 19:59:55
## 22:                              rda 2019-07-08 19:59:55
## 23: f16511288cc20c4dd9cc04339f0d72f6 2019-07-08 19:59:55
## 24:                          numeric 2019-07-08 19:59:55
## 25:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 26:                 3aef38d1fc02aee5 2019-07-08 19:59:55
## 27:                                a 2019-07-08 19:59:55
## 28:                            runif 2019-07-08 19:59:55
## 29:                             1008 2019-07-08 19:59:55
## 30:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 31:                 vweave_rmarkdown 2019-07-08 19:59:55
## 32:                     process_file 2019-07-08 19:59:55
## 33:                    process_group 2019-07-08 19:59:55
## 34:              process_group.block 2019-07-08 19:59:55
## 35:                       call_block 2019-07-08 19:59:55
## 36:                       block_exec 2019-07-08 19:59:55
## 37:                           in_dir 2019-07-08 19:59:55
## 38:                        timing_fn 2019-07-08 19:59:55
## 39:                           handle 2019-07-08 19:59:55
## 40:                      withVisible 2019-07-08 19:59:55
## 41:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 42:            .FUN:881ec847b7161f3c 2019-07-08 19:59:55
##                             tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows and 4 cols): artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 77f28ba78ebb7dc186a3e14ba41aeff7         format
##  2: 77f28ba78ebb7dc186a3e14ba41aeff7           name
##  3: 77f28ba78ebb7dc186a3e14ba41aeff7          class
##  4: 77f28ba78ebb7dc186a3e14ba41aeff7           date
##  5: 77f28ba78ebb7dc186a3e14ba41aeff7        cacheId
##  6: 77f28ba78ebb7dc186a3e14ba41aeff7     objectName
##  7: 77f28ba78ebb7dc186a3e14ba41aeff7       function
##  8: 77f28ba78ebb7dc186a3e14ba41aeff7    object.size
##  9: 77f28ba78ebb7dc186a3e14ba41aeff7       accessed
## 10: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 11: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 12: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 13: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 14: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 15: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 16: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 17: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 18: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 19: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 20: 77f28ba78ebb7dc186a3e14ba41aeff7      preDigest
## 21: 77f28ba78ebb7dc186a3e14ba41aeff7      preDigest
## 22: f16511288cc20c4dd9cc04339f0d72f6         format
## 23: f16511288cc20c4dd9cc04339f0d72f6           name
## 24: f16511288cc20c4dd9cc04339f0d72f6          class
## 25: f16511288cc20c4dd9cc04339f0d72f6           date
## 26: f16511288cc20c4dd9cc04339f0d72f6        cacheId
## 27: f16511288cc20c4dd9cc04339f0d72f6     objectName
## 28: f16511288cc20c4dd9cc04339f0d72f6       function
## 29: f16511288cc20c4dd9cc04339f0d72f6    object.size
## 30: f16511288cc20c4dd9cc04339f0d72f6       accessed
## 31: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 32: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 33: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 34: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 35: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 36: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 37: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 38: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 39: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 40: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 41: f16511288cc20c4dd9cc04339f0d72f6      preDigest
## 42: f16511288cc20c4dd9cc04339f0d72f6      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:55
##  2: 77f28ba78ebb7dc186a3e14ba41aeff7 2019-07-08 19:59:55
##  3:                          numeric 2019-07-08 19:59:55
##  4:              2019-07-08 19:59:55 2019-07-08 19:59:55
##  5:                 f7bee22047b8d0c1 2019-07-08 19:59:55
##  6:                                b 2019-07-08 19:59:55
##  7:                            rnorm 2019-07-08 19:59:55
##  8:                             1008 2019-07-08 19:59:55
##  9:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 10:                 vweave_rmarkdown 2019-07-08 19:59:55
## 11:                     process_file 2019-07-08 19:59:55
## 12:                    process_group 2019-07-08 19:59:55
## 13:              process_group.block 2019-07-08 19:59:55
## 14:                       call_block 2019-07-08 19:59:55
## 15:                       block_exec 2019-07-08 19:59:55
## 16:                           in_dir 2019-07-08 19:59:55
## 17:                        timing_fn 2019-07-08 19:59:55
## 18:                           handle 2019-07-08 19:59:55
## 19:                      withVisible 2019-07-08 19:59:55
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 21:            .FUN:4f604aa46882b368 2019-07-08 19:59:55
## 22:                              rda 2019-07-08 19:59:55
## 23: f16511288cc20c4dd9cc04339f0d72f6 2019-07-08 19:59:55
## 24:                          numeric 2019-07-08 19:59:55
## 25:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 26:                 3aef38d1fc02aee5 2019-07-08 19:59:55
## 27:                                a 2019-07-08 19:59:55
## 28:                            runif 2019-07-08 19:59:55
## 29:                             1008 2019-07-08 19:59:55
## 30:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 31:                 vweave_rmarkdown 2019-07-08 19:59:55
## 32:                     process_file 2019-07-08 19:59:55
## 33:                    process_group 2019-07-08 19:59:55
## 34:              process_group.block 2019-07-08 19:59:55
## 35:                       call_block 2019-07-08 19:59:55
## 36:                       block_exec 2019-07-08 19:59:55
## 37:                           in_dir 2019-07-08 19:59:55
## 38:                        timing_fn 2019-07-08 19:59:55
## 39:                           handle 2019-07-08 19:59:55
## 40:                      withVisible 2019-07-08 19:59:55
## 41:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 42:            .FUN:881ec847b7161f3c 2019-07-08 19:59:55
##                             tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm", ask = FALSE)
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 77f28ba78ebb7dc186a3e14ba41aeff7         format
##  2: 77f28ba78ebb7dc186a3e14ba41aeff7           name
##  3: 77f28ba78ebb7dc186a3e14ba41aeff7          class
##  4: 77f28ba78ebb7dc186a3e14ba41aeff7           date
##  5: 77f28ba78ebb7dc186a3e14ba41aeff7        cacheId
##  6: 77f28ba78ebb7dc186a3e14ba41aeff7     objectName
##  7: 77f28ba78ebb7dc186a3e14ba41aeff7       function
##  8: 77f28ba78ebb7dc186a3e14ba41aeff7    object.size
##  9: 77f28ba78ebb7dc186a3e14ba41aeff7       accessed
## 10: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 11: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 12: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 13: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 14: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 15: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 16: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 17: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 18: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 19: 77f28ba78ebb7dc186a3e14ba41aeff7 otherFunctions
## 20: 77f28ba78ebb7dc186a3e14ba41aeff7      preDigest
## 21: 77f28ba78ebb7dc186a3e14ba41aeff7      preDigest
## 22: f16511288cc20c4dd9cc04339f0d72f6         format
## 23: f16511288cc20c4dd9cc04339f0d72f6           name
## 24: f16511288cc20c4dd9cc04339f0d72f6          class
## 25: f16511288cc20c4dd9cc04339f0d72f6           date
## 26: f16511288cc20c4dd9cc04339f0d72f6        cacheId
## 27: f16511288cc20c4dd9cc04339f0d72f6     objectName
## 28: f16511288cc20c4dd9cc04339f0d72f6       function
## 29: f16511288cc20c4dd9cc04339f0d72f6    object.size
## 30: f16511288cc20c4dd9cc04339f0d72f6       accessed
## 31: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 32: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 33: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 34: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 35: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 36: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 37: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 38: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 39: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 40: f16511288cc20c4dd9cc04339f0d72f6 otherFunctions
## 41: f16511288cc20c4dd9cc04339f0d72f6      preDigest
## 42: f16511288cc20c4dd9cc04339f0d72f6      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:55
##  2: 77f28ba78ebb7dc186a3e14ba41aeff7 2019-07-08 19:59:55
##  3:                          numeric 2019-07-08 19:59:55
##  4:              2019-07-08 19:59:55 2019-07-08 19:59:55
##  5:                 f7bee22047b8d0c1 2019-07-08 19:59:55
##  6:                                b 2019-07-08 19:59:55
##  7:                            rnorm 2019-07-08 19:59:55
##  8:                             1008 2019-07-08 19:59:55
##  9:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 10:                 vweave_rmarkdown 2019-07-08 19:59:55
## 11:                     process_file 2019-07-08 19:59:55
## 12:                    process_group 2019-07-08 19:59:55
## 13:              process_group.block 2019-07-08 19:59:55
## 14:                       call_block 2019-07-08 19:59:55
## 15:                       block_exec 2019-07-08 19:59:55
## 16:                           in_dir 2019-07-08 19:59:55
## 17:                        timing_fn 2019-07-08 19:59:55
## 18:                           handle 2019-07-08 19:59:55
## 19:                      withVisible 2019-07-08 19:59:55
## 20:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 21:            .FUN:4f604aa46882b368 2019-07-08 19:59:55
## 22:                              rda 2019-07-08 19:59:55
## 23: f16511288cc20c4dd9cc04339f0d72f6 2019-07-08 19:59:55
## 24:                          numeric 2019-07-08 19:59:55
## 25:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 26:                 3aef38d1fc02aee5 2019-07-08 19:59:55
## 27:                                a 2019-07-08 19:59:55
## 28:                            runif 2019-07-08 19:59:55
## 29:                             1008 2019-07-08 19:59:55
## 30:              2019-07-08 19:59:55 2019-07-08 19:59:55
## 31:                 vweave_rmarkdown 2019-07-08 19:59:55
## 32:                     process_file 2019-07-08 19:59:55
## 33:                    process_group 2019-07-08 19:59:55
## 34:              process_group.block 2019-07-08 19:59:55
## 35:                       call_block 2019-07-08 19:59:55
## 36:                       block_exec 2019-07-08 19:59:55
## 37:                           in_dir 2019-07-08 19:59:55
## 38:                        timing_fn 2019-07-08 19:59:55
## 39:                           handle 2019-07-08 19:59:55
## 40:                      withVisible 2019-07-08 19:59:55
## 41:               n:7eef4eae85fd9229 2019-07-08 19:59:55
## 42:            .FUN:881ec847b7161f3c 2019-07-08 19:59:55
##                             tagValue         createdDate
clearCache(tmpDir, ask = FALSE)

1.8 Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

1.9 Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1] -1.0620433 -0.8208001 -0.2239667
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:a7af5367c13aba8f"
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 504 bytes
##   Selected objects (not including Rasters): 504 bytes
##                             artifact         tagKey
##  1: 8dcc0c64364f76a3f69384aaa68845ca         format
##  2: 8dcc0c64364f76a3f69384aaa68845ca           name
##  3: 8dcc0c64364f76a3f69384aaa68845ca          class
##  4: 8dcc0c64364f76a3f69384aaa68845ca           date
##  5: 8dcc0c64364f76a3f69384aaa68845ca        cacheId
##  6: 8dcc0c64364f76a3f69384aaa68845ca       function
##  7: 8dcc0c64364f76a3f69384aaa68845ca    object.size
##  8: 8dcc0c64364f76a3f69384aaa68845ca       accessed
##  9: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 10: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 11: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 12: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 13: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 14: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 15: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 16: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 17: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 18: 8dcc0c64364f76a3f69384aaa68845ca otherFunctions
## 19: 8dcc0c64364f76a3f69384aaa68845ca      preDigest
## 20: 8dcc0c64364f76a3f69384aaa68845ca      preDigest
## 21: d4d9d9bff75041cf30fdb593229d9ff3         format
## 22: d4d9d9bff75041cf30fdb593229d9ff3           name
## 23: d4d9d9bff75041cf30fdb593229d9ff3          class
## 24: d4d9d9bff75041cf30fdb593229d9ff3           date
## 25: d4d9d9bff75041cf30fdb593229d9ff3        cacheId
## 26: d4d9d9bff75041cf30fdb593229d9ff3       function
## 27: d4d9d9bff75041cf30fdb593229d9ff3    object.size
## 28: d4d9d9bff75041cf30fdb593229d9ff3       accessed
## 29: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 30: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 31: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 32: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 33: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 34: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 35: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 36: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 37: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 38: d4d9d9bff75041cf30fdb593229d9ff3 otherFunctions
## 39: d4d9d9bff75041cf30fdb593229d9ff3      preDigest
## 40: d4d9d9bff75041cf30fdb593229d9ff3      preDigest
## 41: d4d9d9bff75041cf30fdb593229d9ff3      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:56
##  2: 8dcc0c64364f76a3f69384aaa68845ca 2019-07-08 19:59:56
##  3:                          numeric 2019-07-08 19:59:56
##  4:              2019-07-08 19:59:56 2019-07-08 19:59:56
##  5:                 a7af5367c13aba8f 2019-07-08 19:59:56
##  6:                            outer 2019-07-08 19:59:56
##  7:                             1008 2019-07-08 19:59:56
##  8:              2019-07-08 19:59:56 2019-07-08 19:59:56
##  9:                 vweave_rmarkdown 2019-07-08 19:59:56
## 10:                     process_file 2019-07-08 19:59:56
## 11:                    process_group 2019-07-08 19:59:56
## 12:              process_group.block 2019-07-08 19:59:56
## 13:                       call_block 2019-07-08 19:59:56
## 14:                       block_exec 2019-07-08 19:59:56
## 15:                           in_dir 2019-07-08 19:59:56
## 16:                        timing_fn 2019-07-08 19:59:56
## 17:                           handle 2019-07-08 19:59:56
## 18:                      withVisible 2019-07-08 19:59:56
## 19:               n:82dc709f2b91918a 2019-07-08 19:59:56
## 20:            .FUN:892a6afc47a63a90 2019-07-08 19:59:56
## 21:                              rda 2019-07-08 19:59:56
## 22: d4d9d9bff75041cf30fdb593229d9ff3 2019-07-08 19:59:56
## 23:                          numeric 2019-07-08 19:59:56
## 24:              2019-07-08 19:59:56 2019-07-08 19:59:56
## 25:                 4ac2b7c0f42d1e46 2019-07-08 19:59:56
## 26:                            rnorm 2019-07-08 19:59:56
## 27:                             1008 2019-07-08 19:59:56
## 28:              2019-07-08 19:59:56 2019-07-08 19:59:56
## 29:                 vweave_rmarkdown 2019-07-08 19:59:56
## 30:                     process_file 2019-07-08 19:59:56
## 31:                    process_group 2019-07-08 19:59:56
## 32:              process_group.block 2019-07-08 19:59:56
## 33:                       call_block 2019-07-08 19:59:56
## 34:                       block_exec 2019-07-08 19:59:56
## 35:                           in_dir 2019-07-08 19:59:56
## 36:                        timing_fn 2019-07-08 19:59:56
## 37:                           handle 2019-07-08 19:59:56
## 38:                      withVisible 2019-07-08 19:59:56
## 39:               n:7f12988bd88a0fb8 2019-07-08 19:59:56
## 40:            mean:22413394efd9f6a3 2019-07-08 19:59:56
## 41:            .FUN:4f604aa46882b368 2019-07-08 19:59:56
##                             tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 252 bytes
##   Selected objects (not including Rasters): 252 bytes
##                             artifact         tagKey
##  1: bb9692243cac0fb2eacb82b88f4af5ec         format
##  2: bb9692243cac0fb2eacb82b88f4af5ec           name
##  3: bb9692243cac0fb2eacb82b88f4af5ec          class
##  4: bb9692243cac0fb2eacb82b88f4af5ec           date
##  5: bb9692243cac0fb2eacb82b88f4af5ec        cacheId
##  6: bb9692243cac0fb2eacb82b88f4af5ec       function
##  7: bb9692243cac0fb2eacb82b88f4af5ec    object.size
##  8: bb9692243cac0fb2eacb82b88f4af5ec       accessed
##  9: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 10: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 11: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 12: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 13: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 14: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 15: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 16: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 17: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 18: bb9692243cac0fb2eacb82b88f4af5ec otherFunctions
## 19: bb9692243cac0fb2eacb82b88f4af5ec      preDigest
## 20: bb9692243cac0fb2eacb82b88f4af5ec      preDigest
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:56
##  2: bb9692243cac0fb2eacb82b88f4af5ec 2019-07-08 19:59:56
##  3:                          numeric 2019-07-08 19:59:56
##  4:              2019-07-08 19:59:56 2019-07-08 19:59:56
##  5:                 33ceb4fb525fd08f 2019-07-08 19:59:56
##  6:                            inner 2019-07-08 19:59:56
##  7:                             1008 2019-07-08 19:59:56
##  8:              2019-07-08 19:59:56 2019-07-08 19:59:56
##  9:                 vweave_rmarkdown 2019-07-08 19:59:56
## 10:                     process_file 2019-07-08 19:59:56
## 11:                    process_group 2019-07-08 19:59:56
## 12:              process_group.block 2019-07-08 19:59:56
## 13:                       call_block 2019-07-08 19:59:56
## 14:                       block_exec 2019-07-08 19:59:56
## 15:                           in_dir 2019-07-08 19:59:56
## 16:                        timing_fn 2019-07-08 19:59:56
## 17:                           handle 2019-07-08 19:59:56
## 18:                      withVisible 2019-07-08 19:59:56
## 19:            mean:22413394efd9f6a3 2019-07-08 19:59:56
## 20:            .FUN:87e2c30917a34d25 2019-07-08 19:59:56
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1, ask = FALSE)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 756 bytes
##   Selected objects (not including Rasters): 756 bytes
##                             artifact         tagKey
##  1: 1a07751dce71234cb2ea0bd9eb9d2300         format
##  2: 1a07751dce71234cb2ea0bd9eb9d2300           name
##  3: 1a07751dce71234cb2ea0bd9eb9d2300          class
##  4: 1a07751dce71234cb2ea0bd9eb9d2300           date
##  5: 1a07751dce71234cb2ea0bd9eb9d2300        cacheId
##  6: 1a07751dce71234cb2ea0bd9eb9d2300       outerTag
##  7: 1a07751dce71234cb2ea0bd9eb9d2300       function
##  8: 1a07751dce71234cb2ea0bd9eb9d2300    object.size
##  9: 1a07751dce71234cb2ea0bd9eb9d2300       accessed
## 10: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 11: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 12: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 13: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 14: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 15: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 16: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 17: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 18: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 19: 1a07751dce71234cb2ea0bd9eb9d2300 otherFunctions
## 20: 1a07751dce71234cb2ea0bd9eb9d2300      preDigest
## 21: 1a07751dce71234cb2ea0bd9eb9d2300      preDigest
## 22: b98abe006d8d21bacb250b63eb1cd8d0         format
## 23: b98abe006d8d21bacb250b63eb1cd8d0           name
## 24: b98abe006d8d21bacb250b63eb1cd8d0          class
## 25: b98abe006d8d21bacb250b63eb1cd8d0           date
## 26: b98abe006d8d21bacb250b63eb1cd8d0        cacheId
## 27: b98abe006d8d21bacb250b63eb1cd8d0       outerTag
## 28: b98abe006d8d21bacb250b63eb1cd8d0       function
## 29: b98abe006d8d21bacb250b63eb1cd8d0    object.size
## 30: b98abe006d8d21bacb250b63eb1cd8d0       accessed
## 31: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 32: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 33: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 34: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 35: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 36: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 37: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 38: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 39: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 40: b98abe006d8d21bacb250b63eb1cd8d0 otherFunctions
## 41: b98abe006d8d21bacb250b63eb1cd8d0      preDigest
## 42: b98abe006d8d21bacb250b63eb1cd8d0      preDigest
## 43: c4feab88b8bf77f95567b7e0f86e1756         format
## 44: c4feab88b8bf77f95567b7e0f86e1756           name
## 45: c4feab88b8bf77f95567b7e0f86e1756          class
## 46: c4feab88b8bf77f95567b7e0f86e1756           date
## 47: c4feab88b8bf77f95567b7e0f86e1756        cacheId
## 48: c4feab88b8bf77f95567b7e0f86e1756       innerTag
## 49: c4feab88b8bf77f95567b7e0f86e1756       outerTag
## 50: c4feab88b8bf77f95567b7e0f86e1756       function
## 51: c4feab88b8bf77f95567b7e0f86e1756    object.size
## 52: c4feab88b8bf77f95567b7e0f86e1756       accessed
## 53: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 54: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 55: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 56: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 57: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 58: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 59: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 60: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 61: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 62: c4feab88b8bf77f95567b7e0f86e1756 otherFunctions
## 63: c4feab88b8bf77f95567b7e0f86e1756      preDigest
## 64: c4feab88b8bf77f95567b7e0f86e1756      preDigest
## 65: c4feab88b8bf77f95567b7e0f86e1756      preDigest
##                             artifact         tagKey
##                             tagValue         createdDate
##  1:                              rda 2019-07-08 19:59:56
##  2: 1a07751dce71234cb2ea0bd9eb9d2300 2019-07-08 19:59:56
##  3:                          numeric 2019-07-08 19:59:56
##  4:              2019-07-08 19:59:56 2019-07-08 19:59:56
##  5:                 88a34e1d033329e5 2019-07-08 19:59:56
##  6:                         outerTag 2019-07-08 19:59:56
##  7:                            outer 2019-07-08 19:59:56
##  8:                             1008 2019-07-08 19:59:56
##  9:              2019-07-08 19:59:56 2019-07-08 19:59:56
## 10:                 vweave_rmarkdown 2019-07-08 19:59:56
## 11:                     process_file 2019-07-08 19:59:56
## 12:                    process_group 2019-07-08 19:59:56
## 13:              process_group.block 2019-07-08 19:59:56
## 14:                       call_block 2019-07-08 19:59:56
## 15:                       block_exec 2019-07-08 19:59:56
## 16:                           in_dir 2019-07-08 19:59:56
## 17:                        timing_fn 2019-07-08 19:59:56
## 18:                           handle 2019-07-08 19:59:56
## 19:                      withVisible 2019-07-08 19:59:56
## 20:               n:82dc709f2b91918a 2019-07-08 19:59:56
## 21:            .FUN:5f06fb5fbffe9e3b 2019-07-08 19:59:56
## 22:                              rda 2019-07-08 19:59:56
## 23: b98abe006d8d21bacb250b63eb1cd8d0 2019-07-08 19:59:56
## 24:                          numeric 2019-07-08 19:59:56
## 25:              2019-07-08 19:59:56 2019-07-08 19:59:56
## 26:                 b06af03d5a73dc7d 2019-07-08 19:59:56
## 27:                         outerTag 2019-07-08 19:59:56
## 28:                            inner 2019-07-08 19:59:56
## 29:                             1008 2019-07-08 19:59:56
## 30:              2019-07-08 19:59:56 2019-07-08 19:59:56
## 31:                 vweave_rmarkdown 2019-07-08 19:59:56
## 32:                     process_file 2019-07-08 19:59:56
## 33:                    process_group 2019-07-08 19:59:56
## 34:              process_group.block 2019-07-08 19:59:56
## 35:                       call_block 2019-07-08 19:59:56
## 36:                       block_exec 2019-07-08 19:59:56
## 37:                           in_dir 2019-07-08 19:59:56
## 38:                        timing_fn 2019-07-08 19:59:56
## 39:                           handle 2019-07-08 19:59:56
## 40:                      withVisible 2019-07-08 19:59:56
## 41:            mean:22413394efd9f6a3 2019-07-08 19:59:56
## 42:            .FUN:7ad10bc1ae528d8c 2019-07-08 19:59:56
## 43:                              rda 2019-07-08 19:59:56
## 44: c4feab88b8bf77f95567b7e0f86e1756 2019-07-08 19:59:56
## 45:                          numeric 2019-07-08 19:59:56
## 46:              2019-07-08 19:59:56 2019-07-08 19:59:56
## 47:                 4ac2b7c0f42d1e46 2019-07-08 19:59:56
## 48:                         innerTag 2019-07-08 19:59:56
## 49:                         outerTag 2019-07-08 19:59:56
## 50:                            rnorm 2019-07-08 19:59:56
## 51:                             1008 2019-07-08 19:59:56
## 52:              2019-07-08 19:59:56 2019-07-08 19:59:56
## 53:                 vweave_rmarkdown 2019-07-08 19:59:56
## 54:                     process_file 2019-07-08 19:59:56
## 55:                    process_group 2019-07-08 19:59:56
## 56:              process_group.block 2019-07-08 19:59:56
## 57:                       call_block 2019-07-08 19:59:56
## 58:                       block_exec 2019-07-08 19:59:56
## 59:                           in_dir 2019-07-08 19:59:56
## 60:                        timing_fn 2019-07-08 19:59:56
## 61:                           handle 2019-07-08 19:59:56
## 62:                      withVisible 2019-07-08 19:59:56
## 63:               n:7f12988bd88a0fb8 2019-07-08 19:59:56
## 64:            mean:22413394efd9f6a3 2019-07-08 19:59:56
## 65:            .FUN:4f604aa46882b368 2019-07-08 19:59:56
##                             tagValue         createdDate

1.10 cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:7072c305d8c69df0"
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] TRUE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,".Cache")
## attr(,".Cache")$newCache
## [1] FALSE
## 
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

1.11 Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 3.3 Kb
##   Selected objects (not including Rasters): 3.3 Kb

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

2 Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

2.1 Nested Caching

2.1.0.1 Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.