Connect to a local or remote database | Extract | Transform | Load | Do it all at once | Step-by-step | Do Your Analysis | Extending etl | Create your own ETL packages | Use other ETL packages
Package: etl 0.4.3
etl: Extract-Transform-Load Framework for Medium Data
A predictable and pipeable framework for performing ETL (extract-transform-load) operations on publicly-accessible medium-sized data set. This package sets up the method structure and implements generic functions. Packages that depend on this package download specific data sets from the Internet, clean them up, and import them into a local or remote relational database management system.
Authors:
etl_0.4.3.tar.gz
etl_0.4.3.zip(r-4.7)etl_0.4.3.zip(r-4.6)etl_0.4.3.zip(r-4.5)
etl_0.4.3.tgz(r-4.6-any)etl_0.4.3.tgz(r-4.5-any)
etl_0.4.3.tar.gz(r-4.7-any)etl_0.4.3.tar.gz(r-4.6-any)
etl_0.4.3.tgz(r-4.6-emscripten)
manual.pdf |manual.html✨
DESCRIPTION |NEWS
card.svg |card.png
etl/json (API)
| # Install 'etl' in R: |
| install.packages('etl', repos = c('https://beanumber.r-universe.dev', 'https://cloud.r-project.org')) |
Bug tracker:https://github.com/beanumber/etl/issues
Last updated from:a6f568e81d. Checks:9 OK. Indexed: yes.
| Target | Result | Time | Files | Syslog |
|---|---|---|---|---|
| linux-devel-x86_64 | OK | 144 | ||
| source / vignettes | OK | 217 | ||
| linux-release-x86_64 | OK | 158 | ||
| macos-release-arm64 | OK | 100 | ||
| macos-oldrel-arm64 | OK | 91 | ||
| windows-devel | OK | 112 | ||
| windows-release | OK | 102 | ||
| windows-oldrel | OK | 137 | ||
| wasm-release | OK | 128 |
Exports:create_etl_packagedb_typedbRunScriptdbWipeetletl_cleanupetl_createetl_extractetl_initetl_loadetl_transformetl_updateextract_date_from_filenamefind_schemais.etlmatch_files_by_year_monthssmart_downloadsmart_uploadsrc_mysql_cnfvalid_year_month
Dependencies:askpassbitbit64blobclicliprcpp11crayoncredentialscurlDBIdbplyrdescdigestdownloaderdplyrfsgenericsgertgitcredsgluehmshttrhttr2inijanitorjsonlitelifecyclelubridatemagrittrmimeopensslpillarpkgconfigprettyunitsprogresspurrrR6rappdirsreadrrlangrprojrootrstudioapirvestselectrsnakecasestringistringrsystibbletidyrtidyselecttimechangetzdbusethisutf8vctrsvroomwhiskerwithrxml2yamlzip
Using etl
Last update: 2026-02-16
Started: 2017-08-07
Extending etl
The basics | The details | Main etl methods | Other etl methods | etl_foo object attributes | References
Last update: 2025-07-24
Started: 2017-08-07
Readme and manuals
Help Manual
| Help page | Topics |
|---|---|
| Create an ETL package skeleton | create_etl_package |
| Return the database type for an ETL or DBI connection | db_type db_type.DBIConnection db_type.src_dbi |
| Execute an SQL script | dbRunScript |
| Wipe out all tables in a database | dbWipe |
| Initialize an 'etl' object | etl etl.default is.etl print.etl summary.etl |
| ETL functions for working with medium sized data | etl_cleanup etl_cleanup.default etl_create etl_create.default etl_extract etl_extract.default etl_extract.etl_cities etl_extract.etl_mtcars etl_load etl_load.default etl_transform etl_transform.default etl_transform.etl_cities etl_update etl_update.default |
| Initialize a database using a defined schema | etl_init etl_init.default find_schema |
| Match year and month vectors to filenames | extract_date_from_filename match_files_by_year_months |
| Download only those files that don't already exist | smart_download |
| Upload a list of files to the DB | smart_upload |
| Connect to local MySQL Server using ~/.my.cnf | src_mysql_cnf |
| Ensure that years and months are within a certain time span | valid_year_month |
