Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export(resource)
export(resource_names)
export(resources)
export(schema)
export(upgrade_package)
export(version)
export(write_package)
import(rlang)
Expand Down
2 changes: 1 addition & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ frictionless now supports Data Packages using the [v2](https://datapackage.org/)

* frictionless no longer relies on v1 properties deprecated in v2, meaning all functions support both versions. They always return a package, resource and schema in the same version as provided.
* `create_package()`, `add_resource()` and `create_schema()` create a v2 package, resource and schema respectively. **This can be a breaking change for some workflows!** Also note that this can lead to mixed versions (e.g. a v1 package with a v2 resource).
* `upgrade_package()` can be used to upgrade a package, its resources and verbose schemas from v1 to v2 (harmonizing mixed versions) (#357, #343, #363).
* `upgrade_package()` can be used to upgrade a package, its resources and verbose schemas from v1 to v2 (harmonizing mixed versions) (#376, #357, #343, #363).

## Changes for v2

Expand Down
4 changes: 2 additions & 2 deletions R/example_package.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@
#' @family sample data
#' @export
#' @examples
#' # Version 2 (default)
#' # Load the v2 example Data Package (default)
#' example_package()
#'
#' # Version 1
#' # Load the v1 example Data Package
#' example_package(version = "1.0")
example_package <- function(version = "2.0") {
version_dir <- switch(
Expand Down
4 changes: 1 addition & 3 deletions R/upgrade_descriptor.R
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#' Upgrade a package object from v1 to v2
#'
#' Upgrades the descriptor, not the associated resources, etc.
#' Upgrades the descriptor, not its resources, etc.
#'
#' @inheritParams read_resource
#' @returns Upgraded `package`.
Expand All @@ -22,7 +22,6 @@ upgrade_descriptor <- function(package) {
)

# Set $schema to profile if URL (to custom profile)
# https://datapackage.org/standard/data-package/#dollar-schema
profile <- package$profile %||% "undefined"
if (is_url(profile)) {
purrr::pluck(package, "$schema") <- profile
Expand All @@ -38,7 +37,6 @@ upgrade_descriptor <- function(package) {
package$profile <- NULL

# Update contributor "role" = "value" to "roles" = ["value"]
# https://datapackage.org/overview/changelog/#packagecontributors-updated
package$contributors <- purrr::modify_if(
package$contributors,
function(x) "role" %in% names(x),
Expand Down
90 changes: 90 additions & 0 deletions R/upgrade_package.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
#' Upgrade a Data Package to v2
#'
#' Upgrades a Data Package, its Data Resources and Table Schemas to the
#' [v2](https://datapackage.org/) specification.
#'
#' @section Upgrade details:
#'
#' ### Data Package
#'
#' `upgrade_package()` upgrades a [v1](
#' https://specs.frictionlessdata.io/data-package/) descriptor to [v2](
#' https://datapackage.org/standard/data-package/) as follows:
#'
#' - Adds `$schema` as first property and sets it to the recommended v2 value
#' (`"https://datapackage.org/profiles/2.0/datapackage.json"`), except for
#' certain `profile` values.
#' - Removes `profile`, but retains its value in `$schema` if it is a URL to a
#' custom profile (see [backwards compatibility](
#' https://datapackage.org/standard/data-package/#dollar-schema)).
#' - Converts `contributors` `"role": "value"` to `"roles": ["value"]` (see
#' [changelog](
#' https://datapackage.org/overview/changelog/#packagecontributors-updated)).
#'
#' ### Data Resource
#'
#' `upgrade_package()` upgrades any [v1](
#' https://specs.frictionlessdata.io/data-resource/) resource to [v2](
#' https://datapackage.org/standard/data-resource/) as follows:
#'
#' - Adds `$schema` as first property and sets it to the recommended v2 value
#' (`"https://datapackage.org/profiles/2.0/dataresource.json"`).
#' - Removes `profile`, but converts `"profile" = "tabular-data-resource"` to
#' `"type" = "table"` (see [backwards compatibility](
#' https://datapackage.org/standard/data-resource/#type)).
#'
#' ### Table Dialect
#'
#' `upgrade_package()` leaves the dialect as is for all resources.
#'
#' ### Table Schema
#'
#' `upgrade_package()` upgrades any verbose [v1](
#' https://specs.frictionlessdata.io/table-schema/) schema to [v2](
#' https://datapackage.org/standard/table-schema/) as follows:
#'
#' - Adds `$schema` as first property and sets it to the recommended v2 value
#' (`"https://datapackage.org/profiles/2.0/tableschema.json"`).
#' - Converts `primaryKey` single values to an array (see [changelog](
#' https://datapackage.org/overview/changelog/#schemaprimarykey-updated)).
#' - Converts `foreignKeys` single values in `fields` to an array and removes
#' `reference$resource` if it is self-referential (see [changelog](
#' https://datapackage.org/overview/changelog/#schemaforeignkeys-updated)).
#'
#' Schemas referenced by path or URL are left as is.
#'
#' @inheritParams read_resource
#' @returns Upgraded `package`.
#' @family versioning functions
#' @export
#' @examples
#' # Load the v1 example Data Package
#' (package <- example_package(version = "1.0"))
#'
#' # Upgrade
#' (package_upgraded <- upgrade_package(package))
#'
#' # Check version of one of the resources
#' version(resource(package_upgraded, "deployments"))
upgrade_package <- function(package) {
check_package(package)

# Upgrade descriptor
package <- upgrade_descriptor(package)

# Upgrade resources
for (resource_name in resource_names(package)) {
resource <- upgrade_resource(resource(package, resource_name))

# Upgrade schema if verbosely included
schema_property <- resource$schema %||% "undefined"
if (is.list(schema_property)) {
schema <- schema(package, resource_name)
schema <- upgrade_schema(schema, resource_name)
resource$schema <- schema
}
resource(package, resource_name) <- resource
}

return(package)
}
1 change: 0 additions & 1 deletion R/upgrade_resource.R
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ upgrade_resource <- function(resource) {
)

# Set type to table if resource is tabular
# https://datapackage.org/standard/data-resource/#type
profile <- resource$profile %||% "undefined"
if (profile %in% c(
"tabular-data-resource",
Expand Down
2 changes: 0 additions & 2 deletions R/upgrade_schema.R
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,9 @@ upgrade_schema <- function(schema, resource_name) {
)

# Update primaryKey to array
# https://datapackage.org/overview/changelog/#schemaprimarykey-updated
schema$primaryKey <- character_to_list(schema$primaryKey)

# Update foreignKeys fields to array and remove self-referential resource
# https://datapackage.org/overview/changelog/#schemaforeignkeys-updated
schema$foreignKeys <- purrr::modify_if(
schema$foreignKeys,
is.list,
Expand Down
2 changes: 1 addition & 1 deletion R/version.R
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
#' @param x A list describing either a Data Package, Data Resource, Table
#' Dialect or Table Schema.
#' @returns Data Package standard version number.
#' @family version functions
#' @family versioning functions
#' @export
#' @examples
#' # Data Package
Expand Down
4 changes: 2 additions & 2 deletions R/write_package.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' Write a Data Package to disk
#'
#' Writes a Data Package and its related Data Resources to disk as a
#' `datapackage.json` and CSV files.
#' Writes a Data Package and its Data Resources to disk as a `datapackage.json`
#' and CSV files.
#'
#' @section Writing data to CSV files:
#'
Expand Down
30 changes: 23 additions & 7 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ navbar:
bg: dark
articles:
- title: Standard implementation
desc: These articles describe how frictionless implements the Data Package standard.
desc: >
These articles describe how frictionless implements the Data Package
standard.
navbar: ~
contents:
- data-package
Expand All @@ -14,22 +16,36 @@ articles:
- table-schema
reference:
- title: Read a Data Package
desc: Read a `datapackage.json` file and its associated Data Resources from path or URL.
desc: >
Read a `datapackage.json` file and its Data Resources from path or URL.
contents:
- has_concept("read functions")
- title: Create and edit a Data Package
desc: Create and edit a Data Package, its associated Data Resources and Table Schemas.
desc: >
Create and edit a Data Package, its Data Resources and Table Schemas.
contents:
- has_concept("create functions")
- has_concept("edit functions")
- title: Write a Data Package
desc: Write a Data Package and its associated Data Resources to disk.
desc: >
Write a Data Package and its Data Resources to disk.
contents:
- has_concept("write functions")
- title: Miscellaneous
- title: Versioning
desc: >
Determine what version of the Data Package standard is used
([v1](https://specs.frictionlessdata.io/) or [v2](https://datapackage.org/))
and/or upgrade. frictionless uses the v2 specification when creating a
package, resource or schema.
contents:
- has_concept("versioning functions")
- title: Developing with frictionless
desc: >
Access, set and check Data Package properties.
contents:
- has_concept("accessor functions")
- has_concept("print functions")
- has_concept("check functions")
- has_concept("version functions")
- title: Miscellaneous
contents:
- has_concept("print functions")
- has_concept("sample data")
4 changes: 2 additions & 2 deletions man/example_package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

81 changes: 81 additions & 0 deletions man/upgrade_package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion man/version.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions man/write_package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion tests/testthat/test-create_package.R
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ test_that("create_package() adds class 'datapackage'", {
})

# Version support ----
test_that("check_package() returns package in same version as provided", {
test_that("create_package() returns package in same version as provided", {
p_v1 <- example_package(version = "1.0")
p_v2 <- example_package(version = "2.0")
expect_identical(version(create_package(p_v1)), "1.0")
Expand Down
Loading
Loading