Let each database dialect describe its own column types, fixes #7997

[hansva]

The goal of this PR is to let database dialects map their own types.

This removes the need to use isDatabaseVariant flags inside Database and ValueMeta.
It also introduces a plugin system where developers can add extra mappings for specific database types.
For example, if there is a plugin that wants to introduce Oracle GEOMETRY mapping for their own Hop Value types, they can do so without having to change core code.

This PR also contains golden sets containing the current mapping of Hop types to database types.
It can be used to find current gaps and fix them

All existing code has remained untouched, keeping everything backwards compatible.

Size of the PR

Golden sets to identify mistakes/see conversion results

Most of the code is generated golden sets, which will tell you how each database type is now mapping hop types -> database types. This can be used to identify mistakes.

e.g., PostgreSQL turns integers without length into DOUBLE PRECISION fields:

INTEGER      len=      -1 prec=  -1 -        -> DOUBLE PRECISION
INTEGER      len=       0 prec=   0 -        -> DOUBLE PRECISION

classloader change

It also introduces classloader groups on all database types so external plugins can safely hook into the same classloaders to add rules.

new plugin type

A DatabaseTypeRulesPlugin type has been added to introduce new rules on existing database types

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
• If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
• Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

• I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004
• In any other case, please file an Apache Individual Contributor License Agreement.

[mattcasters]

Issue #7997 is interesting. Related but not the same: I do think that data engineers will want to define (override) Hop Type to Database column types themselves. This is behind work done here: https://github.com/mattcasters/hop-data-vault/tree/main/src/main/java/org/apache/hop/datavault/metadata/targettypemapping

[hansva]

That's interesting indeed!

[mattcasters]

Mind you, it needs support tooling to use this metadata like the DDL transform, or in the case of the plugin, an action that can automatically generate the DDL/DML for data vault, business vault or dimensional model tables and indexes.

[hansva]

I'll investigate a bit further, but I think we are both somewhat on the same path here.

There will be a registry with rules that are being applied to do bidirectional type mapping, and external plugins can add their own mappings.
The only difference I see here is that what's currently in this PR is at a global level, and the TargetTypeMappingRule in your code is at the connection level, which is a nice user-level override.

I'll make sure the code keeps playing along nicely, we should be moving forward not backwards with these types of changes

[mattcasters]

Both inbound and outbound data type mappings can live in user-space. They should never be conflated and be source specific, not even databsae specific. For example, Iceberge v1/2 doesn't provide lengths and precisions for data types. This means that it's something that users would want to supply themselves before the data ends up in an EDW.

All that I see as being separate to what you seem to be doing here, providing better support for, allowing rules to be defined for data typing by database plugins. I think it's a nice improvement. Don't worry about the hop-data-vault code, it's handling a separate challenge: source --> data catalog --> target