Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Connection.GetSchema #185

Merged
merged 16 commits into from
May 20, 2024
Merged

Implement Connection.GetSchema #185

merged 16 commits into from
May 20, 2024

Conversation

hazzik
Copy link
Contributor

@hazzik hazzik commented May 13, 2024

This PR implements Connection.GetSchema.

Standard collections:
✔️ MetaDataCollections
✔️ Restrictions
✔️ ReservedWords
❌ DataSourceInformation
❌ DataTypes

Non standard common collections
✔️ Tables
✔️ Columns
✔️ ForeignKeys
✔️ Indexes

@codecov-commenter
Copy link

codecov-commenter commented May 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.41%. Comparing base (6270bac) to head (1033016).
Report is 35 commits behind head on develop.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #185      +/-   ##
===========================================
+ Coverage    87.83%   88.41%   +0.58%     
===========================================
  Files           42       54      +12     
  Lines         1504     1761     +257     
  Branches       217      239      +22     
===========================================
+ Hits          1321     1557     +236     
- Misses         127      146      +19     
- Partials        56       58       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coveralls
Copy link

coveralls commented May 13, 2024

Pull Request Test Coverage Report for Build 9155702909

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 121 of 121 (100.0%) changed or added relevant lines in 2 files are covered.
  • 8 unchanged lines in 4 files lost coverage.
  • Overall coverage increased (+0.7%) to 90.216%

Files with Coverage Reduction New Missed Lines %
DuckDB.NET.Data/DuckDBCommand.cs 1 89.83%
DuckDB.NET.Data/Internal/Reader/VectorDataReaderBase.cs 1 88.27%
DuckDB.NET.Data/Internal/PreparedStatement.cs 3 93.33%
DuckDB.NET.Data/Internal/Reader/DateTimeVectorDataReader.cs 3 90.2%
Totals Coverage Status
Change from base Build 9119577775: 0.7%
Covered Lines: 1615
Relevant Lines: 1761

💛 - Coveralls

@Giorgi
Copy link
Owner

Giorgi commented May 13, 2024

Is it still WIP?

@hazzik
Copy link
Contributor Author

hazzik commented May 13, 2024

Yes, want to implement a few more essential collections.

@Giorgi Giorgi marked this pull request as draft May 14, 2024 07:31
@Giorgi
Copy link
Owner

Giorgi commented May 14, 2024

I'll mark it as Draft. Please mark it ready for review when it's finished. Thanks

@Giorgi Giorgi changed the title WIP Implement Connection.GetSchema Implement Connection.GetSchema May 14, 2024
@hazzik hazzik marked this pull request as ready for review May 17, 2024 08:33
@hazzik
Copy link
Contributor Author

hazzik commented May 17, 2024

Ok. This is ready for review now. I've implemented almost all metadata I wanted, except IndexColumns, that seems to be impossible to implement. Also, I have not implemented DataTypes and DataSourceInformation standard collections as I don't have interest in them. If that's is required it would be possible to implement later.

@Giorgi
Copy link
Owner

Giorgi commented May 17, 2024

Thanks, I'll try to review it in the next few days. Is there any documentation on how GetSchema should work and what kind of structure it should return?

@hazzik
Copy link
Contributor Author

hazzik commented May 17, 2024

The documentation is really loose, the basic description of how it should work is here: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/retrieving-database-schema-information.

With common schema collections and custom collections that are available via informational_schema catalog it is more-or-less settled, but for custom collections it becomes trickier (in context of this PR it is only Indexes collection): the column names and information do not often match between providers.

Copy link
Owner

@Giorgi Giorgi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good! Left some minor comments.

DuckDB.NET.Data/DuckDBConnection.cs Outdated Show resolved Hide resolved
{ DuckDbMetaDataCollectionNames.Tables, 4, 3 },
{ DuckDbMetaDataCollectionNames.Columns, 4, 4 },
{ DuckDbMetaDataCollectionNames.ForeignKeys, 4, 3 },
{ DuckDbMetaDataCollectionNames.Indexes, 4, 3 },
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do these numbers mean? Can you add a comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is from columns definition above:

                { DbMetaDataColumnNames.CollectionName, typeof(string) },
                { DbMetaDataColumnNames.NumberOfRestrictions, typeof(int) },
                { DbMetaDataColumnNames.NumberOfIdentifierParts, typeof(int) }

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do the numbers 4 and 3 refer too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first number says how many restrictions could the collection have (it is about restrictionValues/restrictionNames arrays). The last number is purely for information purposes. It says that index has 3 components in its full name: catalog.schema.index_name.

DuckDB.NET.Data/DuckDBSchema.cs Show resolved Hide resolved
const string query = "SELECT table_catalog, table_schema, table_name, table_type FROM information_schema.tables";

using var command = BuildCommand(connection, query, restrictionValues, true,
["table_catalog", "table_schema", "table_name", "table_type"]);
Copy link
Owner

@Giorgi Giorgi May 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the numbers above refer to the length of these arrays, can you extract these arrays somewhere and refer to their Length in GetMetaDataCollections ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the numbers above refer to the length of these arrays

Yes,

can you extract these arrays somewhere and refer to their Length in GetMetaDataCollections

I could, try, but it would just complicate the code, I afraid, without providing any benefit.

@Giorgi Giorgi merged commit d333690 into Giorgi:develop May 20, 2024
10 checks passed
@Giorgi
Copy link
Owner

Giorgi commented May 20, 2024

Thanks, merged.

@hazzik
Copy link
Contributor Author

hazzik commented May 20, 2024

Thank you

@Giorgi
Copy link
Owner

Giorgi commented May 22, 2024

@hazzik This is now live on NuGet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants