Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate relationship field names in config as backing columns #2175

Merged
merged 28 commits into from
Apr 30, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f332579
change validation from alias to backing column
aaronburtle Apr 16, 2024
fb5e016
change validation to use backing columns for relationship fields
aaronburtle Apr 18, 2024
e31098b
update commands for creating config in pipeline
aaronburtle Apr 18, 2024
e506806
remove connection string change
aaronburtle Apr 23, 2024
3f6e2b9
fixing tests
aaronburtle Apr 24, 2024
16d0684
fix format for snapshots
aaronburtle Apr 24, 2024
294c388
ignore unused test
aaronburtle Apr 24, 2024
36ce260
conflict for insertion
aaronburtle Apr 24, 2024
7c284bd
mysql syntax
aaronburtle Apr 24, 2024
e724ba3
MySql Syntax
aaronburtle Apr 24, 2024
4bf395c
match name in postgres test
aaronburtle Apr 24, 2024
1251fbb
test language
aaronburtle Apr 24, 2024
9448b46
mysql language
aaronburtle Apr 24, 2024
0012abb
typo in snapshots
aaronburtle Apr 24, 2024
71d42bf
typo in mysql snapshot
aaronburtle Apr 25, 2024
042e129
adjust postgress syntax
aaronburtle Apr 25, 2024
572fc8c
use alias not column name in postgres query
aaronburtle Apr 25, 2024
562ff59
quote case sensitive required names in PGSQL
aaronburtle Apr 25, 2024
119fda5
pgsql syntax
aaronburtle Apr 25, 2024
4a1ff15
Merge branch 'main' into dev/aaronburtle/RelationshipValidationOnBack…
seantleonard Apr 25, 2024
15473fc
revert multi-dab config whitespace
aaronburtle Apr 25, 2024
7c72310
Merge branch 'dev/aaronburtle/RelationshipValidationOnBackingColumns'…
aaronburtle Apr 25, 2024
9c67642
add DW to integration tests
aaronburtle Apr 26, 2024
f43f657
dw tests need new sql
aaronburtle Apr 26, 2024
488c387
fix sql query for DW
aaronburtle Apr 26, 2024
cb51de7
update the DW find api tests
aaronburtle Apr 26, 2024
b2f44da
revert connection string
aaronburtle Apr 26, 2024
fc0f76f
Merge branch 'main' into dev/aaronburtle/RelationshipValidationOnBack…
ayush3797 Apr 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions config-generators/mssql-commands.txt
Original file line number Diff line number Diff line change
Expand Up @@ -124,9 +124,9 @@ update series --config "dab-config.MsSql.json" --relationship comics --target.en
update stocks_price --config "dab-config.MsSql.json" --relationship Stock --target.entity Stock --cardinality one
update Broker --config "dab-config.MsSql.json" --permissions "authenticated:create,update,read,delete" --graphql false
update Tree --config "dab-config.MsSql.json" --rest true --graphql false --permissions "authenticated:create,read,update,delete" --map "species:Scientific Name,region:United State's Region"
update Shrub --config "dab-config.MsSql.json" --permissions "authenticated:create,read,update,delete" --map "species:fancyName"
update Shrub --config "dab-config.MsSql.json" --permissions "authenticated:create,read,update,delete" --map "species:fancyName" --relationship fungus --cardinality one --target.entity Fungus --relationship.fields "species:habitat"
update Fungus --config "dab-config.MsSql.json" --permissions "authenticated:create,read,update,delete" --map "spores:hazards" --rest true
update Fungus --config "dab-config.MsSql.json" --permissions "policy_tester_01:read" --fields.include "*" --policy-database "@item.region ne 'northeast'"
update Fungus --config "dab-config.MsSql.json" --permissions "policy_tester_01:read" --fields.include "*" --policy-database "@item.region ne 'northeast'" --relationship Shrub --cardinality one --target.entity Shrub --relationship.fields "habitat:species"
update books_view_all --config "dab-config.MsSql.json" --permissions "authenticated:create,read,update,delete" --rest true --graphql true
update stocks_view_selected --config "dab-config.MsSql.json" --permissions "authenticated:create,read,update,delete"
update books_publishers_view_composite --config "dab-config.MsSql.json" --permissions "authenticated:create,read,update,delete"
Expand Down
4 changes: 2 additions & 2 deletions config-generators/mysql-commands.txt
Original file line number Diff line number Diff line change
Expand Up @@ -108,9 +108,9 @@ update Comic --config "dab-config.MySql.json" --permissions "authenticated:creat
update series --config "dab-config.MySql.json" --relationship comics --target.entity Comic --cardinality many
update Broker --config "dab-config.MySql.json" --permissions "authenticated:create,update,read,delete" --graphql false
update Tree --config "dab-config.MySql.json" --rest true --graphql false --permissions "authenticated:create,read,update,delete" --map "species:Scientific Name,region:United State's Region"
update Shrub --config "dab-config.MySql.json" --permissions "authenticated:create,read,update,delete" --map "species:fancyName"
update Shrub --config "dab-config.MySql.json" --permissions "authenticated:create,read,update,delete" --map "species:fancyName" --relationship fungus --cardinality one --target.entity Fungus --relationship.fields "species:habitat"
update Fungus --config "dab-config.MySql.json" --permissions "authenticated:create,read,update,delete" --map "spores:hazards" --rest true
update Fungus --config "dab-config.MySql.json" --permissions "policy_tester_01:read" --fields.include "*" --policy-database "@item.region ne 'northeast'"
update Fungus --config "dab-config.MySql.json" --permissions "policy_tester_01:read" --fields.include "*" --policy-database "@item.region ne 'northeast'" --relationship Shrub --cardinality one --target.entity Shrub --relationship.fields "habitat:species"
update books_view_all --config "dab-config.MySql.json" --permissions "authenticated:create,read,update,delete" --rest true --graphql true
update stocks_view_selected --config "dab-config.MySql.json" --permissions "authenticated:create,read,update,delete"
update books_publishers_view_composite --config "dab-config.MySql.json" --permissions "authenticated:create,read,update,delete"
Expand Down
4 changes: 2 additions & 2 deletions config-generators/postgresql-commands.txt
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,9 @@ update Comic --config "dab-config.PostgreSql.json" --permissions "authenticated:
update series --config "dab-config.PostgreSql.json" --relationship comics --target.entity Comic --cardinality many
update Broker --config "dab-config.PostgreSql.json" --permissions "authenticated:create,update,read,delete" --graphql false
update Tree --config "dab-config.PostgreSql.json" --rest true --graphql false --permissions "authenticated:create,read,update,delete" --map "species:Scientific Name,region:United State's Region"
update Shrub --config "dab-config.PostgreSql.json" --permissions "authenticated:create,read,update,delete" --map "species:fancyName"
update Shrub --config "dab-config.PostgreSql.json" --permissions "authenticated:create,read,update,delete" --map "species:fancyName" --relationship fungus --cardinality one --target.entity Fungus --relationship.fields "species:habitat"
update Fungus --config "dab-config.PostgreSql.json" --permissions "authenticated:create,read,update,delete" --map "spores:hazards" --rest true
update Fungus --config "dab-config.PostgreSql.json" --permissions "policy_tester_01:read" --fields.include "*" --policy-database "@item.region ne 'northeast'"
update Fungus --config "dab-config.PostgreSql.json" --permissions "policy_tester_01:read" --fields.include "*" --policy-database "@item.region ne 'northeast'" --relationship Shrub --cardinality one --target.entity Shrub --relationship.fields "habitat:species"
update books_view_with_mapping --config "dab-config.PostgreSql.json" --map "id:book_id"
update BookWebsitePlacement --config "dab-config.PostgreSql.json" --relationship books --target.entity Book --cardinality one
update SupportedType --config "dab-config.PostgreSql.json" --map "id:typeid" --permissions "authenticated:create,read,delete,update"
Expand Down
8 changes: 5 additions & 3 deletions src/Core/Configurations/RuntimeConfigValidator.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1078,9 +1078,9 @@ public void ValidateRelationshipsInConfig(RuntimeConfig runtimeConfig, IMetadata
/// works because C# is pass by reference for referenced class types.
/// </summary>
/// <param name="invalidColumns">List in which to aggregate the invalid fields.</param>
/// <param name="fields">List of the fields to check for existence in backing DB.</param>
/// <param name="fields">List of the backing fields to check for existence in backing DB.</param>
/// <param name="entityName">The name of the entity that we check for backing columns.</param>
/// <param name="sqlMetadataProvider">The sqlMetadataProvider used to lookup if the fields are valid columns in DB.</param>
/// <param name="sqlMetadataProvider">The sqlMetadataProvider used to lookup if the backing fields are valid columns in DB.</param>
aaronburtle marked this conversation as resolved.
Show resolved Hide resolved
private static void GetFieldsNotBackedByColumnsInDB(
List<string> invalidColumns,
string[] fields,
Expand All @@ -1090,7 +1090,9 @@ public void ValidateRelationshipsInConfig(RuntimeConfig runtimeConfig, IMetadata
invalidColumns.Clear();
foreach (string field in fields)
{
if (!sqlMetadataProvider.TryGetBackingColumn(entityName, field, out _))
// We call this function because the keyset are the backing columns
// which is what want to validate.
if (!sqlMetadataProvider.TryGetExposedColumnName(entityName, field, out _))
{
invalidColumns.Add(field);
}
Expand Down
3 changes: 3 additions & 0 deletions src/Service.Tests/DatabaseSchema-MsSql.sql
Original file line number Diff line number Diff line change
Expand Up @@ -522,8 +522,11 @@ INSERT INTO stocks(categoryid, pieceid, categoryName) VALUES (1, 1, 'SciFi'), (2
INSERT INTO stocks_price(categoryid, pieceid, price, is_wholesale_price) VALUES (2, 1, 100.57, 1), (1, 1, 42.75, 0), (100, 99, NULL, NULL);
INSERT INTO stocks_price(categoryid, pieceid, instant, price, is_wholesale_price) VALUES (2, 1, '2023-08-21 15:11:04', 100.57, 1);
INSERT INTO trees(treeId, species, region, height) VALUES (1, 'Tsuga terophylla', 'Pacific Northwest', '30m'), (2, 'Pseudotsuga menziesii', 'Pacific Northwest', '40m');
INSERT INTO trees(treeId, species, region, height) VALUES (4, 'test', 'Pacific Northwest', '0m');
INSERT INTO aow(NoteNum, DetailAssessmentAndPlanning, WagingWar, StrategicAttack) VALUES (1, 'chapter one notes: ', 'chapter two notes: ', 'chapter three notes: ');
INSERT INTO fungi(speciesid, region, habitat) VALUES (1, 'northeast', 'forest'), (2, 'southwest', 'sand');
INSERT INTO fungi(speciesid, region, habitat) VALUES (3, 'northeast', 'test');


SET IDENTITY_INSERT authors_history ON
INSERT INTO authors_history(id, first_name, middle_name, last_name, year_of_publish, books_published)
Expand Down
2 changes: 2 additions & 0 deletions src/Service.Tests/DatabaseSchema-MySql.sql
Original file line number Diff line number Diff line change
Expand Up @@ -378,7 +378,9 @@ INSERT INTO type_table(id, byte_types, short_types, int_types, long_types, strin
(4, 255, 32767, 2147483647, 9223372036854775807, 'null', 3.4E38, 1.7E308, 2.929292E-14, true, '9999-12-31 23:59:59', 0xFFFFFFFF),
(5, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
INSERT INTO trees(treeId, species, region, height) VALUES (1, 'Tsuga terophylla', 'Pacific Northwest', '30m'), (2, 'Pseudotsuga menziesii', 'Pacific Northwest', '40m');
INSERT INTO trees(treeId, species, region, height) VALUES (4, 'test', 'Pacific Northwest', '0m');
INSERT INTO fungi(speciesid, region, habitat) VALUES (1, 'northeast', 'forest'), (2, 'southwest', 'sand');
INSERT INTO fungi(speciesid, region, habitat) VALUES (3, 'northeast', 'test');
INSERT INTO notebooks(id, notebookname, color, ownername) VALUES (1, 'Notebook1', 'red', 'Sean'), (2, 'Notebook2', 'green', 'Ani'), (3, 'Notebook3', 'blue', 'Jarupat'), (4, 'Notebook4', 'yellow', 'Aaron');
INSERT INTO journals(id, journalname, color, ownername)
VALUES
Expand Down
2 changes: 2 additions & 0 deletions src/Service.Tests/DatabaseSchema-PostgreSql.sql
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,9 @@ INSERT INTO type_table(id, short_types, int_types, long_types, string_types, sin
(5, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
INSERT INTO type_table(id, uuid_types) values(10, 'D1D021A8-47B4-4AE4-B718-98E89C41A161');
INSERT INTO trees("treeId", species, region, height) VALUES (1, 'Tsuga terophylla', 'Pacific Northwest', '30m'), (2, 'Pseudotsuga menziesii', 'Pacific Northwest', '40m');
INSERT INTO trees("treeId", species, region, height) VALUES (4, 'test', 'Pacific Northwest', '0m');
INSERT INTO fungi(speciesid, region, habitat) VALUES (1, 'northeast', 'forest'), (2, 'southwest', 'sand');
INSERT INTO fungi(speciesid, region, habitat) VALUES (3, 'northeast', 'test');
INSERT INTO notebooks(id, noteBookName, color, ownerName) VALUES (1, 'Notebook1', 'red', 'Sean'), (2, 'Notebook2', 'green', 'Ani'), (3, 'Notebook3', 'blue', 'Jarupat'), (4, 'Notebook4', 'yellow', 'Aaron');
INSERT INTO journals(id, journalname, color, ownername)
VALUES
Expand Down