Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rails 7.1 compatibility #601

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

marcoroth
Copy link

Hey there 👋🏼

First of all, thanks for the long-running support and continued maintenance of the composite_primary_keys gem! 🙏🏼

While following Rails Edge in one of my apps I noticed that the pull request rails/rails#48241 changed the method signature for ConnectionAdapters#sql_for_insert from three to four arguments and Persitance#_insert_record from one to two arguments.

This caused applications using composite_primary_keys fail to boot and crash with:

ArgumentError: wrong number of arguments (given 4, expected 3) 
lib/composite_primary_keys/connection_adapters/postgresql/database_statements.rb:5:in 'sql_for_insert'`
Full Backtrace
bundle exec rails db:setup

Created database '[project]_test'
rake aborted!
ArgumentError: wrong number of arguments (given 4, expected 3)
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/composite_primary_keys-1ffb4340bc01/lib/composite_primary_keys/connection_adapters/postgresql/database_statements.rb:5:in `sql_for_insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:152:in `exec_insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/connection_adapters/postgresql/database_statements.rb:96:in `exec_insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/composite_primary_keys-1ffb4340bc01/lib/composite_primary_keys/connection_adapters/abstract/database_statements.rb:6:in `insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/connection_adapters/abstract/query_cache.rb:25:in `insert'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/internal_metadata.rb:123:in `create_entry'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/internal_metadata.rb:106:in `update_or_create_entry'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/internal_metadata.rb:68:in `create_table_and_set_flags'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/schema.rb:62:in `define'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/schema.rb:50:in `define'
/home/runner/work/[project]/[project]/db/schema.rb:13:in `'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:358:in `load'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:358:in `load_schema'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:453:in `block (2 levels) in load_schema_current'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:501:in `block in with_temporary_connection'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:518:in `with_temporary_pool'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:500:in `with_temporary_connection'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:452:in `block in load_schema_current'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:562:in `block (2 levels) in each_current_configuration'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:559:in `each'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:559:in `block in each_current_configuration'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:558:in `each'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:558:in `each_current_configuration'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/tasks/database_tasks.rb:451:in `load_schema_current'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-48e4d99c75b7/activerecord/lib/active_record/railties/databases.rake:476:in `block (3 levels) in '
:88:in `require'
:88:in `require'
-e:1:in `'
Tasks: TOP => db:setup => db:schema:load
(See full trace by running task with --trace)
Error: Process completed with exit code 1.

And

ArgumentError: wrong number of arguments (given 1, expected 2)
rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:564:in `_insert_record'`
lib/composite_primary_keys/persistence.rb:77:in `_create_record'
Full Backtrace
bundle exec rails db:setup

Created database '[project]_test'
Running via Spring preloader in process 2631
rake aborted!
ArgumentError: wrong number of arguments (given 1, expected 2)
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:564:in _insert_record' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/composite_primary_keys-2a117c40935c/lib/composite_primary_keys/persistence.rb:77:in _create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/counter_cache.rb:187:in _create_record' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/locking/optimistic.rb:84:in _create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/encryption/encryptable_record.rb:174:in _create_record' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/attribute_methods/dirty.rb:236:in _create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:445:in block in _create_record' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:110:in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:949:in _run_create_callbacks' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:445:in _create_record'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/timestamp.rb:114:in _create_record' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:1212:in create_or_update'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:441:in block in create_or_update' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:121:in block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/autosave_association.rb:379:in around_save_collection_association' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:130:in block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:141:in run_callbacks' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:949:in _run_save_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/callbacks.rb:441:in create_or_update' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/timestamp.rb:125:in create_or_update'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:712:in save' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/validations.rb:49:in save'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:309:in block in save' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:365:in block in with_transaction_returning_status'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:340:in transaction' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:361:in with_transaction_returning_status'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:309:in save' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/suppressor.rb:52:in save'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/persistence.rb:38:in create' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:899:in _create'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:103:in block in create' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:914:in _scoping'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:452:in scoping' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:103:in create'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:216:in block in create_or_find_by' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:487:in block in within_new_transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/concurrency/null_lock.rb:9:in synchronize' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/transaction.rb:484:in within_new_transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:342:in transaction' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/transactions.rb:212:in transaction'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation/delegation.rb:122:in public_send' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation/delegation.rb:122:in block in method_missing'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:914:in _scoping' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:452:in scoping'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation/delegation.rb:122:in method_missing' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:216:in create_or_find_by'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/relation.rb:176:in find_or_create_by' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/querying.rb:23:in find_or_create_by'
/home/runner/work/[project]/[project]/db/seeds.rb:22:in block in <top (required)>' /home/runner/work/[project]/[project]/db/seeds.rb:21:in each'
/home/runner/work/[project]/[project]/db/seeds.rb:21:in <top (required)>' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:556:in load'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:556:in block in load_seed' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:121:in block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/execution_wrapper.rb:92:in wrap' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:642:in block (2 levels) in class:Engine'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:130:in instance_exec' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:130:in block in run_callbacks'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activesupport/lib/active_support/callbacks.rb:141:in run_callbacks' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/railties/lib/rails/engine.rb:556:in load_seed'
/home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/tasks/database_tasks.rb:468:in load_seed' /home/runner/work/[project]/[project]/vendor/bundle/ruby/3.2.0/bundler/gems/rails-99f8b049f28f/activerecord/lib/active_record/railties/databases.rake:405:in block (2 levels) in <top (required)>'
internal:/opt/hostedtoolcache/Ruby/3.2.2/x64/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb:85:in require' <internal:/opt/hostedtoolcache/Ruby/3.2.2/x64/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in require'
-e:1:in `

'
Tasks: TOP => db:setup => db:seed
(See full trace by running task with --trace)
Error: Process completed with exit code 1.

This pull requests updates the methods signature for sql_for_insert in composite_primary_keys so it could take either 3 or 4 arguments and the call to _insert_record from one to two inside _create_record.

I'm also aware that Rails 7.1 is going to get some kind of support for composite primary keys, but I guess it would still make sense for composite_primary_keys to be Rails 7.1+ compatible to provide a smoother upgrade/migration path.

I'm not sure if this is the proper and correct way to fix this. At least it's making my apps test pass and lets the app boot again. So I think this pull request is at least a starting point for making composite_primary_keys Rails 7.1 compatible.

rails/rails#48241 changed the method signature for `sql_for_insert` from three to four arguments. This caused some applications to fail with:

`ArgumentError: wrong number of arguments (given 4, expected 3)` in `lib/composite_primary_keys/connection_adapters/postgresql/database_statements.rb:5:in `sql_for_insert'`
new_id = self.class._insert_record(
attributes_with_values(attribute_names)
attributes_with_values(attribute_names),
returning_columns_for_insert
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a low-effort "fix" from my side.

For reference, upstream changed this from

       new_id = self.class._insert_record(
         attributes_with_values(attribute_names)
       )

       self.id ||= new_id if @primary_key

to

       returning_columns = self.class._returning_columns_for_insert

       returning_values = self.class._insert_record(
         attributes_with_values(attribute_names),
         returning_columns
       )

       returning_columns.zip(returning_values).each do |column, value|
         _write_attribute(column, value) if !_read_attribute(column)
       end if returning_values

@cfis
Copy link
Contributor

cfis commented Jul 18, 2023

Thanks for the PR. Looks like we also need to implement the other changes though including using returning parameter?

@marcoroth
Copy link
Author

If I understand the versioning right we can "break" things in a new major version since every version is just designed to work with one version of ActiveRecord, right?

So a potential 15.x release doesn't have to be compatible with ActiveRecord 7.0

@cfis
Copy link
Contributor

cfis commented Jul 25, 2023

Yes, that is correct. Are there big differences between 7.0 and 7.1?

Note the tests failed with this change.

@machty
Copy link

machty commented Sep 13, 2023

Hello,

I'm seeing the following in Rails 7.1 beta notes:

Support for composite primary keys in Active Record

Shopify improved the performance of common queries against their largest tables by 5-6x and reduced the number of slow queries by 80% by switching to composite primary keys. The trade-off is that inserts can become significantly slower, but for very large tables that see many more reads than writes, it can be a dramatic improvement. This work has been extracted into full support for composite primary keys in Active Record.

What does this mean for this gem? Does Rails 7.1 obsolete this gem?

@cfis
Copy link
Contributor

cfis commented Sep 19, 2023

Good question - don't know. Hopefully.

@igor-alexandrov
Copy link

I am not sure about performance of composite_primary_keys VS composite primary keys in AR 7.1 or any other aspects, but from it seems like with Rails 7.1 this gems becomes obsolete. The only difference is that in Rails 7.1 you need to use self.primary_key = [] instead of self.primary_keys = [] with this gem.

@igor-alexandrov
Copy link

More about composite keys support here: https://guides.rubyonrails.org/active_record_composite_primary_keys.html

@cfis
Copy link
Contributor

cfis commented Oct 10, 2023

It seems from #608 there isn't any reason to update CPK for Rails 7.1. Thoughts?

@marcoroth
Copy link
Author

I think it might make sense to support 7.1 so you have an easier upgrade path. So that you can upgrade to Rails 7.1 without also having to migrate to the new built-in composite primary keys at the same time.

@cfis
Copy link
Contributor

cfis commented Oct 11, 2023

Maybe, but it sounded from the other issue the upgrade was pretty easy. But I haven't tried it myself.

@codeodor
Copy link
Collaborator

When I get around to it, I'll see if I can upgrade my app by changing CPK to just add the code:

def self.primary_keys = (array)
  self.primary_key = array
end

Which should theoretically work?

@cfis
Copy link
Contributor

cfis commented Oct 11, 2023 via email

@jarl-dk
Copy link

jarl-dk commented Oct 12, 2023

FYI: I have opened a bug report on rails regarding composite primary keys:
rails/rails#49597

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants