do not delete accessory relationship while still referenced

[MinerYang]

Thank you for contributing to Harbor!

Comprehensive Summary of your change

Issue being fixed

#20377

Please indicate you've done the following:

• Well Written Title and Summary of the PR
• Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
• Accepted the DCO. Commits without the DCO will delay acceptance.
• Made sure tests are passing and test coverage is added if needed.
• Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 46.33%. Comparing base (b7b8847) to head (065e70f).
Report is 170 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main   #20360       +/-   ##
===========================================
- Coverage   67.56%   46.33%   -21.23%     
===========================================
  Files         991      247      -744     
  Lines      109181    13749    -95432     
  Branches     2719     2830      +111     
===========================================
- Hits        73768     6371    -67397     
+ Misses      31449     7054    -24395     
+ Partials     3964      324     -3640     

Flag	Coverage Δ
unittests	46.33% <ø> (-21.23%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1238 files with indirect coverage changes

[wy65701436]

lgtm