Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce TransferCommitment API #410

Merged
merged 42 commits into from Mar 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
fe61148
Add Skeleton transferCommitment endpoint
VoigtS Mar 1, 2024
fcaab34
Add token check
VoigtS Mar 1, 2024
e3c70cf
Commitment: Implement commitment GET from DB. Test: Implement DB insert
VoigtS Mar 4, 2024
98aa497
Test: define working request/response objects
VoigtS Mar 4, 2024
51c0c8a
Add event logging
VoigtS Mar 4, 2024
dcea033
Generate transfer token: Implement function to be used in unit test a…
VoigtS Mar 5, 2024
6c43ca1
Add missing changes from last commit
VoigtS Mar 5, 2024
91bb534
Finalize: start-transfer API
VoigtS Mar 5, 2024
c8a94a8
Add transferStatus and token to splitted commitment
VoigtS Mar 5, 2024
4cc8f6c
Check: unlisted or public status and amount (greater zero)
VoigtS Mar 5, 2024
811f866
Add sekelton for transfer-commitment API
VoigtS Mar 6, 2024
79a988f
Implement Transfer Commitment API logic
VoigtS Mar 6, 2024
b70b3de
Only allow commitment transfer on confirmed commitments
VoigtS Mar 11, 2024
30b6ad4
Fix typo in message: project -> commitment
VoigtS Mar 11, 2024
1278203
Update API documentation
VoigtS Mar 11, 2024
94c4816
Update docs/users/api-spec-resources.md
VoigtS Mar 11, 2024
f4e5423
Update docs/users/api-spec-resources.md
VoigtS Mar 11, 2024
810c25a
Update docs/users/api-spec-resources.md
VoigtS Mar 11, 2024
fe53bb5
Update docs/users/api-spec-resources.md
VoigtS Mar 11, 2024
42b6992
Update internal/api/utils.go
VoigtS Mar 11, 2024
94703ba
Return error if project not found
VoigtS Mar 11, 2024
d4c3635
Tests: Change time usage to clock usage
VoigtS Mar 11, 2024
ddc1bca
Add missing error handlings
VoigtS Mar 11, 2024
2f94e47
Update docs/users/api-spec-resources.md
VoigtS Mar 15, 2024
6c3fbe8
Update internal/api/commitment.go
VoigtS Mar 15, 2024
66b7e54
Update internal/api/commitment.go
VoigtS Mar 15, 2024
ccc016d
Update internal/api/commitment.go
VoigtS Mar 15, 2024
726f248
Update internal/api/commitment_test.go
VoigtS Mar 15, 2024
00d9ddb
Fix: Remaining Delta from the PR code suggestions
VoigtS Mar 15, 2024
5652337
Change naming: GenerateToken to GenerateTransferToken
VoigtS Mar 15, 2024
c777d8d
Fix: start transfer parseTarget struct now uses explicit fields.
VoigtS Mar 15, 2024
abf3a0f
Fix: Errors now returned as text.
VoigtS Mar 15, 2024
7a233cc
Request with amount > commitment amount will be treated as an error
VoigtS Mar 15, 2024
92de216
Update internal/api/commitment.go
VoigtS Mar 15, 2024
8af9563
Fix: SQL update now takes both return values into account
VoigtS Mar 15, 2024
944f2e6
Fix: remove unused sql update query
VoigtS Mar 15, 2024
f49743e
Fix: Remove SQL update queries and replace them with proper ORM
VoigtS Mar 15, 2024
0cfc807
Fix docstring quotes
VoigtS Mar 15, 2024
2553e6c
Fix: Transfer Commitment did not use service or resource identifier
VoigtS Mar 19, 2024
fa512f6
Transfer commitment: no uses commitment ID and token to get the sourc…
VoigtS Mar 19, 2024
fc307ae
Transfer commitment: now uses transfer token in header - not as query…
VoigtS Mar 19, 2024
7765102
Update transfer commitment documentation
VoigtS Mar 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
31 changes: 31 additions & 0 deletions docs/users/api-spec-resources.md
Expand Up @@ -752,6 +752,37 @@ Returns 200 (OK) on success, and a JSON document like `{"result":true}` or `{"re

The `result` field indicates whether this commitment can be created without a `confirm_by` attribute, that is, confirmed immediately upon creation.

### POST /v1/domains/:id/projects/:id/commitments/:id/start-transfer
Prepares a commitment to be transferred from a source project to a target project. Requires a project-admin token, and a request body that is a JSON document like:
```json
{
"commitment": {
"amount": 100,
"transfer_status": "unlisted"
}
}
```
If the amount to transfer is equal to the commitment, the whole commitment will be marked as transferrable. If the amount is less than the commitment, the commitment will be split in two and the requested amount will be marked as transferrable.
The transfer status indicates if the commitment stays `unlisted` (private) or `public`.
The response is a JSON of the commitment including the following fields that identify a commitment in its transferrable state:
```json
{
"commitment": {
"transfer_token": "token",
"transfer_status": "unlisted"
}
}
```
### POST /v1/domains/:id/projects/:id/transfer-commitment/:id
Transfers the commitment from a source project to a target project.
Requires a project-admin token.
Requires a transfer token in the request header:
`Transfer-Token: [value]`.
This endpoint receives the target project ID, but the commitment ID from the source project.
Requires a generated token from the API: `/v1/domains/:id/projects/:id/commitments/:id/start-transfer`.
On success the API clears the `transfer_token` and `transfer_status` from the commitment.
After that, it returns the commitment as a JSON document.

### DELETE /v1/domains/:domain\_id/projects/:project\_id/commitments/:id

Deletes a commitment within the given project. Requires a cloud-admin token. On success, returns 204 (No Content).
Expand Down
238 changes: 238 additions & 0 deletions internal/api/commitment.go
Expand Up @@ -83,6 +83,24 @@ var (
JOIN project_services ps ON pr.service_id = ps.id
WHERE par.id = $1
`)
getCommitmentWithMatchingTransferTokenQuery = sqlext.SimplifyWhitespace(`
SELECT * FROM project_commitments WHERE id = $1 AND transfer_token = $2
`)
findTargetAZResourceIDBySourceIDQuery = sqlext.SimplifyWhitespace(`
WITH source as (
SELECT ps.type, pr.name, par.az
FROM project_az_resources as par
JOIN project_resources pr ON par.resource_id = pr.id
JOIN project_services ps ON pr.service_id = ps.id
WHERE par.id = $1
)
SELECT par.id
FROM project_az_resources as par
JOIN project_resources pr ON par.resource_id = pr.id
JOIN project_services ps ON pr.service_id = ps.id
JOIN source s ON ps.type = s.type AND pr.name = s.name AND par.az = s.az
WHERE ps.project_id = $2
`)

forceImmediateCapacityScrapeQuery = sqlext.SimplifyWhitespace(`
UPDATE cluster_capacitors SET next_scrape_at = $1 WHERE capacitor_id = (
Expand Down Expand Up @@ -434,3 +452,223 @@ func (p *v1Provider) DeleteProjectCommitment(w http.ResponseWriter, r *http.Requ
})
w.WriteHeader(http.StatusNoContent)
}

// StartCommitmentTransfer handles POST /v1/domains/:id/projects/:id/commitments/:id/start-transfer
func (p *v1Provider) StartCommitmentTransfer(w http.ResponseWriter, r *http.Request) {
httpapi.IdentifyEndpoint(r, "/v1/domains/:id/projects/:id/commitments/:id/start-transfer")
token := p.CheckToken(r)
if !token.Require(w, "project:edit") {
return
}
dbDomain := p.FindDomainFromRequest(w, r)
if dbDomain == nil {
http.Error(w, "domain not found.", http.StatusNotFound)
return
}
dbProject := p.FindProjectFromRequest(w, r, dbDomain)
if dbProject == nil {
http.Error(w, "project not found.", http.StatusNotFound)
return
}
// TODO: eventually migrate this struct into go-api-declarations
var parseTarget struct {
Request struct {
Amount uint64 `json:"amount"`
TransferStatus limesresources.CommitmentTransferStatus `json:"transfer_status,omitempty"`
} `json:"commitment"`
}
if !RequireJSON(w, r, &parseTarget) {
http.Error(w, "json not parsable.", http.StatusBadRequest)
return
}
req := parseTarget.Request

if req.TransferStatus != limesresources.CommitmentTransferStatusUnlisted && req.TransferStatus != limesresources.CommitmentTransferStatusPublic {
http.Error(w, fmt.Sprintf("Invalid transfer_status code. Must be %s or %s.", limesresources.CommitmentTransferStatusUnlisted, limesresources.CommitmentTransferStatusPublic), http.StatusBadRequest)
return
}

if req.Amount <= 0 {
http.Error(w, "delivered amount needs to be a positive value.", http.StatusBadRequest)
return
}

//load commitment
var dbCommitment db.ProjectCommitment
err := p.DB.SelectOne(&dbCommitment, findProjectCommitmentByIDQuery, mux.Vars(r)["id"], dbProject.ID)
if errors.Is(err, sql.ErrNoRows) {
http.Error(w, "no such commitment", http.StatusNotFound)
return
} else if respondwith.ErrorText(w, err) {
return
}

// reject commitments that are not confirmed yet.
if dbCommitment.ConfirmedAt == nil {
http.Error(w, "commitment needs to be confirmed in order to transfer it.", http.StatusUnprocessableEntity)
return
}

// Mark whole commitment or a newly created, splitted one as transferrable.
tx, err := p.DB.Begin()
if respondwith.ErrorText(w, err) {
return
}
defer sqlext.RollbackUnlessCommitted(tx)
transferToken := p.generateTransferToken()

// Deny requests with a greater amount than the commitment.
if req.Amount > dbCommitment.Amount {
http.Error(w, "delivered amount exceeds the commitment amount.", http.StatusBadRequest)
return
}

if req.Amount == dbCommitment.Amount {
dbCommitment.TransferStatus = req.TransferStatus
dbCommitment.TransferToken = transferToken
_, err = tx.Update(&dbCommitment)
if respondwith.ErrorText(w, err) {
return
}
} else {
now := p.timeNow()
transferAmount := req.Amount
remainingAmount := dbCommitment.Amount - req.Amount
transferCommitment := p.buildSplitCommitment(dbCommitment, transferAmount)
transferCommitment.TransferStatus = req.TransferStatus
transferCommitment.TransferToken = transferToken
remainingCommitment := p.buildSplitCommitment(dbCommitment, remainingAmount)
err = tx.Insert(&transferCommitment)
if respondwith.ErrorText(w, err) {
return
}
err = tx.Insert(&remainingCommitment)
if respondwith.ErrorText(w, err) {
return
}
dbCommitment.SupersededAt = &now
_, err = tx.Update(&dbCommitment)
if respondwith.ErrorText(w, err) {
return
}
dbCommitment = transferCommitment
}
err = tx.Commit()
if respondwith.ErrorText(w, err) {
return
}

var loc azResourceLocation
err = p.DB.QueryRow(findProjectAZResourceLocationByIDQuery, dbCommitment.AZResourceID).
Scan(&loc.ServiceType, &loc.ResourceName, &loc.AvailabilityZone)
if errors.Is(err, sql.ErrNoRows) {
//defense in depth: this should not happen because all the relevant tables are connected by FK constraints
http.Error(w, "no route to this commitment", http.StatusNotFound)
return
} else if respondwith.ErrorText(w, err) {
return
}

c := p.convertCommitmentToDisplayForm(dbCommitment, loc)
logAndPublishEvent(p.timeNow(), r, token, http.StatusAccepted, commitmentEventTarget{
DomainID: dbDomain.UUID,
DomainName: dbDomain.Name,
ProjectID: dbProject.UUID,
ProjectName: dbProject.Name,
Commitment: c,
})
respondwith.JSON(w, http.StatusAccepted, map[string]any{"commitment": c})
}

func (p *v1Provider) buildSplitCommitment(dbCommitment db.ProjectCommitment, amount uint64) db.ProjectCommitment {
now := p.timeNow()
return db.ProjectCommitment{
AZResourceID: dbCommitment.AZResourceID,
Amount: amount,
Duration: dbCommitment.Duration,
CreatedAt: now,
CreatorUUID: dbCommitment.CreatorUUID,
CreatorName: dbCommitment.CreatorName,
ConfirmBy: dbCommitment.ConfirmBy,
ConfirmedAt: dbCommitment.ConfirmedAt,
ExpiresAt: dbCommitment.ExpiresAt,
PredecessorID: &dbCommitment.ID,
}
}

// TransferCommitment handles POST /v1/domains/{domain_id}/projects/{project_id}/transfer-commitment/{id}?token={token}
func (p *v1Provider) TransferCommitment(w http.ResponseWriter, r *http.Request) {
httpapi.IdentifyEndpoint(r, "/v1/domains/:id/projects/:id/transfer-commitment/:id")
token := p.CheckToken(r)
if !token.Require(w, "project:edit") {
http.Error(w, "insufficient access rights.", http.StatusForbidden)
return
}
transferToken := r.Header.Get("Transfer-Token")
if transferToken == "" {
http.Error(w, "no transfer token provided", http.StatusBadRequest)
return
}
commitmentID := mux.Vars(r)["id"]
if commitmentID == "" {
http.Error(w, "no transfer token provided", http.StatusBadRequest)
return
}
dbDomain := p.FindDomainFromRequest(w, r)
if dbDomain == nil {
http.Error(w, "domain not found.", http.StatusNotFound)
return
}
targetProject := p.FindProjectFromRequest(w, r, dbDomain)
if targetProject == nil {
http.Error(w, "project not found.", http.StatusNotFound)
return
}

// find commitment by transfer_token
var dbCommitment db.ProjectCommitment
err := p.DB.SelectOne(&dbCommitment, getCommitmentWithMatchingTransferTokenQuery, commitmentID, transferToken)
if errors.Is(err, sql.ErrNoRows) {
http.Error(w, "no matching commitment found", http.StatusNotFound)
return
} else if respondwith.ErrorText(w, err) {
return
}

// get target AZ_RESOURCE_ID
var targetResourceID db.ProjectAZResourceID
err = p.DB.QueryRow(findTargetAZResourceIDBySourceIDQuery, dbCommitment.AZResourceID, targetProject.ID).Scan(&targetResourceID)
if respondwith.ErrorText(w, err) {
return
}

dbCommitment.TransferStatus = ""
dbCommitment.TransferToken = ""
dbCommitment.AZResourceID = targetResourceID
_, err = p.DB.Update(&dbCommitment)
if respondwith.ErrorText(w, err) {
return
}

var loc azResourceLocation
err = p.DB.QueryRow(findProjectAZResourceLocationByIDQuery, dbCommitment.AZResourceID).
Scan(&loc.ServiceType, &loc.ResourceName, &loc.AvailabilityZone)
if errors.Is(err, sql.ErrNoRows) {
//defense in depth: this should not happen because all the relevant tables are connected by FK constraints
http.Error(w, "no route to this commitment", http.StatusNotFound)
return
} else if respondwith.ErrorText(w, err) {
return
}

c := p.convertCommitmentToDisplayForm(dbCommitment, loc)
logAndPublishEvent(p.timeNow(), r, token, http.StatusAccepted, commitmentEventTarget{
DomainID: dbDomain.UUID,
DomainName: dbDomain.Name,
ProjectID: targetProject.UUID,
ProjectName: targetProject.Name,
Commitment: c,
})

respondwith.JSON(w, http.StatusAccepted, map[string]any{"commitment": c})
}