Skip to content
This repository has been archived by the owner on Oct 12, 2023. It is now read-only.

HTTPError: 403 Client Error In merge task foreach %dopar% #370

Open
Pullarg opened this issue Jul 19, 2020 · 0 comments
Open

HTTPError: 403 Client Error In merge task foreach %dopar% #370

Pullarg opened this issue Jul 19, 2020 · 0 comments

Comments

@Pullarg
Copy link

Pullarg commented Jul 19, 2020

I have been using Azure Batch for a while and have come across a new error in the merge task even on the sample code. this appears to happen with any size of data while running the foreach %dopar%. has anyone else noticed this?

HTTPError: 403 Client Error: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. for url: https://_batchaccount._blob.core.windows.net/job12345?sv=2016-05-31&sr=c&st=2020-07-18T21%3A09%3A30Z&se=2020-07-21T21%3A09%3A30Z&sp=rwcl&sig=signature%3D&include=metadata&restype=container&comp=list&maxresults=1000

Reproduce Example
generateCredentialsConfig("credentials.json")

setCredentials("credentials.json")
generateClusterConfig("cluster.json")

cluster <- makeCluster("cluster.json")

Register your parallel backend
registerDoAzureParallel(cluster)

setVerbose(FALSE)
setAutoDeleteJob(FALSE)

mean_change = 1.001
volatility = 0.01
opening_price = 100

getClosingPrice <- function() {
days <- 1825 # ~ 5 years
movement <- rnorm(days, mean=mean_change, sd=volatility)
path <- cumprod(c(opening_price, movement))
closingPrice <- path[days]
return(closingPrice)
}

Optimize runtime. Chunking allows running multiple iterations on a single R instance.
opt <- list(chunkSize = 10)
start_p <- Sys.time()
closingPrices_p <- foreach(i = 1:1000, .combine='c', .options.azure = opt) %dopar% {
replicate(100000, getClosingPrice())
}
end_p <- Sys.time()

hist(closingPrices_s)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant