Containerize ts-bridge, remove GAE dependencies, and make it deployable in K8S

[Dnefedkin]

ts-bridge is currently implemented as GAE application, which is fine for most of the potential users, but we've already faced with challenges trying to deploy ts-bridge in environments that are included into Google VPC Service Controls service perimeters. An attempt to deploy ts-bridge with end up with an error:
{"errors":[{"code":"DENIED","message":"Request is prohibited by organization's policy. vpcServiceControlsUniqueIdentifier: "}]}.

GAE is currently not supported by VPC service controls.

The proposal is to containerize ts-bridge and remove any GAE-specific dependencies (ideally - Cloud Datastore dependencies as well). As a result it will be possible to deploy ts-bridge inside K8S cluster (e.g. in GKE) or as GAE Flex application or in any other environment that support running containers.

[Dnefedkin]

@knyar @yanske1 - appreciate your thoughts on this.

[knyar]

The largest change that will be necessary to make ts-bridge run without App Engine is probably migrating metric records from App Engine Datastore API to Cloud Datastore (trivial) or another database (a bit harder). I would recommend Cloud Datastore, since the API is very similar. I suspect at some point this might become necessary even on App Engine, since newer App Engine Go runtimes no longer support proprietary App Engine APIs.

The second significant change is to make something trigger the regular sync (which is currently done by App Engine cron):

• if you are moving to a container, you can just use a periodic timer within the running process.
• if you are moving to GCF, you could use Cloud Scheduler API.

As part of this you might also want to implement some basic locking to prevent multiple sync operations from running concurrently (which can happen if you start multiple containers with the same configuration), since they will fight while trying to write data to Stackdriver. Since App Engine cron does not execute multiple concurrent operations, this has not been necessary on App Engine.

[Dnefedkin]

if you are moving to GCF, you could use Cloud Scheduler API.

Cloud Scheduler does not support VPC Service Controls as well. In case of K8S/GKE (which is a preferred product for a customer that is blocked at the moment) we could implement ts-bridge the following way:

• K8S deployment of size=1 that has single container pod that exposes / , /sync and /cleanup endpoints
• K8S ClusterIP service that exposes the deployment inside the cluster. To access index page(/) one can use kubectl port-forward , this allows to not expose index page to the outside world and do not have any external load balancers
• K8S CronJob that is scheduled every minute, it can be based on busybox or similar lightweight container, it just calls the /sync endpoint of ClusterIP service . As CronJobs have some limitations we should prevent multiple sync operations to run simultaneously.
• K8S CronJob that is scheduled every 12 hours and calls /cleanup enddpoint.

[Dnefedkin]

#37 that @Temikus has pushed recently is the first step in addressing this issue.