Use moto to run functional tests

[mello7tre]

Issue Type

• Feature Pull Request

Summary

The idea is to use moto to run functional tests.

This first commit provide the feature to use a custom Endpoint Url using the environment variable
AWS_ENDPOINT_URL.

Current moto version is missing some feature/methods needed by AutoSpotting, so i have added them to my working fork:
https://github.com/mello7tre/moto/tree/mello-feats-and-fixes-01

i will open PRs to merge them in moto master branch.

Until merged, you can install my current fork using the command:
pip3 install git+https://github.com/mello7tre/moto.git@mello-feats-and-fixes-01#egg=moto[ec2,autoscaling,server]
(warning overwrite any current moto installation)

You can then run moto server using command:
moto_server

Once running you can create a base infrastructure for AutoSpotting using any language that you like, configuring it to use the moto local endpoint.
As example, in python you can create ec2 and autoscaling services using:

conn_ec2 = boto3.client('ec2', 'us-east-1', endpoint_url='http://127.0.0.1:5000')   
conn_asg = boto3.client('autoscaling', 'us-east-1', endpoint_url='http://127.0.0.1:5000')

Once created the infrastructure you can run a local AutoSpotting executable like:
AWS_ENDPOINT_URL="http://127.0.0.1:5000/" /opt/AutoSpotting-build/build/dist/AutoSpotting -allowed_instance_types current -regions us-east-1 -tag_filters "spot-enabled=true" -event_file /opt/AutoSpotting-build/event_cron.json

where event_cron.json is like:

{  
   "id":"7bf73129-1428-4cd3-a780-95db273d1602",
   "detail-type":"Scheduled Event",
   "source":"aws.ec2",
   "account":"432915485918",
   "time":"2019-11-11T21:29:54Z",
   "region":"us-east-1",
   "resources":[
      "arn:aws:ec2:us-east-1:123456789012:instance/i-abcd1111"
   ],
   "detail": {}
}

In following posts, when we will choose how to implement them, i will add scripts to create the infrastructure and run the tests.

Moto support sqs and lambda too, but i doubt that it will be able to fully support a complete AutoSpotting workflow (CW -> Lambda -> SQS -> Lambda).
So the idea is to create ad hoc events file and tests the specific event, simulating events originating from both SQSQueue and CloudWatch Events.
Now that spot request, launch and following ondemand replacement is done all in one step, is easy to accomplish this.

Code contribution checklist

1. [ X] I hereby allow the Copyright holder the rights to distribute this piece of
   code under any software license.
2. The contribution fixes a single existing github issue, and it is linked
   to it.
3. The code is as simple as possible, readable and follows the idiomatic Go
   guidelines.
4. All new functionality is covered by automated test cases so the overall
   test coverage doesn't decrease.
5. No issues are reported when running make full-test.
6. Functionality not applicable to all users should be configurable.
7. Configurations should be exposed through Lambda function environment
   variables which are also passed as parameters to the
   CloudFormation
   and
   Terraform
   stacks defined as infrastructure code.
8. Global configurations set from the infrastructure stack level should also
   support per-group overrides using tags.
9. Tags names and expected values should be similar to the other existing
   configurations.
10. Both global and tag-based configuration mechanisms should be tested and
    proven to work using log output from various test runs.
11. The logs should be kept as clean as possible (use log levels as
    appropriate) and formatted consistently to the existing log output.
12. The documentation is updated to cover the new behavior, as well as the
    new configuration options for both stack parameters and tag overrides.
13. A code reviewer reproduced the problem and can confirm the code
    contribution actually resolves it.

[codeclimate]

Code Climate has analyzed commit 8c7710c and detected 0 issues on this pull request.

View more on Code Climate.