Synthtool will not run if generatedFiles is in synth.metadata

[sofisl]

1. Run synthtool on this library.
2. Notice that synth.metadata should only have sources and destinations
3. Re-add generatedFiles property in synth.metadata
4. Run synthtool. You should see this error:

Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/sofialeon/gcp/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/Users/sofialeon/gcp/synthtool/env/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/sofialeon/gcp/synthtool/env/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/sofialeon/gcp/synthtool/env/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/sofialeon/gcp/synthtool/env/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/sofialeon/gcp/synthtool/synthtool/__main__.py", line 93, in main
    with synthtool.metadata.MetadataTrackerAndWriter(metadata):
  File "/Users/sofialeon/gcp/synthtool/synthtool/metadata.py", line 231, in __enter__
    self.old_metadata = _read_or_empty(self.metadata_file_path)
  File "/Users/sofialeon/gcp/synthtool/synthtool/metadata.py", line 97, in _read_or_empty
    return google.protobuf.json_format.Parse(text, metadata_pb2.Metadata())
  File "/Users/sofialeon/gcp/synthtool/env/lib/python3.7/site-packages/google/protobuf/json_format.py", line 431, in Parse
    raise ParseError('Failed to load JSON: {0}.'.format(str(e)))
google.protobuf.json_format.ParseError: Failed to load JSON: Expecting value: line 45 column 3 (char 1036).

Now, in order to get it back to working condition, you need to:

5. Delete destinations
6. Re-run synthtool (you should see the same file you saw in step 2).

I would like to have generatedFiles not disappear on synth.metdata, as I believe this is necessary if we want to eventually delete files from this repo from synthtool.

[SurferJeffAtGoogle]

Please reply with:

1. The command line you used to invoke synthtool.
2. The output of pip list.

Also, as I mentioned in a meeting, I have little insight into how, why, and when people run synthtool manually. Could you share why you're invoking synthtool manually?

[sofisl]

Hi Jeff, I think I figured this one out - I wasn't setting the track obsolete files variable to true locally!

I was looking to run synthtool since autosynth was repeatedly failing. I don't know exactly what fixed it, but I followed the steps above to get synthtool to run locally, and then set the environment variable to track obsolete files. That seemed to fix it!

[tswast]

I wasn't setting the track obsolete files variable to true locally

@sofisl What is the variable that needs to be set? I notice that synthtool is removing generatedFiles when I run locally to test my synth.py changes in googleapis/python-bigquery-storage#79