Allowing filtering/formatting of Stackdriver logs

[joshburkart]

Hello,

I'm using the instructions here to send Python logging module logs to Stackdriver. Thanks for making this so easy! However, doing this naively seems to make all logs from potentially many jobs/VMs/etc. get merged together in the Stackdriver UI. When I filter to e.g. a specific VM, none of the logs show up (screenshot) -- I have to set the filtering to "Global" to see them (screenshot). Am I doing something wrong?

Also, what about logging things like files, line numbers, etc.? Is this possible? Does Stackdriver respect formatting I apply using the Python logging API? What about automatic links to files in connected code repos etc. -- any info available on that?

Thanks!

[tseaver]

@liyanhui1228 Can you point @joshburkart in the right direction?

[joshburkart]

I figured this out (I think) -- I now make calls to the metadata endpoint to determine info about the current VM, then pass these as labels when making logging handlers. Rough code snippet in case it's useful to someone:

import logging as pylogging

from google.cloud import logging as cloudlogging

...

handler = cloudlogging.handlers.CloudLoggingHandler(
    <stackdriver_client>,
    resource=cloudlogging.resource.Resource(
        # See e.g. https://cloud.google.com/monitoring/api/resources
        # for more info on codified Stackdriver "Resources".
        type='gce_instance',
        labels=<gce_instance labels retrieved from metadata endpoint>,
    ),
    labels=<any extra labels>,
)

logger = pylogging.getLogger('whatever')
logger.addHandler(handler)

(I think this should be added to the documentation or something, though...)

[tseaver]

@liyanhui1228 I'm not sure where such an example should go: who manages the snippets / examples used in the cloud.google.com docs?

[tseaver]

@dazuma Can you help direct this to the right target for updating the API docs?

[tseaver]

@JustinBeckwith, @theacodes can either of you point me to the person / team responsible for managing the cloud.google.com/logging docs samples?

[daniel-sanche]

As of #200, the resource should be auto-detected for each GCP environment. It sounds like that was the ask for this issue, so I'll close this. Feel free to re-open if there are any remaining concerns