Google Location Utility

Utility for Location History data from Google Takeout.

Overview

GoogleLocationUtility, GLU is a command-line interface (CLI) tool for processing and utilizing location history data from Google Takeout built with Python. See below for requirements and installation instructions. A detailed usage guide is provided in the documentation.

Requirements

Python version 3.8 or newer is required. See the beginner's guide for help installing Python on your system.

Installing GLU within a virtual environment (venv) will install all required packages. The requirements and versions specified during installation are listed below. A list of all installed packages (output to pip freeze > requirements.txt) is provided in the docs, installed packages.

• Click >= 8.0, used to build CLI
• exif >= 1.3, reads/adds GPS tags from/to image files
• plotly >= 5.6, generates HTML interactive maps
• pandas >= 1.4, main tool for data handling
• numpy >= 1.22, used with pandas for calculations
• matplotlib >= 3.5, graphing engine for location reports
• seaborn >= 0.11.2, used with matplotlib
• fastparquet >= 0.8, efficient storage for processed location data
• Jinja2 >= 3.0, required for pandas HTML exports, report building

Installation, Quickstart

The following steps provide installation instructions within a virtual environment. It may be possible to install and use GLU in your base environment, but it is not recommended.

1. In a directory of your creation, <your-directory>. Within this folder, create and activate a virtual environment in venv.

# Unix / macOS
	cd <your-directory>
	python3 -m venv venv
	. venv/bin/activate	
# Windows
	cd <your-directory>
	py -3 -m venv venv
	venv\Scripts\activate

The following steps assume you are in <your-directory> and the virtual environment is activated. These are accomplished in the first, cd ..., and third, ...activate, lines.

---

Option 2A - PyPi installation

2A. Install package from PyPi.

• Download Configuration.ini as <your-directory>/Configuration.ini, be sure not to change the file extension.

# Unix / macOS / Windows
	pip install GoogleLocationUtility

Obtain Configuration.ini through command line. Use curl or wget. Note capitilization of "o/O" for curl/wget

# Unix / macOS / Windows
	curl https://raw.githubusercontent.com/NBPub/GoogleLocationUtility/main/Configuration.ini -o ./Configuration.ini
	# OR
	wget https://raw.githubusercontent.com/NBPub/GoogleLocationUtility/main/Configuration.ini -O ./Configuration.ini

Option 2B - Download and install from Github

2B. Download and extract GLU into your directory <your-directory>. Install package.

• Click the green Code button at the top of the page for various download options.
• Note that the "docs" folder can be deleted

# Unix / macOS / Windows
	pip install --editable .

Optional, delete docs folder

# Unix / macOS / Windows
	rm -r docs
	rm README.md	
# Windows
	rmdir /s docs
	del README.md

---

3. Add Location History Takeout export.

• Enter home to ensure GLU is functioning, and LocationData, Outputs folders exist within <your-directory>
• Request, and then extract, your Location History data from Google Takeout.
• Copy location data, Records.json, and optionally Settings.json, into <your-directory>/LocationData.
  • Other exported files are not used by GLU.

4. See Getting Started for detailed usage instructions.

• Filestatus and available functions are listed with home, all available options are listed with home --help.
• Modify <your-directory>/Configuration.ini to specify various configuration settings.
  • Open configuration file for editing with home --config
• Documentation can be accessed from GLU with home --docs or home --docs_read.

Features

For more information about the functions available, see their respective files in the documentation. Configuration settings for the functions are detailed in Configuration.ini Usage. GLU functions stem from the command, home, which provides an overview of files and functions available. home --help will provide all the function options.

"home" printout, with no location data to process and no processed data to use

Location History Export

GLU works with exported Location History Records from Google Takeout. Settings are optional, and may provide additional information about devices, which are reported as 10-digit integers in Records. After extraction, the exported Location History files from Google should be in JSON format.

Semantic Location History and Tombstones are not used by GLU.

"home" printout with location history available, as well as processed data and reports.

---

The details links will open the corresponding function page in the docs.

Processing

• home --loc_parse details

Before location data can be used, Records.json must be processed. GLU detects Records.json within the LocationData folder in the project root: <your-directory>/LocationData.

Python's JSON decoder is used to convert the data into a dictionary (this process may be memory intensive), which is then read into a pandas DataFrame. Timestamps, GPS coordinates, device IDs, and sources are kept from the exported data. Additionally, the timedelta in between each point is calculated.

The resulting DataFrame is saved as a Parquet file, which allows for data type persistence and fast loading/saving, in the LocationData directory: <your-directory>/LocationData/parsed_<date>.parquet

Example processing operation, with ~500MB Records.json file

Reports

• home --location_report details

Reports can be generated from any processed data. Accuracy and time-delta statistics are presented, along with a breakdown of accuracy against source(s) and device(s). Maps detailing locations of each device can optionally be generated. Reports are saved as HTML files, static graphs as PNG images, and maps as HTML files.

Each report, containing these files, is saved as a folder in the Outputs directory: <your-directory>/Outputs/<Report Folder>.

Preview

See function page for sample tables and graphs.
CLI example:

Filtering

• home --loc_filter details

Processed location data from Records.json is considered "bulk" data. These can be further "filtered" by accuracy, source, and device. A report for bulk data may be useful in determining filter parameters. Reports can be generated from filtered location data, too. Notes of filter operations are stored in a CSV table: <your-directory>/LocationData/filter_notes.csv.

Results of filter operations are saved as a new Parquet file in the LocationData directory: <your-directory>/LocationData/filtered.parquet

Preview

Maps

• home --loc_map details

Location data within an input time range can be used to generate an interactive HTML map with panning and zooming capabilities. Map markers can be styled by "time" or "frequency". Street tiles from OpenStreetMap are used in the Plotly graphs, so zooming in provides more detail.

Maps are saved in Outputs/Maps: <your-directory>/Outputs/Maps/MAP-<style>_<date>.html

Preview

geoTag

• home --geoTag details

You made it! This was my main purpose in building this program. See a previous version, and inspiration.

geoTag can add or replace GPS coordinates to an image's EXIF metadata, provided a suitable match is found in the processed location data. A detailed HTML report of an operation can be saved, including maps of tagged photos. If a match is made, image copies are saved in the Outputs directory: <your-directory>/Outputs/<geoTag results>

Preview

GPS metadata can also be removed from images in a folder with home --geoStrip details. Copies are also saved in the Outputs directory.

Preview

Limitations, Issues

Altitude

Altidude is dropped from the exported Location History in this version, as my data was mostly empty. Altidude (m above sea level) is an available tag for an image's EXIF metadata, provided it's > 0.

PhotoTag - Improper EXIF tags

Various failure mechanisms may prevent GPS metadata from being added to an image. These may be endemic to the file itself, but could also be a limitation of my methodology or the package used to read/write EXIF tags. In my previous iteration of a photo-tagger, I remember issues with certain panoramas from a mobile phone, images converted from a raw image file, and older images I didn't know much about.

Verbose errors are captured in a GeoTag detailed report.

Please report any errors you may encounter.

Future

Completed fixes and improvements after initial upload/commit, for alpha release:

• Improved settings load
  • allow graceful exit when inputting Map settings
  • fallback values
  • checks for file and sections
  • Relevant error messages and diagnosis
• publish on PyPi
  • with release, delete old version and upload alpha.
• general fixes
• documentation

Ideas for improvement and future releases:

• Critical / code improvement
  • Implement tests
  • Utilize Jinja2 HTML templates to clean up code for reports (location report, geotag report)
• Python Packaging:
  • Use setuptools_scm properly, harmonize packaging / PyPi / git
  • Include Configuration.ini in PyPi distribution, package data / data_files
• Docs:
  • Spoof my location data to provide HTML examples of interactive maps
• Features:
  • Manual geoTag function, when location match is insufficient. Input coordinates or select on map to tag photo(s)
  • Advanced location reporting
    • Monthly accuracy ECDFs
    • Various analyses presented on calplot, calendar heatmaps

---

• Option for TimeZoneDB integration to check location vs. input timezone
• Include additional location history parameters (altitude, velocity, heading, etc . . .)
• Determine if Pillow vs exif for tag writing is beneficial.