Pivotdude

Pivotdude is a utility to pivot data from an input file in a user-desired way.

Pivotdude can be ran manually, or integrated into an automated-process/work-flow. In general it follows the UNIX philosophy much as possible.

Note: This is the converse-utility of the tool unpivotdude, which can be found here

Project Motivation

We can accomplish pivoting data in multiple ways; however, for most individuals, they typically do it in Excel. The problem with Excel is that to pivot against a combination of columns, things become clunky fast; they append together values to get unique column combinations, pivot tables become nested/indented and need reformatting, processing hangs on vast data (and may not even work when beyond ~50,000 line items), and so forth. Of course, it also goes without saying this process is difficult to automate and is not workable on large data sets.

Pivotdude is a command line approach to easily take tab-separated values from a file and output to standard output (stdout) the data transformation.

Pre-Compiled Binaries

Binaries (.exe) for Windows OS have been pre-compiled and can be found in the 'bin' folder.

With git, you can download all the latest source and binaries with git clone https://github.com/chipnetics/pivotdude

Alternatively, if you don't have git installed:

1. Download the latest release here
2. Unzip to a local directory.
3. Navigate to 'bin' directory for executables.

Compiling from Source

Utilities are written in the V programming language and will compile under Windows, Linux, and MacOS.

V is syntactically similar to Go, while equally fast as C. You can read about V here.

Each utility is its own .v file, so after installing the latest V compiler, it's as easy as executing the below. Be sure that the V compiler root directory is part of your PATH environment.

git clone https://github.com/chipnetics/pivotdude
cd src
v build pivotdude.v

Alternatively, if you don't have git installed:

1. Download the bundled source here
2. Unzip to a local directory
3. Navigate to src directory and run v build pivotdude.v

Please see the V language documentation for further help if required.

Viewing Large Files

As an aside, the author recommends the excellent tool EmEditor by Emurasoft for manually editing or viewing large text-based files for data science & analytics. Check out the tool here. EmEditor is paid software but well worth the investment to increase effeciency.

Running Command Line Arguments

For Windows users, if you want to pass optional command line arguments to an executable:

1. Navigate to the directory of the utility.
2. Hold Shift + Right Mouse Click.
3. Select "Open PowerShell Window Here".
4. Type the name of the exe along with the optional argument (i.e. ./pivotdude.exe [OPTIONS] ).

Command Line Options

Usage: pivotdude [SHORT-OPTION]... [STRING]...
or: pivotdude LONG-OPTION

Pivot input data on specific column combination.
Note that columns are 0-index based.

  -p (required)         comma-separated list of pivot indexes
  -h (required)         column to generate header from
  -v (required)         column of values to extract
  -f (required)         the input file to pivot
  -nh                   indicate input file has no header
     --help             display this help and exit
     --about            author and licence information
     --version          output version information and exit'

---

Examples

---

./pivotdude.exe -f ..\examples\sample1.txt -p 1,2 -v 4 -h 0 > ..\examples\sample1_out.txt

sample1.txt :::

sample1_out.txt :::