Skip to content

Alpha-3 Release

Pre-release
Pre-release
Compare
Choose a tag to compare
@soumith soumith released this 16 Sep 11:01
v0.1.3
b5f7720

What's new?

Usability

  • conda binaries for all Linux (as old as RHEL 6 and Ubuntu 12.04) (we are working on OSX and pip binaries).
    • Now installing pytorch is as simple as:
      • conda install pytorch -c https://conda.anaconda.org/t/6N-MsQ4WZ7jo/soumith
      • it links against MKL, ships CUDA and MAGMA runtime with it, and #justworks
  • Human-ready error messages
  • Started working on documentation and an API Reference
  • Continuous integration with GPU support. Never have a broken master again

New Features and modules

  • The (new) neural network module now has 75% of the modules implemented (71 out of 93), and we are powering through the rest
    • most of the modules in old-nn have been removed because we do not need Containers and many modules such as CAddTable are covered by Autograd
  • autograd now supports all torch functions present in twitter-autograd and a lot more....
  • Added Trainer and Dataset abstractions (like in TorchNet)

Plans for Alpha-4

  • cudnn integration (and CUDA allocator).
    • We have this implemented but are iterating over design #36
  • Multi-GPU support in nn
  • examples, examples, examples
    • we will work on having examples across all domains (vision, NLP, RL, etc.)

Usability

Conda binaries for Linux

PyTorch will be shipped on Linux and OSX (and likely Windows) from the day-1, and we want it to be as simple and intuitive install process.
We have versioned binaries, that do not require the user to install anything (except an NVIDIA Driver if you intend to use the GPU. Not even CUDA is a dependency).

For now, to get started on Linux:

conda install pytorch -c https://conda.anaconda.org/t/6N-MsQ4WZ7jo/soumith

We have built OSX binaries, but have some small bugs on OSX, and we'll fix the issues there over the week.
We are working on “pip install” for non Anaconda python installs.

Human-ready error messages

We've gone through how we report type errors and dispatch errors and make it easy for the user to understand what they did wrong. See this small example:

In [1]: import torch
In [2]: x = torch.FloatTensor(10)
In [3]: x.addmm(torch.ones(1), 1, 'str')
ValueError                                Traceback (most recent call last)
<ipython-input-3-90eb50ea2e35> in <module>()
----> 1 x.addmm(torch.ones(1), 1, 'str')

ValueError: addmm recieved an invalid combination of argument types - got (torch.DoubleTensor, int, str), but expected one of:
 * (torch.FloatTensor mat1, torch.FloatTensor mat2)
 * (float beta, torch.FloatTensor mat1, torch.FloatTensor mat2)
 * (float beta, float alpha, torch.FloatTensor mat1, torch.FloatTensor mat2)

Continuous Builds with GPU support

New Features and modules

Neural Network Modules

  • Added fully functional and fully unit-tested nn modules and criterions for pretty much everything one would need for their current workflows.
  • We have about 25% of the modules missing (mostly exotic and lightly used ones) but will get to those in the coming few days.
  • nn modules have been renamed to be simplified in their naming. For example:
  • Full unit-test coverage for all implemented functions

Autograd

  • We've added autograd support for almost all the torch functions (and operators like +, - etc.)
    • We have all the functions implemented that are presented in twitter-autograd, and we have many more.
    • At this point we have about 75 to 80% of them covered (ball park).
    • Full unit-test coverage for all implemented functions

Trainer & Dataset classes

Trainer

We've added a TorchNet style Trainer class that provides a convenient abstraction

trainer = Trainer(model, criterion, optimizer, dataset)
trainer.register_plugin(ProgressMonitor())
trainer.register_plugin(LossMonitor())
trainer.register_plugin(AccuracyMonitor())
trainer.register_plugin(Logger(['progress', 'accuracy', 'loss'], interval=(5, 'iterations')))
trainer.run(epochs=5)

################################################################################
# progress: 180/60000 (0.30%)     accuracy: 0.00% (3.24%)         loss: 2.3051 (2.2116)
# progress: 280/60000 (0.47%)     accuracy: 5.00% (4.84%)         loss: 2.3045 (2.2891)
# progress: 380/60000 (0.63%)     accuracy: 25.00% (13.04%)       loss: 2.2974 (2.2992)

Dataset

The data loading is implemented using three abstractions:

  • DataSource - a simple object that defines indexing and checking length. Indexing returns a tuple of (sample, label)
  • Sampler - an object that defines the data ordering. it has to be iterable, and it’s iterator should return a string of indices in [0; len(data_source)-1] interval. The end of the iterator indicates completing the epoch.
  • Dataset - an object which wraps a DataSource and a Sampler. Defines all the data loading logic (e.g. all the multiprocessing code).

The Datsets will accept a list of transforms (like image augmentation) that are given to it, which will run on the data before given out.

Assets 2