Attachment Proposal
This is a proposal for implementing attachments in beet as requested here. Implementation is done in the attachment branch. If you would like to contribute open a PR or add your use case to the wiki.
The guiding principle of this proposal is to specify simple yet extendable core funcionality. This means that beets itself will not know whether an attachment is cover art, a rip log, or a text file containing lyrics. Instead it will the responsibility of plugins to provide this the handling of the different types.
An attachment associate resources (in particular files) to items or albums (called entities for short). Resources are identified by URLs. Each entity may have multiple resources attached and each resource may be attached to multiple entities using multiple attachements.
An attachment is a structure that contains the following fields.
-
url A valid url string speciying the location of the resource. The default schema is
file. Relative paths in thefileschema are resolved with respect to the beets directory. The field also serves as a unique identifier of the attachment - type A string determining the behaviour of the attachment and a partial schema for the metadata in meta.
- ref The id of the entity this attachment is attached to as an integer. Together with ref_type it unambiguously defines the entity.
- ref_type One of "album" and "item". Specifies the type of the entity the attachment is attached to.
- metadata A string-string map. The semantics are determined by plugins.
A valid attachment must satisfy the following conditions.
- url is a valid url.
-
type matches the regular expression
^[a-zA-Z][a-zA-Z0-9-_]*. - ref together with ref_type point to an existing entity.
The beets core will only provide a general interface, the Core API, for dealing with attachments. This interface will support creating, querying, moving, and deleting attachments in a generic way. In particular the implementation will be ignorant of the semantics of types and metadata.
In theory, beets’ core should not be responsible for invoking the methods provided by the Core API, but instead delegate the responsibility to plugins. In practice, however, this will lead to a lot of boilerplate code and repetitive user interaction. Therefore, some additional functionality will be integrated into beets.
The Core API consists of
- The Attachment class that represents an attachment in the database.
- The AttachmentFactory class which simplifies the creation of Attachment.
- The abstract AttachmentCommand class intended to be implemented by plugins that provide command line interfaces for managing attachments.
- Mixins for Items, Albums, and the Library to help finding attachments.
An Attachment instance allows us to access properties and metadata and move attachments to a different location. By default attachments are moved to a location dependent on their type. The location is specified by a template that allows use of the attachments and its entities properties.
Attachments should be created through the AttachmentFactory’s create method. The factory also has a discover method that delegates populating type and metadata to plugins. More precisely, discover(path) passes the path argument to each attachment plugin. If an attachment can handle the resource given by path it returns a string denoting the type it want the attachment to have. For each such type an attachment is created. In a second step each plugin in is asked to provide metadata for a given type and path. The metadata is returned as a dictionary by the plugin. This method will be used by the generic commands to create attachments in bulk.
Since one line of code says more than a thousand words just have a look at the code
A user may want to manage attachments of different types, provided by different plugins, in bulk. Using the plugins separately would involve repetitive user interaction. To make it easier to handle this in bulk, beets will provide some generic functionality.
The attach command will use all plugins to discover the type of the resources to be attached and add the to the database using the factory.discover method. It will also allow the user to move attachments to their correct place via attachment.move and delete them.
The implementation is guided by this command line interface.
Instead of attaching files manually through the command line after importing we will hook into the import process to automatically attach files from imported directories.
The list command will be extended to list attachments and provide a type based queries.
Attachment plugins are responsible for two things.
- Providing a command to mutate a specific type of attachment.
- Exposing discoverers and collectors for beets’s generic functionality to use.
An attachment plugin is, just like a normal plugin, a subclass of BeetPlugin, but implements the following additional methods.
Return a type string if the plugin knows how to handle the path.
The path argument points to a file that should be added as an attachment. The plugin needs to check if it can handle that file and then return a corresponding type string for that attachment. If the plugin does not know how to handle that file it must return None
Return a dictionary of metadata for a resource of the given type.
If the plugin knows how to handle attachments of type it may try to extract metadata from the path such as the resolution for images. Otherwise it must return None.
Return a list of subclasses of the AttachmentCommand class.
The command will be available through the command line. Note that the method must return subclasses not instances. For implementing a command, see the API documentation.
There exists a mock implementation of a coverart plugin in the attachment branch.
Instead of paths, attachments are located by URLs. Since we assume file to be the default scheme it is fully compatible with simple paths but offers some advantages.
- URLs allow us to decouple the storage mechanism from attachments and allow us to retrieve large attachments from remtote sources on demand.
- We can store data directly in the database using the
dataURI scheme. Although if this is used on scale, we may have to change the implementation details to handle large URLs. - Plugins can scrape data off the web.
Currently, plugins can only register types for attachments located on the file system. This is because we want a simple interface for plugins and not require the to check whether a path is an url. In the future we have to extend this to allow plugins to hanlde URLs.
There might be need of a mechanism to resolve URLs. For example a plugin could be passed a MusicBrainz URL for a release and resolve it to an URL pointing to the cover art stored by MusicBrainz.
A shortcoming of the move definition is that the default templates only depend on the attachment’s type. While this simplifies the implementation it doesn’t allow to use different templates depending on metadata. This can be (at least partially) circumvented by referencing the metadata as variables in the template.
The regular expression for the type property was chosen to simplify command line usage.
The decision not to pass the attachment to disocverers and collectors was concious. Attachements must not depend on the entities they are attached to.
Attachments are only created if a plugin registered a type. We might want to include a way to deal with generic attachments if no type was found.
The Core API might be extended to store serialied resources in media file tags. This would be particularily usefull for cover art.
Since we introduce an additional command API for plugins I thought it might be a good idea to use the argparse library as a base for this since it is much more flexible than optparse. This might also serve as a prototype for [migrating the old command interace][plugin refactor]. Integrating this into the current implementation of how beets dispatches commands to plugins might prove as a challenge, though.
We have to specify a sensible list cli for attachments.
Creating attachments for imported albums is straight-forward. Just look for all non-music files in the directory the album was imported from. We still have to come up with an idea for albums imported with the group albums option and singleton tracks.
[plugin refactor]: https://github.com/sampsyo/beets/wiki/Refactoring#wiki-plugin-api-overhaul).