From Pixel to Paragraph: A Deep Artwork Analysis Paragraph Generator
Art influences society by shaping our perspective and influencing our sense of self. By portraying subject matter with deliberate composition, color scheme, and other stylistic choices, the artist forms emotional connections between artwork and viewer and communicates their vision with the world. The goal of my project is to develop an artificial neural network system that interprets input artwork and generates a passage that describes the objects and other features (ex. color palette) present in the artwork as well as the ideas and emotions the artwork conveys.
I experimented with and modified SeqGAN and LeakGAN frameworks (both of which are originally unconditional GANs) to condition on artwork image features to generate the final art analysis paragraph.
The implementation of this project is based on code from the following projects:
- SeqGAN-Tensorflow
- Show_and_Tell
- img2poem
- Conditional-GAN
- Texygen
- tensorflow_compact_bilinear_pooling
- tensorflow-seq2seq-tutorials
I wrote scripts for scraping artwork images and their corresponding analysis paragraphs from The Art Story and Smithsonian American Art Museum.
More details about my modifications to existing code can be found as comments in the files.
"sexual of took boy singing communist mixture to above and center emphasizing repetition stand melting."
"1908 to one most movements geometric is when, major one humanize(avant-garde) behind in created flowers theme distribution is playful and the."
"circles depict shore\\ a home white a through features warhol world the, a looking, state to,."