Saliency-Based Artistic Abstraction With Deep Learning and Regression Trees

Abstraction in art often reflects human perception—areas of an artwork that hold the observer’s gaze longest will generally be more detailed, while peripheral areas are abstracted, just as they are mentally abstracted by humans’ physiological visual process. The authors’ artistic abstraction tool, Salience Stylize, uses Deep Learning to predict the areas in an image that the observer’s gaze will be drawn to, which informs the system about which areas to keep the most detail in and which to abstract most. The planar abstraction is done by a Random Forest Regressor, splitting the image into large planes and adding more detailed planes as it progresses, just as an artist starts with tonally limited masses and iterates to add fine details, then completed with our stroke engine. The authors evaluated the aesthetic appeal and effectiveness of the detail placement in the artwork produced by Salience Stylize through two user studies with 30 subjects.

Paper available here

Recommended citation: Shakeri, H., Nixon, M., & DiPaola, S. (2017). Saliency-Based Artistic Abstraction With Deep Learning and Regression Trees. Journal of Imaging Science and Technology, 61(5).