Creating Stylised Maps with Neural Style Transfer
Geographic maps have always had — and still do to this day — an undeniable aesthetic quality capable of providing them with meaning beyond utilitarian and navigational purposes. The tools and data provided by the advancement of technology helped popularise and facilitate this alternate, aesthetics-focused usage of the map and thereby introduced a new opportunity for the visual exploration and customisation of maps. Said opportunity is taken a step further with the introduction of generative and computational design techniques, which facilitate the production of more customisable, diverse, and experimental outputs.
In this context, we present a computational system (available online in the near future) focused on generating stylised geographic maps of any location in the world with the help of a neural style transfer technique. The system uses open-access geographic data from multiple open data projects to render input maps and an existing implementation of arbitrary neural style transfer to style them based on a separate input image. The driving force behind the system is creating map-like artefacts with visual interest that can fulfil aesthetic and decorative needs, thus, as it pertains to this project, spatial accuracy and traditional communication-focused mapmaking conventions are secondary to visual quality and appeal.
Approach
Neural style transfer is a technique that uses convolutional neural networks to extract and combine the semantic content from one image with the style of another, effectively, as the name implies, transferring the style from one image to the other. Because this technique is at the core of how outputs are generated, the system can be understood by its two primary and indispensable inputs: the input map, which provides the semantic content and the style input, which provides the style. The input map is rendered inside the system, requiring only a selected geographic area composed of two sets of latitude and longitude coordinates — the top-left and bottom-right corners of a rectangular region. Conversely, the style input needs to be loaded onto the system but can be any image file. Granted that this image controls the look of the output, and thus, whilst the system accepts any image, not all of them will produce good results. Finally, the output maps are generated in a lossless raster image format (PNG) at pre-adjustable pixel dimensions.
Rendering input maps
Input maps are rendered from dynamically fetched data hosted by Nextzen, a free service that provides vectorial tiles (i.e. square sections of a world map that can be individually retrieved to compose larger areas) based on geographic data from OpenStreetMap and other open data projects. Once all the data necessary to render the selected area has been fetched, the system parses and translates it to visual elements (points, lines, and polygons) to form a single vector-based SVG map. The use of vectorial input maps greatly influences the amount of customisation supported by the system because they allow it to: (i) filter geographic features, e.g. only show highways; (ii) colour and set the stroke sizes, which influences how the style transfer model interprets the semantic content; and (iii) losslessly convert the input map into any pixel resolution.
Map stylisation
As it currently stands, the system uses an existing implementation of arbitrary neural style transfer by Reiichiro Nakano as its neural style transfer model. The stylisation process, contrary to what might be expected, is done incrementally following a tile-based approach. Instead of passing the entire input map through the style transfer model at once, the system splits the input map into a controllable number of equally sized sections and passes each one through the style transfer model individually. The purpose of this less direct approach to stylisation is to allow the system to generate larger maps because, with our computer’s specifications, the style transfer model cannot generate outputs larger than, roughly, 1500×1500 pixels. Splitting the input map into tiles and styling them incrementally effectively means the computational power demanded by the system is divided into multiple lighter chunks, making it possible to generate output maps with dimensions such as 4000×4000 pixels (generated with an 8×8 grid of 500×500 pixel tiles), for example.
The aforementioned tile-based approach is possible because the neural style transfer outputs are not stochastic — meaning that if the inputs are kept the same, input map X plus style Y will always produce the same exact output map. However, this method did introduce some visual artefacts to the borders between tiles (figure 1) because the style transfer model inherently produces some visual artefacts at the edges of the output images. To fix this issue, the system (i) generates each tile with a surrounding margin that is cropped out before the tiles are assembled back into the output map, and (ii) fades between neighbouring tiles instead of just placing them side by side. This solution ensures that (i) each tile has a temporary buffer zone to catch the aforementioned edge artefacts and (ii) any visual inconsistency between neighbouring tiles is concealed in the smooth transition between them.
Map of Coimbra, Portugal, using the tile-based approach with (right) and without (left) margins and fade between neighboring tiles. On the left, without margins and fade, the tile composition is visible because of the lines at the edges of each tile.
Applications
In our admittedly subjective opinion, the output maps generated by the system are visually engaging and do possess aesthetic properties similar to (albeit more abstract) paintings and posters commonly used for interior decoration. In this respect, we concluded that the output maps can be used to decorate homes (figure 2) and business alike, as well as a myriad of other contexts in which they can provide value by being a stylistic connection to a geographic place. To that effect, another possible context for the real-world application of said maps is in design objects that require a more or less subtle connection to a physical place and where it does not make sense (from an aesthetic or conceptual standpoint) to use a “traditional” map. Examples of these applications are book covers, movie posters, or album covers, where the driving narrative is connected to a particular geographic region. To better illustrate the aforementioned point, we followed the first example and used the system to help design a mock-up cover for the 1988 book, The Twenty-Seventh City by Jonathan Franzen (figure 3). To do so, we generated a map of Saint Louis, Missouri (figure 4), the city where the narrative takes place, with a style image that could help express the genre of the book — a complex thriller.
Map of Coimbra, Portugal used for home decoration. The map was generated by the presented system and later digitally composed into a real-world scene using image editing software.
Mock-up cover for the 1988 book The Twenty-Seventh City by Jonathan Franzen, using a stylised map of Saint Louis, Missouri, generated with the presented system.
Input map (left) and style image (middle) used to generate the output map (right) to be used in the book cover shown in figure 3.