Word-As-Image for Semantic Typography
A few examples of our Word-As-Image illustrations in various fonts and for different textual concept. The semantically adjusted letters are created
completely automatically using our method, and can then be used for further creative design as we illustrate here.
A word-as-image is a semantic typography technique where a word illustration
presents a visualization of the meaning of the word, while also
preserving its readability. We present a method to create word-as-image
illustrations automatically. This task is highly challenging as it requires
semantic understanding of the word and a creative idea of where and how to
depict these semantics in a visually pleasing and legible manner. We rely on
the remarkable ability of recent large pretrained language-vision models to
distill textual concepts visually. We target simple, concise, black-and-white
designs that convey the semantics clearly.We deliberately do not change the
color or texture of the letters and do not use embellishments. Our method
optimizes the outline of each letter to convey the desired concept, guided by
a pretrained Stable Diffusion model. We incorporate additional loss terms
to ensure the legibility of the text and the preservation of the style of the
font. We show high quality and engaging results on numerous examples
and compare to alternative techniques.
Description
Official implementation of Word-As-Image for Semantic Typography paper.
Setup
- Clone the repo:
git clone https://github.com/WordAsImage/Word-As-Image.git
cd Word-As-Image
- Create a new conda environment and install the libraries:
conda create --name word python=3.8.15 conda activate word pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113 conda install -y numpy scikit-image conda install -y -c anaconda cmake conda install -y -c conda-forge ffmpeg pip install svgwrite svgpathtools cssutils numba torch-tools scikit-fmm easydict visdom freetype-py shapely pip install opencv-python==4.5.4.60 pip install kornia==0.6.8 pip install wandb pip install shapely
- Install diffusers:
pip install diffusers==0.8 pip install transformers scipy ftfy accelerate
- Install diffvg:
git clone https://github.com/BachiLi/diffvg.git
cd diffvg
git submodule update --init --recursive
python setup.py install
- Paste your HuggingFace access token for StableDiffusion in the TOKEN file.
Run Experiments
conda activate word cd Word-As-Image # Please modify the parameters accordingly in the file and run: bash run_word_as_image.sh # Or run : python code/main.py --experiment <experiment> --semantic_concept <concept> --optimized_letter <letter> --seed <seed> --font <font_name> --use_wandb <0/1> --wandb_user <user name>
--semantic_concept: the semantic concept to insert--optimized_letter: one letter in the word to optimize--font: font name, the .ttf file should be located in code/data/fonts/
Optional arguments:
--word: The text to work on, default: the semantic concept--config: Path to config file, default: code/config/base.yaml--experiment: You can specify any experiment in the config file, default: conformal_0.5_dist_pixel_100_kernel201--log_dir: Default: output folder--prompt_suffix: Default: «minimal flat 2d vector. lineal color. trending on artstation»
Examples
python code/main.py --semantic_concept "BUNNY" --optimized_letter "Y" --font "KaushanScript-Regular" --seed 0
python code/main.py --semantic_concept "LEAVES" --word "NATURE" --optimized_letter "T" --font "HobeauxRococeaux-Sherman" --seed 0
- Pay attention, as the arguments are case-sensitive, but it can handle both upper and lowercase letters depending on the input letters.
Tips
If the outcome does not meet your quality expectations, you could try the following options:
- Adjusting the weight 𝛼 of the L𝑎𝑐𝑎𝑝 loss, which preserves the letter’s structure after deformation.
- Modifying the 𝜎 parameter of the low-pass filter used in the L𝑡𝑜𝑛𝑒 loss, which limits the degree of deviation from the original letter.
- Changing the number of control points, as this can influence the outputs.
- Experimenting with different seeds, as each may produce slightly different results.
- Changing the font type, as this can also result in various outputs.
Acknowledgement
Our implementation is based ob Stable Diffusion text-to-image model from Hugging Face’s Diffusers library, combined with Diffvg. The framework is built on Live.
Licence
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Our word-as-image illustrations concentrate on changing only the geometry of the letters to
convey the meaning.
We deliberately do not change color or texture and do not use embellishments.
This allows simple, concise, black-and-white designs that convey the semantics clearly.
We rely on the prior of a pretrained Stable Diffusion model to connect between text and images, and utilize
the Score Distillation Sampling approach to encourage the appearance of the letter to reflect the
provided textual concept.
Given an input word, our method is applied separately for each letter.
We represent each letter as a closed vectorized shape.
Given an input letter represented by a set of control points 𝑃, and a concept (shown in purple),
our goal is to optimize its parameters to reflect the meaning of the word, while still preserving its original style and design.
we optimize the new positions 𝑃ˆ of the deformed letter iteratively. At each iteration, we use a differentiable rasterizer (DiffVG marked in blue) that allows to backpropagate gradients from a raster-based loss to
the shape’s parameters.
We then augmented the rasterized deformed letter and passed into a pretrained frozen Stable
Diffusion model, that drives the letter shape to convey the semantic concept using the Lsds loss (1).
To preserve the shape of the original letter and ensure legibility
of the word, we utilize two additional loss functions. The first loss
preserves the local tone and structure of the
letter by comparing the low-pass filter (LPF marked in yellow) of the resulting rasterized
letter to the original one to compute L𝑡𝑜𝑛𝑒 (2).
The second loss regulates the shape modification by constraining the deformation
to be as-conformal-as-possible over a triangulation of the letter’s
shape (D marked in green), defining L𝑎𝑐𝑎𝑝 (3).
Автор:
22 февраля 2018 17:56
Проект американского дизайнера корейского происхождения Джи Ли (Ji Byul Lee) «Word as image» лаконичен и понятен без слов. Талантливый мастер, ранее работавший в GOOGLE и продолжающий свою деятельность в Facebooke, кажется, может нарисовать, что угодно — и это всегда будет кратко, доступно и очень креативно.
Источник:
В этом посте не будет комментариев — они абсолютно излишни. Добавим лишь, что этот проект получил продолжение в виде промо-ролика.
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Источник:
Промо-ролик проекта
Больше прекрасных иллюстраций дизайнера можно найти на странице Инстаграм* этого проекта. тут
Источник:
Ссылки по теме:
Новости партнёров
реклама
Ji Lee’s short description:
“This project started nearly twenty years ago as an assignment in my typography class at art school. Students were encouraged to see letters beyond their dull, practical functionality.
“The challenge is to visualize the meaning of a word, using only the graphic elements of the letters forming the word, without adding any outside parts. The challenge was hard, but the reward of ‘cracking’ a word felt great. So this became a lifelong project for me.”
The wordplay idea isn’t new (see Tom Geismar and the late Ivan Chermayeff’s Watching Words Move from 1962), but it’s a light-hearted project that might inspire you to merge the appearance of other words with their definitions.
More of Ji Lee’s here, though I think I chose the strongest.
-
Log in
-
Join
Watch in our app
Open in app
-
-
October 19 2011, 16:05
- Литература
- Cancel
В своей книге, так же как и в ролике ее продвигающем, бывший креативный директор Google Creative Labs автор и дизайнер Ji Lee демонстрирует, как можно превратить слово в картинку, используя только графические элементы букв.
В книге Ji Lee «Word as Image» представлено более 100 иллюстраций на тему превращения слова в образ …
Добавить данный пост в такие социальные сети как:
ПРОГУЛКА ПО ЖУРНАЛУ
 |
|
|
|
|
|
|
|
|
|
Известный американский дизайнер корейского происхождения Джи Ли (Ji Lee) в рамках работы над своим независимым проектом, в котором он играл со словами и превращал их в ожившие образы, издал замечательную книгу «Word as Image». В нее вошли иллюстрации, созданные за многие годы его творческих упражнений с типографикой и символикой. Работы, представленные в книге, иллюстрируют то, как создаются хорошие логотипы и символы из простых слов. Сразу хотим предупредить, что книга не новая, поэтому, возможно, кто-то из вас уже видел его творчество. Но они настолько забавные и интересные, что хочется показать их еще раз. Только это еще не все: впоследствии дизайнер сделал видеоролик, в котором слова и образы действительно оживают с помощью анимации! Видео находится внизу страницы, сразу после иллюстраций. Наслаждайтесь.
Графический дизайн | 28.11.2012
5269 просмотров
Серия плакатов по мотивам мультфильмов Pixar
Художник и дизайнер из Австралии Уончан Ли (Wonchan Lee) создал серию минималистичных плакатов по мотивам мультфильмов Pixar….
Графический дизайн | 28.11.2012
2014 просмотров