FontLerp Article 1 Export 1.pdf - Roey Shapiro

Please enable JavaScript to view the full PDF

Font Interpolation Roey Shapiro August 2022 C ontents (click to jum p to a section): • initial motivation for font interpolation • background info on basic vector-graphic fonts; loose definitions of Bezier curves, Contours, and Glyphs • glyph mapping; simplifying the problem into contour mapping • contour mappings • miscellaneous bits and bobs, conclusion A ssum es you know : • what linear interpolation (lerping) is • set and sequence notations • basic linear algebra • formal definition of functions If you have form al m aths know ledge, there’s som e of that in here, too! This is very m uch a w ork in progress. The code is at this G itH ub link, and throughout the article there w ill be (∗) sym bols w hich indicate places w here I think there is easy room for im provem ent. Feel free to jum p around by using C ontrol + F to find these spots and share your ow n ideas and code! A ll links w ill also appear at the end of the article. 1 What and Why is Font Interpolation? 1.1 Motivation For some motivation and to help picture precisely what Font Interpolation is, consider the following short story I wrote a while ago: Slanted sunlight floated down through the opened windows, gently framed by the wispy curtains flowing in the breeze. The room was a gentle yellow. Grandfather lay on the bed, hypnagogic. He didn’t seem to be looking at anything, his eyes focused to some vague infinity. Watching paint dry or grass grow. Listening to a wail become a “mama” and somehow incrementally a voicemail describing college life. Cooking an egg sunny side up, or whipping cream to soft peaks. Then, it was one thing, and now, it is another. These fragile diagonal rays of sun - how long would it take them to melt this block of ice before me? My eyes pass over the leathery surface of Grandfather’s forehead in optic tiptoe. It would be rude to disturb his reverence. . . or whatever it is. It is rare to see the moment of transformation, perhaps because it is difficult to define. Theseus was unaware, but I will watch this metamorphosis. I owe it to Grandfather. In a moment of ultimate irony, considering these things caused my own eyes to become unfocused from his pale skin. It only took that moment; I had failed to witness the transition into the end. Without my noticing, the final piece of the Grandfather had melted away. Only cold water remained. While not much more than a 2AM ramble, it did manage to speak to me about transitions that happen right under our noses - a fascinating thought for any field... if one can get in the correct headspace. It’s easy to look at something like this and write it off as detached and overly poetic. I started considering other ways to help a reader empathize with the feeling of not realizing that a transition is occurring until it’s already occurred. Maybe I could change the font somehow to convey this? There are ways to gradually change a font’s boldness or italicization gradually – there are tools to do that and it isn’t a particularly difficult issue to solve compared to the more general issue of Font Interpolation outlined here. I wanted to be able to choose any two fonts and have the text start as one and end as the other – that didn’t exist yet. Here's an image to help better visualize the effect: 2 How do Fonts work? We should start by considering how fonts are rendered in most environments on the web or word-processing environments. On the surface, it may seem that a given font’s letters are simply images that are pasted on different parts of a page. Have you ever noticed that when you zoom into a page, though, the letters don’t get more pixelated? That’s because each character, or “glyph” as they’re called in the typographical world, is stored in an abstract “Vector Format”. Basically, this means that each glyph is stored in memory as abstract curves and other drawing instructions. When it comes time to draw the glyph, it can be drawn at an accuracy fitting the current zoom level. The drawing instructions dictate how and which pixels are filled in. In this way, you can experience functionally limitless accuracy and font sizes with a single file. For more info, check out this video by Anya Smilanick: Vector-based images vs. Pixel-based (Bitmap) images I won’t go super in-depth on this, so please do some reading. There are great resources out there! The main thing to note here is that you have Bitmap images, often called “Raster” images, which are in pixels, and Vector images, which have the potential to be “Rasterized” and turned into a fixed pixel image on your screen depending on your desired accuracy. Vector images are the type we’re interested in. 3 Bezier, Contour, Glyph We’re now ready to build up to the glyphs that comprise a font, starting from Bezier curves. Simply put, a Bezier curve is an abstract curve defined by a few control points. Below is a Bezier defined by four control points: You may be able to imagine how stitching these together, one after another, could produce the desired effect of forming pretty much any shape we want. If you want to learn more about specifics of Bezier curves (i.e. how to draw them, how they’re defined, etc.), try out this wonderful interactive website by Pomax. There are two key takeaways here: 1) any given Bezier curve can be represented as a matrix. In this case, since each point exists in 2D space, and there are four points, we could have a (4 x 2) matrix, where each row represents a point. 2) a Bezier curve simply gives instructions of how to get points along its curve; to “draw the curve”, you have to decide how accurate you want to be, pick that many points along the curve, and then when it comes to actually drawing the thing, you just draw straight lines between those points. You never actually draw a curve – just straight lines. Font designers actually do precisely this - by starting each new curve from the end of the previous and, crucially, forming a closed loop, they form what is known as a Contour. For example, consider the following 4 Bezier curves forming this circle contour: So we can see how as a mathematical object, a contour can simply be an ordered list, or finite sequence, of Bezier objects. In this case, if we let the constant 𝑐 = 0.5522847, that long number in the image, we could define this contour 𝐶1 as follows: 0 1 1 0 0 −1 −1 0 𝑐 1 1 −𝑐 −𝑐 −1 −1 𝑐 𝐶1 = ([ ],[ ],[ ],[ ]) 1 𝑐 𝑐 −1 −1 −𝑐 −𝑐 1 1 0 0 −1 −1 0 0 1 Finally, a glyph can simply be defined (for our purposes here) as a set of contours. In theory, this contour, 𝐶1 , could be a glyph all on its own. Let’s say we called it 𝐺1 . So here, we’d have 𝐺1 = {𝐶1 } – just the one contour! If it were, though, we’d obviously expect to see this: And not, well… this: [cricket noises] This, again, is a matter of the rasterizing process. Recall that before I mentioned how a program needs to decide how to fill in certain pixels in order to draw the glyph. If it were trying to draw this circle, it would need to know: is this a “fill in” contour or a “subtract” contour, so to speak. We only see the outline of this contour while editing it in special software! When it’s drawn, it’s pixels inside a closed shape, not an outline! Thus, if the circle was a “subtract” contour, it might not be visible at all. It’ll be easier to see on this letter “a”. Let’s define a new glyph 𝐺𝑎 to represent it. There are two contours in 𝐺𝑎 . Let’s say 𝐺𝑎 = {𝐶𝑖𝑛 , 𝐶𝑜𝑢𝑡 }. 𝐶𝑜𝑢𝑡 defines the larger overall area that’s bounding the whole thing, and 𝐶𝑖𝑛 defines the little hole in the middle – it says, “if some other contour is trying to draw in the area inside of me, don’t let it!”. It’s a bit more complicated than that in practice, but just so you get the idea for now. That’s what I meant earlier, though: it may be that the circle at the start is a “subtract” type contour. If there’s nothing to subtract from, depending on the Rasterizer (the program that turns the abstract curves into pixels), it might just happily subtract from nothing and show nothing at all. I’ve rendered 𝐺𝑎 such that we can see the order of the Bezier curves of each contour by their color. We begin from the first Bezier being colored with red and the last with green. Two things: 1) You might be able to notice, firstly, that the color blending is continuous. You might take this for granted, but it confirms that whoever made this font indeed made the contours one continuous cycle. The curves don’t just form the correct shape – they do it in a connected sequence. 2) A second thing to observe is that 𝐶𝑜𝑢𝑡 goes clockwise in the order of its Beziers and 𝐶𝑖𝑛 counter-clockwise. This is because noting which are clockwise or counter-clockwise can provide an easy way of implicitly defining which contours are “filler-inners” and which are “subtracters”. It is simply interpretable from the point-data itself using, for example, a winding number calculation algorithm. One final bit of notation. I’ll be using the size notation in the following ways: For a glyph 𝐺, |𝐺| ≔ # 𝑜𝑓 𝑐𝑜𝑛𝑡𝑜𝑢𝑟𝑠 𝑖𝑛 𝐺 For a contour 𝐶, |𝐶| ≔ # 𝑜𝑓 𝐵𝑒𝑧𝑖𝑒𝑟𝑠 𝑖𝑛 𝐶 Ok, sweet! We know at least loosely what Beziers, Contours, and Glyphs are, and we’ve come up with simple mathematical representations for them. Now for the interpolation process. 4 The General Goal of Font Interpolation There are two stages to interpolating between two glyphs: finding a mapping between certain elements of the glyphs, and actually performing the interpolation between them given some percentage 𝑡 ∈ [0,1]. The precise goal of the mapping phase of the process is perhaps subjectively definable. One straight-forward mathematical approach is to pick certain points (more on how we pick points later) on each glyph and find the mapping between them which simply reduces the mean squared error (MSE). One qualitative attribute we certainly want to maintain is identifiability. How can we adhered to attributes of each font’s signature look at all percentages, but also always be able to identify the interpolated glyph as the correct character? For example, given two drastically different ”a” glyphs: … it wouldn’t be particularly desirable to obtain the following ”a” 50% between them, since it doesn’t look like an “a” (or… anything, really): The interpolation phase of Font Interpolation can also vary drastically from method to method. Explored here is the naive approach of simply linearly interpolating between any two points in the mapping found. 4.1 Contour-Naivete I employed a ”Contour-Naive” (CN) methodology. Consider these ”k” glyphs: See how each has a single contour? This means that a mapping method would only have to consider how to move points or curves around between a single path in order to consider each glyph in its entirety. You might not even see the issue I’m getting at, so consider these glyphs now in comparison: These each have two contours, as we saw before. What does the Contour-Naïve approach do? Well, it basically makes it so that a given contour does not ”see” any other – you cannot map two points to two different contours on one glyph to a corresponding pair on a single contour in the other glyph. I know that’s confusing… just look at this picture: Suppose we didn’t apply CN and somehow mapped the 𝑝 to 𝑞 and 𝑥 to 𝑦. Maybe this happened because we were using a MSE-based approach on the entire glyph and saw that they were reasonably close, for example. Interpolating 𝑝 to 𝑞 and 𝑥 to 𝑦 would tear the outer contour of the left ”a” and the inner contour of the right ”a”: CN guarantees that there will be no such tearing, even if it limits the final result in more complex cases (as shown in the botched cursive ”a” example). It also simplifies the entire problem. For one, the interpolation step becomes simpler: we only have to worry about interpolating between two contours at a time rather than two entire glyphs. It also simplifies the mapping step, since we can now just look at mapping points on a single contour to some other. Contour-Naivete works especially well in cases where glyphs are reasonably similar to each other, and especially if they have the same number of contours. It doesn’t always perform particularly badly when these conditions aren’t satisfied, either. There’s one more thing to consider. What if, for some reason, the glyphs don’t even have the same number of contours? Where do we map the extra ones? The following algorithm solves all of these issues. 4.2 Glyph mapping under Contour-Naivete We define a Contour-Naive glyph mapping as a set of pairs of contours. One can be built as follows (∗): Let 𝐶 ∗ be the space of all possible contours. Let 𝑀∗ = ⋃∞ 𝑖=0(ℝ × ℝ ) be the space of all mappings of finitely many points in ℝ . 2 2 𝑖 2 Let 𝐺1 , 𝐺2 be glyphs such that |𝐺1 | ≥ |𝐺2 | and let 𝑛1 = |𝐺1 |, 𝑛2 = |𝐺2 |. Finally, let 𝑀𝐶 : 𝐶 ∗ × 𝐶 ∗ → 𝑀∗ × ℝ be a contour mapping function: one which takes two contours and returns a mapping and the score of that mapping, where the higher the score, the better the mapping is. This gets a bit complicated, but just take it slowly and it should be clear. Note that we assume the existence of the function that can find a mapping of points on two contours (referred to as 𝑀𝐶 ). You haven’t missed anything – the final part of this article covers what types of contour mapping functions I came up with and used. For now, just assume we have some 𝑀𝐶 that does what we want it to do: spit out a mapping of points and a score for how good the mapping is. The glyph mapping process I came up with uses two steps: First, map each contour in 𝐺2 , the glyph with less contours, to a contour in 𝐺1 , that with more. Second, map the remaining 𝐺1 curves to whichever curves are individually best in 𝐺2 . The goal is obviously to maximize the score, but we must constrain the problem to hopefully maintain key features of the character we’re trying to produce. This glyph mapping method guarantees that each contour in 𝐺2 is represented at least once – each is mapping to some contour in 𝐺1 . Here are the details: We begin with step one by finding 𝐼 ⊆ 𝐺1 , an initial subset of 𝐺1 ’s contours, such that |𝑆1 | = 𝑛2 . We determine 𝐼 out of all possible subsets of 𝐺1 by exhaustively picking some possible 𝑆 ⊆ 𝐺1 and some permutation of 𝐺2 ’s contours to form an initial mapping set: 𝐼 = {(𝐺1 𝑐𝑜𝑛𝑡, 𝐺2 𝑐𝑜𝑛𝑡)1 , (𝐺1 𝑐𝑜𝑛𝑡, 𝐺2 𝑐𝑜𝑛𝑡)2 , … , (𝐺1 𝑐𝑜𝑛𝑡, 𝐺2 𝑐𝑜𝑛𝑡)𝑛2 } … such that 𝐼 maximizes the sum of scores 𝑀𝐶 returns from being run on each (𝐺1 𝑐𝑜𝑛𝑡𝑜𝑢𝑟, 𝐺2 𝑐𝑜𝑛𝑡𝑜𝑢𝑟) pair. If you want the formal stuff: Denote 𝑆(𝑋) the set of all ordered permutations of a set 𝑋. For 𝑃(𝑋) being the power set of a set 𝑋, we denote 𝑃𝑎 (𝑋) = {𝑥 ∈ 𝑃(𝑋) | |𝑥| = 𝑎}. Therefore, we have: 𝑛2 𝐼= argmax ∑ 𝑀𝐶 (𝐺1 𝑐𝑜𝑛𝑡, 𝐺2 𝑐𝑜𝑛𝑡). 𝑠𝑐𝑜𝑟𝑒 𝐼∈(𝑃𝑛2 (𝐺1 ) × 𝑆(𝐺2 )) 𝑖=1 Now for step two. We’ve bijectively mapped 𝑛2 of 𝐺1 ’s contours to each of 𝐺2 ’s contours, leaving us with 𝑛1 − 𝑛2 𝐺1 contours to map. As mentioned before, we won’t impose any restrictions on them, since all of 𝐺2 ’s contours have already been represented at least once by step one. Thus, we simply iterate through each remaining 𝐺1 contour and find which of 𝐺2 ’s contours it yields the highest score with. Formally, this gives us for each remaining 𝐶1 ∈ 𝐺1 , mapping 𝐶1 to: max 𝑀𝐶 (𝐶1 , 𝐶2 ). 𝑠𝑐𝑜𝑟𝑒 𝐶2 ∈𝐺2 … and we’re done! We have the initial 𝑛2 contours of 𝐼 and the remaining 𝑛1 − 𝑛2 contours mapped here, so altogether we’ve mapped all 𝑛1 contours from 𝐺1 to all 𝑛2 contours of 𝐺2 : an injective glyph mapping, as we wanted! 4.3 Using Contour Fill-types to maintain key features of a character As stated before, the CN-method doesn’t lerp the whole glyph – we’ve reduced the problem to figuring out just how to lerp a contour to another. But that doesn’t mean that we can’t bias the scores so that obvious mistakes don’t happen. Take that example from before: Obviously, we’d want to match the inner contours to each other and outer contours to each other. If the glyphs are reasonably similar to each other, this will happen naturally. It might seem like we can’t influence that choice any more than that once we decided to throw our hands up with Contour-Naivete. But recall from before that simply from the point data we can determine whether a contour is a “fill-in” or “subtract”. This can be used in myriad ways: making contours whose “fill modes” don’t match have half the score, or −∞ score, or adding them to some sort of queue if you have lots of −∞ ones, etc. (∗) Point is, you don’t have to do much to get it to do pretty much what you want. 5 Contour Mappings We’ve subdivided all the way down, going from fonts to glyphs to contours. We’re now ready to tackle the problem we’ve reduced this all down to: what good ways are there to map a given contour to another? This is where our review of Bezier curves will come in handy. There are three main methods I explored, each of which relies on final subdivision, either into curves or points along the contour. At those levels, lerping is easy to understand: To lerp between two points, simply lerp! To lerp between two curves, simply ensure that they have the same number of control points, and then lerp the start points together, the first control points together, and so on, until the end points are lerped together. Similar to the macro-situation of two glyphs with different numbers of contours, things really get tricky when two contours have different numbers of curves. In all of the following algorithms and examples, I’ll maintain the convention that for two contours 𝐶1 , 𝐶2 , |𝐶1 | ≥ |𝐶2 |. We’ll also call |𝐶1 | = 𝑛1 and |𝐶2 | = 𝑛2 . This is where most of the work went in – where most of the magic happens. I’d love to see what others come up with! (∗) I’ll be detailing these sort of as a story, working through ideas roughly in the order I thought about them as I worked through this problem myself. 5.1 The Reduction Method This technique uses a very simple idea. 𝐶1 has more Bezier curves than 𝐶2 . Let’s find which 𝑛2 of the 𝑛1 curves in 𝐶1 are the “most important” or “best fit” the 𝑛2 curves of 𝐶2 . Any other 𝐶1 curve shrinks down to a single point: it is reduced to nothing. For example, let’s say we had the following two contours: Let’s denote the hexagon and triangle 𝐶𝐻 , 𝐶𝑇 respectively. The reduction method sees that |𝐶𝐻 | = 6 ≥ 3 = |𝐶𝑇 | and therefore looks for the 3 most fitting curves in 𝐶𝐻 . For example, it might make this choice: What happens to the remaining 3 𝐶𝐻 curves? They’re “reduced” down to a single point – whichever point separates the corresponding curves in 𝐶𝑇 – in the interpolation process. In this example, it would look like this: I’ve bracketed the three remaining, unpaired curves and the 𝐶𝑇 points they’ll be reduced into in the interpolation process. In addition, I’ve marked 3 𝐶𝐻 curves and their corresponding elements in 𝐶𝑇 to show how this works for one of these reduced, “null curves”. In our mapping earlier, we mapped 𝑥 → 𝑎 and 𝑧 → 𝑐. As we’re interpolating between these two contours, we walk around 𝐶𝐻 , the larger contour, and see ok… 𝑥’s partner is 𝑎. Next curve. 𝑦 doesn’t have a partner. What’s the last endpoint we’ve seen? The endpoint of 𝑎! We can create a new curve that just has all of its points being that endpoint – let’s call it 𝑁𝑎 for “Null curve a” and suddenly 𝑦 does have a partner: 𝑁𝑎 ! And we continue in this way. If multiple successive 𝐶𝐻 curves need to be mapped to null curves, it works in the same way. The last endpoint we’ve seen is only updated when we map another non-null curve. In this case, we’d continue to 𝑧, see it has a non-null curve partner 𝑐, and update the last endpoint to be the endpoint of curve 𝑐. So that’s all great, but how do we find that mapping in the first place? Sometimes when solving a very open ended problem it’s easier to begin by asking the opposite of what you want first. In this case, we don’t ask, “what do we want” but rather “what don’t we want” to give us some structure. In this case, something we definitely don’t want is tearing. Think about that “a” that we pinched and tore up (RIP) in section 4.2. We can prevent this by making sure that our contour mapping function maps curves in order. So that’s one constraint: we can only look at in-order sequences of curves on each contour. So that’s how I started. I literally made a list of all in-order subsets of 𝐶1 ’s curves that were of size 𝑛2 . For any subset of curves in that list and a rotation offset, I compute the mean squared error (MSE) of each curve in 𝐶2 . For example, let’s say I label the curves as such: … and I begin by checking the 𝐶1 subset {4, 5, 6} and the MSE of (4, 1), (5, 2), (6, 3), where each pair is of the form (𝐶1 𝑐𝑢𝑟𝑣𝑒, 𝐶2 𝑐𝑢𝑟𝑣𝑒). Now for that same choice of 𝐶1 ’s 4, 5, and 6, I can try 2 more rotated choices of 𝐶2 ’s 1, 2, and 3: 𝑜𝑓𝑓𝑠𝑒𝑡 = 1: (4, 2), (5, 3), (6, 1) 𝑜𝑓𝑓𝑠𝑒𝑡 = 2: (4, 3), (5, 1), (6, 2) So, just for one more iteration of this for example, I could then pick the 𝐶1 subset {1, 4, 5}. I do the same thing with 𝐶2 , checking the three offsets: 𝑜𝑓𝑓𝑠𝑒𝑡 = 0: (1, 1), (4, 2), (5, 3) 𝑜𝑓𝑓𝑠𝑒𝑡 = 1: (1, 2), (4, 3), (5, 1) 𝑜𝑓𝑓𝑠𝑒𝑡 = 2: (1, 3), (4, 1), (5, 2) For each of these, we check what the MSE was; if that subset of 𝐶1 and rotational offset of 𝐶2 yielded a lower MSE than we’ve seen before, we save them and continue looking. Exhaustively. … yeah, I don’t really know what I thought would happen. It worked really quickly on my test contours, which had, like, 6 curves max. So quickly I could have it run every frame and recalculate the best mapping as I dragged around curves in real time (see it in action in this video). But then when I started importing fonts off the internet and saw even basic glyphs had 30 curves…. Yeah. This exhaustive search |𝑛 | grows factorially, since it’s a function of the choose operation. There are ( 1 ) = |𝑛2 | |𝑛1 |! |𝑛2 |!|𝑛1 −𝑛2 |! subsets, and for each of those 𝑛2 offsets to run through. To give you an idea, one of my first ignorant tests had 𝑛1 = 88, 𝑛2 = 37, which yields 3.214 ∗ 1026 iterations. Optimistically assuming that each takes a nanosecond, that’s 10 billion years. So… no. Not very practical. It does work well when it’s done, though! If you find contours whose difference of number of curves is small, then the Reduction algorithm is actually still quite feasible in terms of time. This makes sense – the closer the number of curves, the closer you’re getting to asking about a bijective mapping, meaning you must be “more decisive” or “more constrained” in your options. [You may also notice that this is similar to the exhaustive approach of the first part of the glyph mapping algorithm. Why is it feasible there and not here? Well, glyphs tend to have at most 3 contours, so the numbers just stay so small that the factorial doesn’t matter.] If we give up on constraints, can we make it faster? … How about greedily? 5.1.2 Greedy Reduction In computer science, a “greedy” algorithm is one which only ever picks the locally optimal choice, but this always leads to a globally optimal solution. Well, as much as I banged my head against it, I couldn’t figure out a way to greedily pick while guaranteeing that the order of the curves would be maintained. Still, I tried it out. The idea is to calculate the MSE (or whatever other scoring system you like – there might be a better one!) for every combination of curves. Then, you simply sort the list and pick the highest scoring (𝐶1 𝑐𝑢𝑟𝑣𝑒, 𝐶2 𝑐𝑢𝑟𝑣𝑒) pair that hasn’t already had either of its curves picked yet. And this… doesn’t really work. It simply can’t be relied on that the glyphs are similar enough (in general features or scale) that this will work in its current form. I tried many variations of mixing greedily picking with random subsets, sorting by the highest score relative to which curve was chosen last, and more. It’s likely that there’s a way to make greedily choosing feasible. I just couldn’t find it. Which lead me to another approach… 5.2 Pillow Projection The Pillow Projection (PP) method doesn’t look at curves at all. It looks at the contour as a whole. Given some number 𝑚 of sample points, PP walks along the contour and places 𝑚 equidistant points. In this case, 𝑚 = 100: 𝐶𝑇 upper left 𝐶𝐻 upper left How do we approximate the length of the contour? Well, if you were paying attention to the Bezier refresher at the start, a Bezier curve is drawn at a given accuracy. That accuracy simply determines how many straight lines are drawn to give the impression of drawing a curve. We can use a relatively high accuracy to get a decent approximation of each Bezier’s length by adding up the lengths of those straight line segments. By extension, we can calculate an approximate contour length. You might already see where this is going. We literally just match each point on the left with a corresponding point on the right. How do we decide which point goes to which? Here are two ways: 1) Basically do what we did with the rotational offset in the Reduction method and check 100 offsets: first matching the points (1,1), (2,2), … , (100,100), then the next offset, doing (1,2), (2,3), … , (100,1), and so on. See which gives minimal MSE, and choose it; this inherently gives a point mapping as well 2) Choose some “anchor point” relative to each contour’s bounding box. In the example above, it’s the upper-left, marked with an “X”. For each contour, go through each of its 100 points to find which is closest to the anchor. That point can now be marked as “point 0” on each contour, and every point after it 1, 2, and so on, until 100. We match the point 0’s, the point 1’s, and so on. There might be a better way to choose an anchor point! (∗) Now, having a bunch of points is great, but all we can do with points is draw straight lines between them, right? Not great if we want to retain the cool, continuous curves that we had access to before stripping these contours down to discrete points. Luckily, there are algorithms that fit Bezier curves to predetermined points, like this one. We can use that to rebuild the glyph with curves, so the process is like this: 1) Find 𝑚 points on each contour 2) Index the points based on some anchor which is relative to the contour’s bounding box 3) When lerping, simply lerp a given point on one contour with its corresponding point on the other; doing this for all points gives 100 lerped points that form a new contour 4) Re-smooth the points of this discrete contour into a continuous contour with a Bezier point-fitting algorithm That’s it! The score returned is simply the total MSE of all point pairs. This technique feels like cheating to me. It’s dead-simple, but not mathematically satisfying. It’s elegant in a sort of naïve way. That is, it doesn’t really retain the precise abstract points that comprise the two contours – it basically approximates them and then gives you something that looks like the originals, but will probably have pillowed corners everywhere there used to be sharp ones if you zoom in closely enough (hence the name “Pillow Projection”). When one of the benefits of vector- based graphics is basically infinite precision and zooming as much as we please, this feels a bit like ruining a foundational aspect of the fonts were using. That said… it gives some damn great results, and quite quickly (seconds, not billions of years). Here you can see it working even on quite drastically different contours (not only in feature proportions, but also scale): The reasoning as to why is pretty basic, too: if two contours are even relatively similar to each other, it’s because we identify them as such. How do we identify “similar” contours? If they have the same relative distances between their main features. The line-and-circle structure of a “p” or “q” is pretty consistent, for example. It’s fast, simple, and doesn’t care about discrepancies in scale. If one font happens to have one character look really small compared to the same character in another font, Pillow Projection will gladly find the same mapping. 5.3 Relative Projection My final attempt, the Relative Projection (RP) method was supposed to remedy some of the imperfect maths happening in Pillow Projection. It’s a bit more involved, and, if I’m being honest, I haven’t even fully implemented it, but it’s my favorite of all of these contour mapping algorithms. There’s yet a bit of debugging to do, but I’m fairly sure it’s mostly working as intended. Here’s the concept, again on our friends 𝐶𝐻 and 𝐶𝑇 . Begin by getting 𝑚 sample points on each contour, as before, and find some “agreement point” that’s determined relative to some anchor point. Again, we’re using the upper left of the contour’s bounding box. We now have all of these sample points labelled from 0 to 𝑚 − 1. Unlike PP, we don’t use the indices to match the points. Instead, we use them to create a new unit: the “contour-fraction” (oooh, exciting). The contour-fraction is simply the fraction along the contour a given point is, relative to the position of sample point 0. Suppose that in the following examples, the large black dot on each contour is what’s been determined to be sample point 0. We’ll see some points along each contour labelled with their contour-fraction values. The next step is to now mark each Bezier’s start point with its contour-fraction value. Since these are polygons, the numbers will turn out quite even, but don’t let that distract from how generic the values can be: Now comes the magic: we’ve put the points into the same terms! We can mark each contour with the other contour’s contour-fractions to get two “marked contours”. Note this crucial fact: we have the same number of marked points on both contours: in this case 6 + 3 = 9 points. For this final part of the mapping process, begin by looking just at the marked 𝐶𝐻 on the left-hand side. As it stands, it’s still just comprised of its 6 original Bezier curves. However, whenever there’s a point mapped from 𝐻𝑇 onto one of these curves, we’ll split the curve. By the end, we’ll have 9 curves. I’ll label them, starting from the point with the lowest contour-fraction value: One thing to note before moving on is that the only 3 curves that weren’t interrupted were the same ones that the Reduction method chose to map directly. I suppose that means that we’re doing something right! Anyway, we now do the same thing for the marked 𝐶𝑇 : And there we go! We’ve made it so that both contours have the same number of curves, and we already know exactly which curve to map to which – their indices are already right there. From here, all we need to do is interpolate, which can simply be done by interpolating between the curves we found normally. This should give results at least as good as Pillow Projection, and quickly, and maintains the mathematical integrity of the contour data: the best of all worlds! … but my implementation was sort of rushed. Feel free to finish it for me :/ Score can also be returned as the MSE of the corresponding Bezier curves. 6 Performance Optimizations and Other Cool Small Problems There were a few other small problems I felt smart for figuring out along the way. Why not share ‘em? Being able to walk along the contour and find those sample points seems simple, but it turns out that it opens up a whole new space of solutions. Before, I had just been considering staying on the Bezier level. It also allowed me to find an accurate glyph center, which is useful for all sorts of glyph-manipulating goodness. Simply take, say 100 sample points, spread them evenly along the glyph (each contour gets a certain number of points in proportion to how much of the glyphs total length that contour accounts for), and then take their average. Bam: center of a glyph. I also experimented with using the sample-point technique rather than MSE in the Reduction method, so that I wouldn’t need to count on glyphs being the same scale. Rather than searching for which Beziers reduce MSE the most, you search for which Bezier on the other contour includes the largest number of points of the same indices as you include on your contour. Remember how I said a Bezier curve could be represented as a matrix? Well, it turns out that those lerps you do to generate a point on the curve from those control points is also pretty easy to represent as a symmetric matrix (expand, rearrange, yada yada) that then only needs to be multiplied by different powers of 𝑡, the amount along the curve we want to find. If we know what accuracy we want to render everything at (let’s say 15 straight lines), then we can precompute a matrix of those 𝑡 values once for that accuracy. It’s then just a matter of multiplying every Bezier in the scene by that 𝑡-matrix and bam, we have a totally vectorized method of calculating Bezier render points. In my editor, you can pan and zoom around the scene. To further save on performance, I noted that you only really need to redraw a Bezier when it’s been changed by the user dragging its points around, or when the user zooms in and we need to see it with higher accuracy. Thus, if either of those things happened on a given frame, we draw the curve from scratch onto a special draw surface which we then store. On any frame where we just pan around or fiddle around elsewhere, we don’t need to redraw the contents of the draw surface – just draw the whole surface somewhere else. 7 Ideas for Future Exploration (∗) There are many, many ways to take this. First and foremost, new contour mapping methods. There are already different metrics I can think of using rather than MSE, for example, that I just haven’t pursued much. Angle from the contour’s center, total length contributed to the contour, etc. At the end of the day, the goal was one of beauty – why leave this up to a computer? I got a little lazy and didn’t implement it fully, but I wanted to have a mode where some algorithm would suggest a mapping, and then the user could edit it as they please. Then, the user could select any valid interpolation method they want. This could be pseudo-automated by the user instead highlighting regions to constrain. For example, they highlight the stems and circles of the respective fonts’ “p” glyphs so that the algorithm will only map Beziers/points in the stems to each other and Beziers/points in the circles to each other. That sounds really easy to implement and might make certain unfeasible algorithms realistic. Obviously, there’s one final elephant in the room. How would one approach the problem through a holistic glyph approach, rather than my Contour-Naïve one? I’m sure there’s some Machine-Learning-y way to do that (that isn’t just “oh, let’s learn from Raster images” – we want abstract beauty here!!). Maybe those interpolations methods wouldn’t even be linear like they are here, taking strange, parabolic paths – or even Bezier curve paths as they interpolate from one point to another! It really is Beziers all the way down… Maybe we’d be using B-splines or something more generic, a la the drawing instructions in SVG files. Who knows? 8 Conclusion That’s all I’ve got! I’d call the project a reasonable success, and I’m now burned out on fonts to boot! It isn’t perfect, but it’s good enough for now. The codebase (shitty and unkempt, but the algorithms are there for you to see if you want ‘em) is on my GitHub. It’s called FontLerp for now. Please let me know if you have any ideas for improvements! Thanks for reading, and I hope your day only gets better from here. Roey R esources and A dditional R eading: FontLerp Github: https://github.com/Roey-Shap/FontLerp Videos of FontLerp in Action: https://youtube.com/playlist?list=PLRIDiXsuDNc0miu8VrzF59OjqiXGUh6ZN Vector Graphics video: https://www.youtube.com/watch?v=fy9Pby0Gzsc&ab_channel=AnyaSmilanick Interactive Bezier tutorial: https://pomax.github.io/bezierinfo/#curvefitting Algorithm for splitting Beziers into more Beziers: https://en.wikipedia.org/wiki/De_Casteljau%27s_algorithm Scalable Vector Graphics: https://en.wikipedia.org/wiki/Scalable_Vector_Graphics Winding Number: https://en.wikipedia.org/wiki/Winding_number