My graduation thesis on how to do Otomad sentence mixing 1st edition By : someone who primarily gets asked to do visuals 1 Summary Intro ................................ ................................ ................................ ................................ ..................... 2 This document’s purpose ................................ ................................ ................................ .............. 2 What to expect ................................ ................................ ................................ ................................ 2 What’s sentence mixing? ................................ ................................ ................................ ................... 3 Purposes of sentence mixing ................................ ................................ ................................ ........ 3 Singing ................................ ................................ ................................ ................................ ........ 3 Rhythm ................................ ................................ ................................ ................................ ........ 3 Uniquen ess ................................ ................................ ................................ ................................ 3 Humor ................................ ................................ ................................ ................................ ......... 3 Sentence mixing basics ................................ ................................ ................................ ..................... 4 Get in the good mindset, think syllables ................................ ................................ ....................... 4 Soft and hard consonants ................................ ................................ ................................ ............. 5 Stretching your syllables ................................ ................................ ................................ ................ 6 About gaps ................................ ................................ ................................ ................................ ...... 6 Filling gaps ................................ ................................ ................................ ................................ .. 6 ... Or not ................................ ................................ ................................ ................................ ...... 6 Summary ................................ ................................ ................................ ................................ ......... 7 Advanced sentence mixing concepts ................................ ................................ ................................ 7 The styles of sentence mixing ................................ ................................ ................................ ....... 7 Meaning - focused sentence mixing ................................ ................................ ........................... 7 Catchphrase - focused sentence mixing ................................ ................................ ..................... 9 Going even further with v oice mixing ................................ ................................ ...................... 10 Utter chaos ................................ ................................ ................................ ............................... 11 Mixing styles together ................................ ................................ ................................ .................. 11 Examples ................................ ................................ ................................ ................................ ... 12 Categorization and blurring the line ................................ ................................ ............................ 13 Conclusion ................................ ................................ ................................ ................................ ........ 13 2 Intro This document’s purpose Sentencing is a tedious process of audio that relies on small considerations and habits. Because so much of it is internalized, it can look like pure intuition or innate talent to others Sentence mixing concepts aren’t verbalized a lot B ecause t hey’re a pain to explain, they’re very rarely taught in the community like pitching or sampling are As a result, audio makers are left to themselves , with their only learning resource being other’s videos. While I do approve of examples as a learning resource , I be lieve in “ learning to learn ” , and that the process is a lot more efficient if you already have some base knowledge to use as a steppingstone That’s why I wrote this document , to make sentence mixing easier to grasp for beginners and to put labels on some concep t which can facilitate discussions and mutual aid on the subject What to expect After a quick introduction to sentence mixing, I’ll share tips on how to work with sentences and voice Then I’ll talk about more abstract concepts like the different “ styles ” of sentence mixing and what they ’ re used for, t his section will showcase and analyze real audio examples 1 to illustrate my points. Throughout the document, I’ll use the words “sentence mixing” and “sentencing” interchangeably. 1 Any overrepresentation of certain Otomad creators is purely intentional and is the author ’ s way of imposing his tastes on the reader, complaints shall be addressed to and ignored by the author 3 What’s sentence mixing ? Sentence mixing is the concept of timing/pitching voice to your BGM 2 P urposes of sentence mixing Singing Sung sections have mul tiple roles. They can be the climax of your audio or be there through all of it. Singing can add meanin g and emotion. When making a singing track , l ess work is given to intuition as the timing, pitch, and lyrics, are already dictated by the BGM (if you choose to follow it) , however, more work needs to be done on processing the (often uncooperative) voice to reach the desired sound There are different workflows (UTAU , Kerovee , Vocalshifter, manual pitching, etc) , and I won’t cover any of them here , but some elements discussed here will still be useful in timing your lyrics before you get to the pitch - shifting part Rhythm Rhythm makes your audios catchier and more memorable. Sente nce mixing creates rhythm based on the flow of syllables Good rhythm will follow or even complement your BGM. Uniqueness Sentenc e mixing played an undeniable role in defining Otomads as what they are , most videos in NicoNico’s all - time rankings ha ve some element of sentencing. Humor Humor is another way to make your works stick in people’s heads. Options include turning the dialog of your source ma terial into dirty jokes, halting the rhythm at awkward places, etc. There are also niche methods such as referencing other works or BGM PVs in clever ways, messing with the pitch (vibrato + wave warp FTW ), breaking expectations set by the fad you’re working with, etc. Don’t try to explain an Otomad joke to someone , if you accidentally start doing it, drop the subject immediately It’s like how explaining the joke kills it except you must explain every cross - reference, you’ll feel the awkward ness settle in and your interlocutor doesn’t care anyway. T hey don ’ t need to care and should not care. 2 If you got confused by th at last sentence, you probably need to stop rea ding here and get more acquittanced with audio making before in general before resuming 4 Sentence mixing basics The first step toward great sentence mixing is being able to do mediocre sentence mixing , while it might sound sarcastic , senten ce mixing that i s just “okay ” is a genuine good start This section will be all about rhythm, and the many pitfalls that can ruin it To make the most of it , you should be familiar with manipulating tracks and items in your DAW (cutting, stretching, moving ) Basic music vocabulary (BPM, beats, measures) will also be useful Get in the good mindset, think syllables As basic as it sounds , it i s still important to know that you ’ ll be working with Syllables . This frame of mind allows you to have a ment al image of your timing that i s closer to reality You should not cut by - word when making sentence mixing! While your items may look fine , syllables tell a wholly different story. Timing by - word is not precise enough, if you do it, the result will sound a wkward because syllables often aren’t evenly timed in normal speech By thinking “syllables first” and arranging the timing that way, you have more precise control over the rhythm of your sentencing. Do note that y ou must still take the words into account to preserve meaning, but instead of thinking of them as “ words ” , you can see them as “ group s of syllables ” 5 Soft and hard consonants This is a concept I’ve learnt thanks to Mikami’s Utau Tutorial The idea is simple but not being aware of it can ruin the timing of your audio Soft consonants are drawn out (m, n, sh, s, ...) while hard consonants sound percussive and shorter (t, k, p, b, ...). Let’s look at the impact consonants can have : You’ve timed all your syllables correctly, but the audio still sounds off, why? The reason is that, for timing, the start of vowels is what matters . As a result, soft consonants can sometime delay the vowel to a point where timing suffers The “s” and “sh” sounds are regular culprits, but this can happen with any o ther soft conson ant In this example, the “ s ” ta kes such a big po rtion of its syllable that the listener truly “ registers ” the sound 1/4 th of a beat late Solution : stretch or mov e your syllables a bit to the left if they sound off - tim e . You can either crossfade the items (overlap them ) or create a new track to avoid overlaps. “ so ” , “ no ” and “ wa ” have been stretched to the left. “ s o ” and “ wa ” are given the ir own track And don’t overdo it. Key words being: “ IF they sound off - time ”. There are cases where the timing will be right without any correction, in such cases, leave it as it is Don’t waste time reading waveforms to get every vowel’s start perfectly aligned , it might make it sound worse. Obviously, none of this even needs to be considered with hard consonants. 6 Stretching your syllables Y ou need to be careful about how you stretch syllables, as once again , consonants are here to make your life miserable. When stretched, soft consonants can sound bad and hard consonants will sound terrible , so make sure you only stretch the vowel sound Monophonic : the monophonic algor ithm can improve how voice sounds when stretched N on - stretchable cases : In spoken Japanese, t he vowel part of syllables like “shi” , “su” or “tsu” , can be silent These shouldn ’ t be stretc hed, but still be treated as a syllable when timing your sentence s About gaps Gaps will sometimes show up between sentences, words, or syllables. Filling gaps Gaps between syllables can be filled by stretching the previous syllabl e. Gaps between words and sentences might be bigger, if so , you can fill them by repeat ing the first syllable of the next word ... Or not Truth is. Gaps don’t necessarily have to be filled; people who are already experienced with sentence mixing might find the example above boring if they heard it By adding some gaps on purpose , you can emphasize the importance of certain syllables and add some variety to your audio’s rhythm. Th is will sound more interestin g I f you ’ ve got breathing room, try moving syllables around 3 (see the “ to ” ) 3 Look up “ syncopation ” for more details , I don ’ t have the knowl edge to discuss it confidently yet 7 Summary Before wrapping this section up, let’s quickly review what we’ve learnt : - Cut by syllables - M ake sure soft consonants aren’t throwing off your rhythm - If you stretch, only stretch the vowel part - Know when to fill gaps and when to keep them W e can now move on to more advanced ideas, in the following sections , we ’ ll talk style o ver method and abstract over technical Advanced sentence mixing concepts As you get used to manipulating syllables, you ’ ll notice that just knowing that isn ’ t enough to make sentence mixing If at this point you still feel unsatisfie d with every thing you try to make, it might come from a lack of direction. Knowing the different wa ys you can do sentence mixing , and what effect s each of them c an achieve, is the next step in your learning process . It will help you t o get a sense of what style best fits a particular section of a song. The styles of sentence mixing The way sentencing is done can vary from person to person , video to video, and even scene to scene Through diversified and attentive listening, you ’ ll learn to pick up on reoccurring characteristics in many v ide os , by g rouping them up, you can build mental categories , you can then focus on r eplicating the ones you like best. Meaning - focused sentence mixing This is when you commit to using fully structured and meaningful sentences. You can use it when you want to tell a story out of love for the source material. Characteristics - W orks with songs and music alike - There is continuity between the different sentences. - The rhythm ha s the right amount of unpredictability ( enough to be d iverse, not enough to be distracting) - Can stick with the BGM ’s lead (or lyrics) or go at its own unique pace o For the latter, it means you must come up with a good rhythm on your own. 8 Examples The se examples will show how Otomad creators add small variations to their sentencing. First is an Uma Musume Otomad by Kagerio The sentences used in this video are only altered in rhythm , and there is a continuity in meaning Syllables are timed in a consistent rhythm, with occasional variations. Taking a closer look at 00:48 , we’ll see that m ost syllables are the same length (1/4 th of a beat). In this case, the variations are dictated by the dialog itself: syllables are being elongated ( ― = ~ ) and put together (like the “ ni n ” ) Here, most syllables fit naturally. W ithout additional steps required. Our second example is Zetsubou ’s Telecaster Bboy video, using Sayonara Zetsubou Sensei The sentencing in this one is a lot less consistent, due to th is anime ’ s frequent tirades and the song ’ s fast tempo; syllables are a lot more crammed together. D epen ding on the song, th is pacing may sound good, otherwise, you ’ ll need to consider el iminating some sentences to get more space. L ooking at 00:32 , t he shortest syllables are still 1/4 th of a beat The gaps and the composed syllables ( “ sen ” , sei ” , “ ren ” , “ ai ” ) help in making the pacing more digest They give the listener time to catch up on the meaning 4 4 It also helps that the faster syllables are common words that don ’ t take a lot of time to mentally process : 私 、 した 、 ありですか 9 More examples: - 魔女っぽいな (spoiler warning) by GainA - サーバント × ウガルル by Sakurei2015 - 塵紙人間 by ケツからスパゲッテ - ピクニック日野茜 by 芋タルト - Shinonome DISCO by beat_shobon Catchphrase - focused sentence mixing “A group of words ha s a nice rhythm, so you just use it , and use it , you keep using it, repeating it in every combination possible, because it just sounds so catchy.” - No one Creativity is enhanced by limitations, and Otomads are no exception. Having less to work with pushes you to try more things , a nd when meaning becomes secondary , you become more likely to explore the limits of your source material This style of sentencing ca n make your audio into a real earworm for everyone else Characteristics - Multisource becomes easier to do as you don ’ t have to worry about continuity. o On the othe r end of th e s pectrum, it can be limited to 1 or 2 sentences repeated to the point where they don ’ t feel like words anymore - There’s more repetition to make the most of your source , parts that sound worse can be discarded since the sentences don’t need to make sense - Works bett er with instrumental s o It can follow the main melody or the percussion ’ s rhythm. o It can be pitched like a lead, and even entirely replace it o It can do both at the same time with two different tracks. Examples - havent uploaded in a while huh cuties? by Quality Control - ひなたのツッコミ by 豊臣秀吉の埋蔵金 o Precisely follows the lead and drums ( 00:42 ) of the music. - ダーリンがきたに ょ by お皿タウン さ o Fast - paced syl lables, almost ignoring the original song - リンゴの音 MAD by Sakurei2015 o Constant l y swi tc hes between sources relating to fruits - テイク 2 by 不覚 暁 o Follows the rhythm of the lead but fo cuses more on arranging the se ntence in every possible order than on the lead ’ s musical aspects 10 Going even further with voice mixing 5 When even word s ’ completeness is considered optional, you end up with a style of sentence mixing that’s entirely focused on rhythm It can be found in regular songs like PSYQUII’s your voice so... (at 0:22) or t+pazolite’s Chrome VOX Charact eristics - Can be heavily driven by the lead and perc ussions o May even surp ass the lead and add more to it - M akes use of everything : words, yells, gasps, impacts, explosions, sfx, etc. - Harder to do multisource with o T he listener can feel overloaded when source switches every syllable. o Finding sources that flow well together get s harder the more they switch around. E xamples - ENERGY 酔拳 MATRIX by Sakurei2015 o 00:13 start of the voice mixing o 00:25 an extra layer of sentencing is added to the previous one, driven by the BGM o 00:49 panned backing tracks - 恋愛サーキュレーションでスーパーボンバーマン R モリモリスター by Sakurei2015 o Syllables are covering the lead melody. o The sentencing has no discernable meaning, it ’ s made to sound nice 5 “Voice mixing” isn’t a recognized audio - making term and only serves illustrative purposes. I’m not responsible for any embarrassment that you might encounter if you try to use this term in discussions with audio - savvy people. 11 Utter chaos T his style consists of overload ing THE FUCK OUT of your listener. A pow erful love letter to Otomads that makes for a great collab finale or a short, supercharged part. Characteristics - Switches sources non - stop o Despite that, can be relatively easy to keep up with if you know the sources. ▪ Then again, it can be pure chaos even for regular Otomad fans. - C an be an all - stars video or use many different scenes from a single source. - W ill get you comment s that say “ drugs ” 6 E xamples - BEAT - NICONICO - WORLD by namacream - ヤバい合作 ED by 阿保草 o Credit part featuring source s from all previous parts , goes at a fast but regular pace (1 source per beat). ▪ Because each scene ha s a varying number of syllables, this video is a good case study of “how can I fit X syllables in a set timeframe ” o The lead sample also changes every measure , adding to the chaos. - 音 MAD 合作晒しイベント OP by Sakurei 2015 and Y. むるか .S - CAMELLO MULTIVERSE by camel y tpmv Mixing styles together If you’ve watched Otomads for a long time, you’ll quickly notice that it’s common for more than one style to be used at once. Here’s some general advice when you do this. - Aim for 1 meaning - focused track at most o Two or more legible sentences playe d at the same time will immediately turn intelligible. - You don’t need that many track s at once for an audio that sounds full. o While sentencing can be a bunch of syllables without meaning, the sound they make is still more complex than regular samples, so y ou need less track s for “complete - sounding sentencing” than for a “complete - sounding no - BGM ” - Unless you’re aiming for an “in your face” kind of feeling, you should introduce/switch the sentencing tracks one or two at a time o Adding complexity over time is a great way to make a long (> 1 min 30s) and repetitive BGM feel less repetitive. 6 More than usual I mean 12 Examples - 恋愛サーキュレーションでだいじょばない by Sakurei2015 o All the tracks in this video use sentence mixing : Although pitched , it’s all syllables and no samples. o It builds up over time: no more than 2 new tracks are added or switched at a time. o The lyrics are mixed in, but don’t have any other purpose than sounding good ▪ This is a case full sentences that are NOT focused on meaning o Here is a complete breakdown of the audio at 01:01 ▪ 4 tracks of sentence mixing are all it took to make the audio borderline chaotic - StarrySky feat.Fuura Kafuka by 絶望 o 01:31 : Zetsubou stacks the sentence mixing part from the very beginning video with the one at 00:31 ▪ In this case, 2 is the limit . With another track, the result would become too chaotic. This is because the sentence mixing for both tracks is more complex. - ただいまーっ！！！ b y 七枚 o 01:39 : A few track s are playing at once, most of them would sound empty if isolated (like the bottom right one). ▪ 4 measures (8 seconds) later, the “yabai” track gets switched for the “yuuki” one. ▪ 2 measures later , the “oya wa henna...” track is added, since the previous “yuuk i” track wouldn’t conflict too much, it doesn’t get removed, it just gets its volume lowered. ▪ Every 4 measures, The “yo” yells help make the transition smoother. 13 Categorization and blurring the line Although I presented each style as its own thing, you might feel that some examples fit many styles at once. That’s because sentence mixing “styles” don’t exist, there are no agreed - upon rules in the community to categorize them. The styles presented are more like rough guidelines o n what you can do to achieve different kinds of results. - If you want to make a touching video on an anime you love, you ’ll want to showcase scenes from across the whole season and include emotional dialogue. Because meaning would be so important, you migh t want to aim for meaning - focused sentencing - Doing a Kenshin video? Catchphrase - focused sentencing would work very well with the many dubs the source ha s. - Doing a haunted dance video? Catchphrase - focused and utter chaos are both very good fits. While I am confident that the categories presented can be a good recipe, you shouldn’t feel limited by them. If you feel confident that it’ll make your audio better, you can take some liberties and deviate from this document’s teachings as you please. We ’ ll wrap th ings up in the next sect ion C onclusion Throughout this document, we’ve first learnt what sentence mixing is and what its use cases are. Then w e’ve looked at the benefits of decomposing sentences into syllables and what the basics of working with them are. Finally, we touched on more abstract sentence mixing knowledge by categorizing example videos based on common characteristics, we saw how these categories affected the listener’s experience and how you can use them as a starting point as to where you want to take your sentence mixin g In some examples, I showed diagrams of how the syllables were timed . You should be aware that when it comes to pauses and variations in the rhythm, it heav ily depends on the BGM and your personal style. There is no single answer for how to time your sentence mixing, your “sense of timing” will develop with practice and experience That ’ s about everything I had to say about sentence mixing Thanks for reading all this, I hope you’ve found it helpful, informative, or a t least entertaining. If you have more questions on the subject or want to share information that you think might be useful in this document, you can contact me on discord at CH FR#4710 14