|Crypton Future Media, Inc.
Offices in Sapporo, Hokkaido. Main business: import and sales of sound-effect materials. Currently, in addition to software sound development, import and sales, the company engages also in software development and sales of products like Hatsune Miku and supply of products and services to creators.
The software and character name created by Crypton Future Media, Inc. in 2007 using the Vocaloid voice synthesizing technology developed by Yamaha Corporation. The voice data is based on the voice of the voice actress Saki Fujita. To give greater reality to the vocals, a body was created with the image of a 16-year-old girl, height 158 cm and weight of 48 kg, with the original animation drawings by the illustrator KEI. With Crypton as the owner of the copyright a PIAPRO CHARACTER LICENSE was created, and with widespread recognition of the actively created and released secondary products, the Hatsune Miku software spread as a new creative platform that many amateur musicians used to create and publish Hatsune Miku songs. This led to the birth of many hit songs and other manifestations that garnered a big influence in the music scene. Also, many derivative characters have been created by the software’s users.
illustration by KEI © Crypton Future Media, INC. www.piapro.net
*1 Nico Nico Douga
This video site was started on an experimental basis in December 2006. Unlike YouTube, in addition to the user-uploaded videos, it offers a lot of original contents from the site operators such as live broadcasts. One of the site’s unique features is that users can overlay the video window with their own comments in real-time (called bullet comments), and this feature creates a sense of community among the users not seen on other video sites.
Launched in May of 1999, 2channel became Japan’s largest textboard. Management problems led to a name change to 5channel from 2017. Its large international influence has led to the creation of sites like the 4chan image board website in the U.S.
This Konami arcade game first appeared in December 1997. It lets the player take the role of a club DJ. Its popularity led to the release of a version home video game consoles.
*4 DTM software
This is a computer music software referred to as Desk Top Music software in Japan.
This is the dance company founded by Setsuko Yamada, who had studied improvisational Butoh as a member of Akira Kasai’s Butoh research group Tenshikan.
*6 Christophe Charle
Born in France in 1964. Professor in the Department of Video at Musashino Art University. He is a noted researcher on media art, network art, sound art and video art.
*7 FM synthesis (frequency modulation synthesis)
Sound created by the tonal synthesis method using frequency modulation. This makes possible complex synthesizer harmonic components not possible with an analog synthesizer.
|*8 Hatsune Miku live concerts
The first time Miku appeared at a live concert venue at the Animilo Summer Live 2009, an anime-related artist festival held over the two days of August 22 and 23, 2009. However, at this event it was in regular video form. It wasn’t until the “Miku Fes ’09 Summer” held on the 31st of the same month to celebrate the 2nd anniversary of the launch of Hatsne Miku that the type of projection from behind on a transparent screen as used today to achieve a greater surrounding effect was adopted. From 2013, the MAGICAL MIRAI event has been held annually to introduce the creative culture based on Hatsune Miku.
Yahoo! JAPAN Hack Day 10th Anniv. (Dec. 2017)
© Crypton Future Media, INC. www.piapro.net
*9 Sega and Hatsune Miku
Following the collaboration by Sega and Crypton on the Virtua Fighter 5R game released in 2008, the rhythm game Hatsune Miku -Project DIVA- was released in July 2009. Now there is a popular ongoing series of products. The 2009 Miku live concert used Sega CG technology (technology from the globally popular Virtua Fighter fight game).
*10 Masahiko Satoh meets Hatsune Miku
Jazz pianist Masahiko Satoh performed songs written by Isao Tomita for anime such as Nella the Princess Knight (or, Knight of Ribbons) and Jungle Emperor Leo with Hatsune Miku or Kasane Teto singing the vocals. Band members included Shinichi Kato (bass), Hiroshi Murakami (drums) and Yoichi Okabe (percussion).
*11 Improvisation Technologies
As the first use of electronic media in the history of dance, William Forsythe created “Improvisation Technologies” as an educational resource that analyzed and systematized his language of dance movement with Forsythe himself showing the movements and explaining them. In 1994 it was published as a multimedia CD, and in 1999 a simplified version was released on CD-ROM.
Yu Higuchi created this software. In February of 2008 it was released as freeware in 3DCG software form. Originally it was a tool that Higuchi developed to create his own Miku CD videos and it enabled even beginners to quickly create their own 3DCG character videos on a low-spec personal computer. Thus it became a tool that led to the rapid spread of Miku videos.
A name combining the “vo” of voice and coder. It is an audio compression technology for vocal sound transmission, and the devices involved.
*14 Kenshi Yonezu
From 2008, Yonezu created many Hatsune Miku videos under the name HACHI that he uploaded on the Nico Nico Douga site. He created a large number of hit like MATORYOSHKA and Panda Hero that ran up over a million plays. In 2013, he made his major debut as a musician under the name Kenshi Yonezu. His song with a Kitsune no Yomeiri style video is Musunde Hiraite Rasetsu to Mukuro released under the name HACHI.
*15 Senbonzakura (One Thousand Cherry Trees) feat. Hatsune Miku
This video with lyrics, music and editing by Kurousa P was released on Nico Nico Douga in 2011.It was the third video on Nico Nico Douga to run up more than 10 million plays, following ♪Miku Miku ni Shite Ageru♪, Shiteyanyo and Melt. It also became the 3rd most played Karaoke songs of the year in 2012, thus making it a hit beyond the Net platform as well. Many cover versions were also released by Japanese traditional instrument bands and singers like Sachiko Kobayashi.
May. 16, 2018
|Wataru Sasaki (born 1979) of Crypton Future Media, Inc.* is the creator of the virtual singer software “Hatsune Miku.” Since it was released 2007, a new creative culture has swept the Internet as huge numbers of users began uploading new Hatsune Miku songs and videos they created with it. This new culture has spread with the birth of various types of collaborative productions, including even concerts where fans can go to hear Hatsune Miku perform “live” on stage, and to disseminate this creative culture the “MAGICAL MIRAI event has been held annually since 2013. In 2017 it drew a largest-ever number of visitors at 30,000. In this full-length interview, Sasaki talks about the new future that digital technology like this can open up in the performing arts, and also his personal vision that extends to genres like Butoh.
Interviewer: Hisashi Yamana [Institute For the Arts (IFA)]
Background to the birth of the virtual singer software
Among professionals in the performing arts, there must be many, especially overseas, who don’t have any clear idea what Hatsune Miku is. Would you begin by giving us a simple introduction to Hatsune Miku?
The Hatsune Miku software product was first released at the end of August in 2007. It was at the time when the sites like YouTube and Nico Nico Douga (*1)
that would later be the stage for Miku videos were becoming popular, but there wasn’t much in the way of contents yet on these sites. That led to a movement to create contents that the users could use to create their own videos. One of the central platforms for this activity was Net bulletin boards like 2channel (*2)
where people and programmers who liked digital tools gathered and people who like to do things like create music on visual art and music bulletin boards like Illustrator and beatmania (*3)
. At first there were mainly interesting videos being uploaded on these sites, but once the Hatsune Miku software was released, these people started uploading videos that used it to have the character sing theme songs from anime cartoons or popular songs. Amateur creators like these then gradually began to gather on the Nico Nico Douga site.
Until Hatsune Miku’s release there hadn’t been a virtual singer software, so people who had written melodies or lyrics in hopes of someday getting a singer to sing them had nothing to do but keep them on file. But when they heard that you could get attention by using Hatsune Miku to sing the songs they wrote and uploading it on Nico Nico Douga they decided to try uploading their creations using our software. That is why so many users began using it.
Probably there are many people who don’t know how you make a video using the Hatsune Miku software, so I will explain it quickly. With the software you use your computer’s mouse to input the notes and lyrics. That gives you the Miku vocal data and then you generate the audio data and load it into the music composition software [DTM software (*4)
] and mix it the Karaoke recording or the music you have composed yourself. Then you have your song done. Then you write it onto a still image or video image in a MP4 or Flash video format that you can then upload onto Nico Nico Douga or YouTube.
The Hatsune Miku software uses the Vocaloid voice synthesizing technology developed by Yamaha Corporation, and it was combined with the Hatsune Miku illustration and name designed by Crypton in a joint development project with Yamaha to create the software that we sell today. By the way, Crypton had been doing test projects with Yamaha from the early 2000s using the DAISY software that was the forerunner to Vocaloid, but there was the problem that voice synthesizing technology alone was not something that could attract a broad user base. So we thought that if we gave the voice a body and marketed it like an anime character it would be easier to get people to use it.
There doesn’t seem to be much of a connection between people who are making videos to upload on the Web or hanging out on 2channel and people who are active as musicians.
Rather than musicians who play a musical instrument, Hatsune Miku users tended to be programming oriented people of the type that make ring tones or MIDI (musical instrument digital interface) programs. But somehow, there is a connection between them and people who are regular musicians, and that is probably why Hatsune Miku became popular among musicians as well. Also, a lot of Hatsune Miku users were stay-at-home recluses. Often it was a case of people finding Miku by chance on the Net and then coming to play with the software for personal amusement. Also, I believe there were many cases of wannabe musicians who were working at part-time jobs to support themselves while applying to competitions like SMAP and Hello! Project or people actually working as studio musicians in the music industry who wanted to test their abilities to become known.
Wataru Sasaki and the Hatsune Miku character
When did you join Crypton Future Media?
It was in 2005, two years before the release of Hatsune Miku. I liked things like electronica music and sampling collage, so when I was in high school I went to Crypton with a friend several times to buy sound data. That friend worked part-time at Crypton for about ten years, so I also started helping out there too.
I always liked avant-garde things, and I also in my late teens watched things like contemporary dance where the dancer Yukiko Amano of the dance company BIWAKEI (*5)
did sessions with an android doll using sound data created by Christophe Charle (*6)
. In other words I was interested in what you might call sound art or sound technology. Vocaloid uses voice synthesis technology derived from research done at Pompeu Fabra University, so it is connected to the acoustics research of France’s INAGRM or IRCAM research institutes that have contributed a lot to the industry in the area of contemporary music. This type of sound art goes back to the contemporary music and electronic music of people like the composer Xenakis. As for me personally, this is the kind of aspect I was interested in more than enjoying music like a musician, so I looked at Vocaloid and Hatsune Miku more as phenomena, with a more detached point of view. This kind of analytical approach is something that I shared with Crypton’s founder Hiroyuki Itoh (CEO of Crypton Future Media, Inc.).
Since Crypton itself was originally a sound data company, this all seems natural. There would seem to be a strong interest in sound research at the intellectual level.
I think that is true. At the time, people of the generation liked the music of popular artists like Tetsuya Komuro and Ryuichi Sakamoto of the Avex or Sony Music labels who liked to use synthesizers, and I myself was interested in things that stretched to boundaries of music itself to new limits. At the same time, I was listening to the music of David Tudor and of Ryoji Ikeda, who was a member of Dumb Type. Before that there was AACM and Fluxus and the places that such creator as Dumb Type gathered to do their creative work such as Rhizomatiks Co., Ltd. and TeamLab Inc., and I was interested in the way those companies functioned.
How did you get involved working with Vocaloid?
At the time there wasn’t a department of that type in Crypton, so there was no one to do that type of work, so I volunteered to give it a try. And because that was the situation, I was able to work with relative freedom from the outset.
From there, the character Hatsune Miku was born, but what was it that gave you the idea to create such a character?
To make a Vocaloid, you begin by recording a human voice and then you have to piece them together in a patchwork. It would sound strange if I used the term the voice of a dead person, but what you have to do is to peel away the parts that sound like they used to be human. Like the dry contents of a cup of instant noodles, you have to get it down to something that sounds dry, without a sense of the living flesh and blood. In other words you get it down to something simple that has nothing left of a person’s voice or emotions, and then like adding the hot water to bring the noodles back to their original moist start, you have to put life back into it. And this was all done under the premise that you find some image for the final quality of the voice that you want.
Before we released Hatsune Miku, there were makers in the UK and other English-speaking countries that been involved in speech synthesis, or it could be called Wavetable synthesis, and used it to create characters. But their model was in the direction of a Frankenstein. I thought that we should make a humanoid character, but I didn’t think it should take that horror direction. But making something like a real human is difficult. So, I though that it might be good to develop it in the direction of personification.
Hatsune Miku’s high-pitched voice is actually a sort of homage to FM synthesis (*7)
type synthesizer. As for the nuance Hatsune Miku’s white skin and slender figure, I believe it is like the image of the vital spirit projected by the thin, whitish bodies of some Butoh dancers, which appear to be on the borderline between life and death. Rather than an image in the direction of a ghost or a Frankenstein, my image was that of a light-spirited girl with skin lacking in color and vitality; I wanted to get an image of someone that looked like they were about to die, or like they had just died. Wouldn’t that be a look that would suggest human love? And, if she had the look of someone wandering on the borderline between life and death, I thought she would be a character that people could become attached to? I wanted to get a voice quality that would bring the users to a borderline consciousness where they might say either that it was a voice patched together from human voices or that it was something that had the potential to sing, once they bought the Hatsune Miku software and started to use it.
Other things that I referenced were characters like Aegis in the Persona3 game by ATLAS, or an android of the type that appears in The Five Star Stories manga. The things I referenced were mostly rather slender figures. And since I don’t prefer manga characters that have big breasts and a mix of Lolita qualities and maternal qualities, I wanted an impression that was different from that. I wanted a character setting that didn’t project a clear sense of attention or focus but rather that of a vaguely unfocused young girl.
It seems that the Butoh you like to refer to is not the body of Ankoku Butoh with it language of ritual and the primordial like Dairakudakan, but that of improvisational Butoh like that of Akira Kasai and Setsuko Yamada’s BIWAKEI.
That’s right. If likened to contemporary dance, Miku would never dance in the robust and powerful style of a company like La La La Human Steps. She can only compete base on her posture and the atmosphere of her voice. She is good at dancing with repeated unorganized sequences like to the music of Steve Reich, but she would do very poorly if you tried to make her sing with the full emotional strength and expressiveness that the human voice is capable of. So the feeling is one of pretending to add subculture-like symbols and deliberately avoid the inherent heart of song and emotional expression. Looking back now, I perhaps I gave her little too much of an underground-like attitude that maybe a bit strange (laughs).
What about the image of the Hatsune Miku voice?
In terms of a Japanese singer, I felt that the direction should be something like that of Tomoyo Harada with her combination of appeal as an idol but also being a bad example as a singer with her flat, non-vibrato style singing voice. I felt a special aversion to adding vibrato to a Vocaloid voice, so I wanted to keep a distance from the kind of singing like in R&B and Japan’s enka
popular folk songs where a vibrato is necessary to give it a gritty, soulful flavor. At the time I liked the songs of the musician Nobukazu Takemura sung by Kiku-san and the kind of innocent nuance that connected to the songs of Robert Wyatt, and I thought that a somewhat idyllic image like that rather strong self-expression would fit Hatsune Miku better.
The “sick quality” that gives reality to a Vocaloid
One of the representative pieces of the early period was “Hatsune Miku no Shoushitsu (Dead End)” (By: cosMo@BousouP). Do you think that that was the kind of piece that honestly fit Miku’s expressive potential well?
Personally, I believe that that song with its use of the high voice at a machine-gun sixteenth note pace was a skillful use of the Vocaloid potential for expression in a sort of sporty way. If I start talking about the significance of the high voice I could go on and on, but the fact is that at the time in Japan’s pop music scene in the 1990s, Tetsuya Komuro was very successful at bringing out the appeal of a female vocalist’s high tones, but in fact it presented difficulties for the singers themselves. But, since Miku is good at high notes and doesn’t sound good in the low range, the feeling is unpleasant when you lower her voice too much. With that high voice, at the same time it becomes less human and starts to sound more like a musical instrument, which I believe gives Miku a suitable individuality to her sound.
After Miku you created a series of virtual singers in Kagamine Rin, Kagamine Ren and Megurine Ruka. How did you differentiate them from Hatsune Miku?
We had been developing Rin and Ren almost at the same time as Miku, and the feeling was that even though we changed the quality of their voices, their style of singing remained basically the same. So, rather than tuning them like different instruments, the main focus was on differentiating them as characters. Rin and Ren are a girl-boy pair and Ruka is bilingual, so we gave them completely different orientations, which created a differentiation in terms of the fans’ impression, but in fact that was a rather dangerous thing to do. If we had known from the start that the Vocaloid voice quality would remain basically the same in terms of direction, I don’t know if we would have embarked on such a project.
If a Vocaloid is too high or too low, it gives a slightly sick-sounding quality to the voice I believe. That is not a complaint but something similar I feel to what takes place in the artistic expression in Butoh or contemporary dance of the sides of a human being that are normally hidden, such as abnormal individual fetishism or sickness so it feels like you are seeing something very private. The reason I think people accept Miku is because of that appeal of the feeling that she is exposing those sides of herself, and I believe that a certain percent of the fans feel an affinity to that state she seems to reveal.
Something that is perhaps related to this feeling of sickness is that everyone writes the lyrics to the songs in the videos in very big letters (characters). What’s more, the lyrics are things like Kanzakiiori’s Inochi ni Kirawareteiru (Life hates me). Many of the songs use incessant verbomania to express a state of being psychologically overwhelmed. It is completely different from J-POP.
I think it is a question of what the things being expressed are bound by. For example, if it is a regular singer, they are bound by the desire to keep singing successfully, to have more people hear their songs, to become famous. And, those who have experienced fame all take on the same kind of airs, have the same smell of success. However, from the perspective of many of today’s young people who are disillusioned with contemporary society and my in fact be stay-at-home recluses and have experienced the undoable irrationality of their lives, when they see these “successful” singers they are likely to wonder what they are actually doing behind the scenes, so it is not easy for them to feel an affinity for them. But outside this circle of success there is the doll Miku with her unique voice and her Vocaloid songs. Since the Vocaloid is not bound by the chains of human desires, they are in a place where they can sing ephemeral songs and songs that express the trembling of the emotions of a heart that knows its own weakness. It appears that these songs about the other side of the human soul touch the hearts of these young people. But in fact this is a very normal phenomenon and it think it is as inevitable as the fact that water always flows downhill. In the end, it is easy for a Vocaloid to commit to the world outside the accepted social framework, in a place where the people there can enjoy their own music unrelated to the commercially based music industry. I feel this is close to the avant-garde anti-pops music and noise music that I discovered when I was young. Noise music had an outsider feeling to it, and from our side, I believe that, rather than dancing to the tune of the trendy dramas on TV and listening to the popular songs of Dreams Come True while having convenient romances, for those of us who didn’t believe in politics or love games, it was a more ideal reality for me to be listening to noise music. It is because of this desire to listen to real music and the fact that people are willing to go to extremes in pursuit of it that music continues to evolve. That is the side I belong on and I love the very feeling of seeking it out and that is where my ideal image of essence is.
So you believe that alternative movements emerge as a result of the natural processes of society?
Yes. In times like these when the buildings built in Japan’s era of high economic growth rare now deteriorating, and an increasing number of rural communities are unable to make the transition to the next generation, the things people long believed in are gradually crumbling down, I believe that we are facing new realities that can’t be glossed over with the types of products designed in the image of, “Let’s bet on the future, let’s have fun, let’s join in this happiness together.” In that sense, isn’t it not just Japan but the whole world that is heading in the direction of alternatives? Already, there is an increase in unique Millennium-generation music, and there will continue to be new conscious arising and new music like we have never heard before emerging in the next 20 years. I hope that Vocaloid will be used increasingly in the music of this era.
In fact, the foreign artists that confirm that they like Vocaloid tend to be very progressive artists, and many of them use their art in part to give voice to their gender identity disorder. Considering this and other factors, I believe it shows that this music that enables people to connect because of the Internet is real. But not in the sense of a recognition of the reality of one’s actual situation as seen in such acts as posting a picture of oneself for others to see, but rather real in the sense of something close to a fully digital world of virtual reality in which mechanical sound is prepared along with a pitch-controllable voice in a way that you can’t distinguish between the synthesizer-generated sound and live sound. With a consciousness of this, I believe that a field of expression that until now was difficult for people of the next older generation to feel will continue to spread on the Internet.
The Net offers points of access to ways of looking at things that are different from the usual viewpoints, and as they continue to swipe a tablet, children encounter these things. I think this gives people an opportunity to escape from the constraints of their region or the everyday, and I think that is the side that Miku is on.
Although Miku was born and raised in the world of the Internet where you can be transported instantly by a single swipe, she is now also giving live performances in the real world.
The Miku live (*8)
concerts are something that came about due to requests from various companies, such as Sega (*9)
that created the Hatsune Miku game. By creating live concerts in which Miku could appear in a very easy to understand way, it has become easier to explain what Miku is and to see the processes of show business and the ground on which Vocaloid can continue to function. Like the Sega game, the live concerts have an aspect of functioning as a place to introduce the place where Miku can be introduced.
When the Miku live concerts first started to become popular, I realized we have got to do this right, I have got to do it, so it was a very serious attitude we brought to doing the job of a concert organizer properly. It was a situation where, although she [Miku] wasn’t actually there, we had to communicate the fact that all the people involved were there, and since the real stars were the creators that had made Miku’s song that became its on the Internet, we realized things like the need to communicate more with these creators. There were actually quite a number of people involved in managing the borderline between the virtual and the real, and I was feeling the heat generated by all of their diverse feelings, being projected around us like the images of a revolving lantern.
A central part of the flow that builds the excitement focused on the popular song creators on the Net is the rush of comment postings expressing viewers’ feelings whenever one of their new songs is uploaded onto the Nico Nico Douga site. The comments say things like how moved they were by the song, or how it inspired new motivation in their lives. Some of the comments come from overseas and some from small children. This kind of pressure in a positive sense, this roar in the world of the Internet includes people who say they have found something new, found satisfaction and fulfillment and a sense of being alive, but there are also people who express their disdain. About five years ago, this crossover of feelings from a diverse group of people was pretty amazing.
Is it a question of the pluses and minuses of the boom phenomenon?
The number of replays on the Net is something that the creators and the fans and a lot of people are focused on. Although the contents of the video or the depth of its effect on the fans or their passion are not necessarily reflected in the number of plays, we nonetheless tend to rely on the numbers as an index. However, when we look at the number of plays we find that there are very few creators who continue to create hit songs/videos. And in that sense, the decrease in the number of plays has caused many people to do fewer uploads. This is very unfortunate. At one time, when hit songs like Senbonzakura
(One Thousand Cherry Trees) or The Kagerou Project
appeared, the impression of Miku and Vocaloid popularity would directly be boosted by them. But, rather than those hits becoming symbols, I believe each hit song sparked new analysis of the elements involved and repeated subdivision of the elements.
As the momentum of the Nico Nico Douga website has faded, the Miku boom also seemed to have subsided, and we were told things like, “Hatsune Miku is finished,” or “Hatsune Miku is dead,” and when we compare the before and after, I believe that it can indeed be said to be true. But for me, I believe Miku is still the subject of random access in a mixed-up time order, so in one sense she has become neither old nor new. We can also say that in anther sense she has become more easily accessible and taken for granted. My impression is that in not only the Internet but in media extending to the commercial live concerts, she continues to be supported by demand that comes out naturally from the fans, the listeners and the users. There are songs like the earlier mentioned Inochi ni Kirawareteiru
(Life hates me) that are not suited for the live concerts but are more introverted and appreciated quietly, and I think it is wonderful that it has become natural for people to seek them and accept them. So, I think the Miku is not finished, it is just that she has something very diverse in areas and contexts like these.
Collaborations in live shows
Miku has done live concert and show collaborations with the drum performance group Kodo, with the NHK Symphony Orchestra, with Isao Tomita, Keiichiro Shibuya and Super Kabuki. Are there any of these productions that that you planned and directed?
I wasn’t involved in the planning of any of these you have just mentioned. Until one point the music of Miku was almost exclusively Nico Nico Douga’s music, and we can’t tie up with any other categories without some new outside force coming into play. Besides the shows you have just mentioned, people also doing planning work like Kenzo Saeki-san and Bajune Tobeta have made strategic use of Miku, and this happened at a time when we couldn’t even think of joining as project leaders. In recent years, people like Big Boi of the American hip hop unit OutKast and the progressive new British sound artist Laurel Halo have used Miku. We would like to be able to answer the demands of these kinds of increasingly complex artistic uses, so we are constantly thinking about what kinds of mechanical sounds would please the users. By the way, as an extension of the collaboration with Isao Tomita, there was a new collaboration project launched just after Tomita passed away (*10
: jazz pianist Masahiko Satoh and Miku performing Tomita’s music). This was one in a series of Tomita projects from which CDs were to be made, and it was a rare case in which I made the arrangements and got the cooperation of Masahiko Satoh, who was a fan.
Until now, a number of artists asked us to collaborate with them because of Hatsune Miku’s innovative presence and her connection with Net music, but after ten years there has been a rapid increase in virtual characters, like Vtuber for example. And as the trends of the times have changed, musicians can now do real-time sessions with them, and we feel that we now have to increase our efforts in projects with specific timing and commitment. This is a new issue we face.
The work of sound making
There are some collaborations where Miku has to sing things like classical pieces that aren’t necessarily in her range, are they?
For those I have to do the work of adjusting the sound to fit the style. If it is Japanese folk song I have the folk singer vocal and I analyze the data along the time axis and the wavelength axis and analyze the voice and breath elements and then transfer it to Miku. In other words, the work involves using the human voice as the base and then I translate the quivering-type vibrations into Miku’s voice. The sound that comes out the phenomenon of this process is then broken down into parts and then the parts of the human voice and Miku voice are interchanged and connected together. Then I test the result and make adjustments to improve the balance and such. Right now, I am creating the voice sound for a tie-up with the anime SHINKALION, and the work I am doing now might be described as taking the voice of the voice actor that Miku’s voice is based on and the finished Miku voice and taking them apart and then putting the parts together again in a process that might be likened to fusing them together of repositioning the parts.
That means you can’t just use the voice of the voice actor Fujita-san that served as the sound source for Miku’s voice?
To be precise, it is in fact a case where a person can’t imitate Miku’s voice. When they actually try to do it, many problems such as connecting the breaths arise. Hatsune Miku’s voice was created by process of fixating the voice ingredients that give the “personality” to the voice of voice actor Fujita-san, and in the process of making the Vocaloid, various voice ingredients are removed, so it is not the same. The work I am doing now is to once again add voice ingredients from a number of voice actors and synchronize them part by part and then distribute them to the either the Miku side or the human side and finally balance them. So, it is like creating a mean value between Miku’s voice and the human voice.
In this synthesizing process you can put together ingredients of three people’s voices or four people’s voices as you please. It is like a job of taking apart the voices and examining the parts. And it is not just with the sound element. Since it is words being spoken/sung, we think about leaving the parts involved with the meaning of the words. In short, we are doing digitally what a singer does when the express deep sorrow or deep anger in their voice as they sing. In the work process, we may get the voice actor to act out a part or sing, so it is a process of actually working with people, and in that sense I think it is actually quite progressive.
Since the work you are doing is digital, you can send it, archive it and recreate it.
With the advent of the YouTube era, an increasing number of creators started recording and uploading the entire music creation process onto YouTube. I believe we will now see people selecting parts of the process someone uploaded and the using AI or devices that can piece together the work semi-automatically and then add their own innovations to enable a new creation.
I think of my work as taking apart the mix of expressive elements in the jumble we call human emotional expression and expand them in other directions. By doing this, there is no need to chase after vocal expression that until now one had to train and practice in order to acquire, and instead you I may be able to create vocal expression with new impact from a completely different direction, and in this way, the work of digital music creation can become a form of design-like endeavor. I think there are certainly good and bad aspects to that believe, but since it enables an expanded range of freedom, wouldn’t it be good if it can help people go beyond the rules of composition set by composers of the past and breathe fresh air into the field by introducing new types of artistic expression.
In any case, for people who to do the things they to want without having to spend too much time on training and practice, it won’t be too long in the future before we have the means to learn what is necessary and create things. And when that time comes, I believe it will be led by instruments using things like Vocaloid or other technologies of the kind.
When that day comes, the things that will be quite important for the people on the creator side is their human individuality as measured by their ability to see things from a different perspective and perceptiveness from others, and the sensitivity of their perception. The things that can be described as Depth or keenness. Twenty years from know when the range of vocal expression that Vocaloid is capable of makes the physical capabilities of the human voice sound rather limited, people will be competing to create a wider range of deeper and more eccentric expression by means including synthesis with mechanical devices.
In the end, all sound and the human voice come down the vibration waves in the air. Although we now live in an era where these physical phenomena and human knowledge have already intersected, it seems that no real research is being done about the relationship between the two.
From now on, there will a need to create technology and software that is capable of analyzing vibration itself as a multi-layered phenomenon made up of numerous elements. These technologies don’t exist today but they will certainly be developed in the near future. And knowing that, we realize that today’s music is in a transient state and there are surely changes coming in the future. Now the technology has finally becomes inexpensive enough that people of the general public can use it and it is setting new movements in motion, so I believe that we are going to see some really substantial developments in the future.
What are your thought on technology and human physicality in the arts?
In 1994, William Forsythe released a multimedia CD titled “Improvisation Technologies (*11)
: A Tool for the Analytical Dance Eye” as a guide for learning his language of dance movement. In 2014, when the free CD software “MikuMikuDance (MMD) (*12)
to teach users how to make Miku dance was created, it was a contemporary manifestation of what Forsythe wanted to do, and when I saw it was being used later by people who love CG characters, it was a bit maddening. At the time I felt a distance between the artists and the ideals of the nerds who were using it, but today that distance has collapsed and I believe young people don’t care about such things now.
The small world of sound art is in the same realm as what is now the media art of Hatsune Miku sound art. With this, Sound art, which had until then been a progressive art for dedicated enthusiasts, now became part of the larger circle of Hatsune Miku media causing an intersection with a large number of people. I believe this was a big movement. In addition, the digital media makes it possible to clear away the prerequisite human relationships involved.
Hatsune Miku may appear to be unrelated to the complicated realm of art, but I believe it has naturally grown close to becoming an integral part of it. If you intend to do music, dance or other performing arts, I believe that the effort spent on building human relationships can spoil the effort that should be spent on creative work. Given the fact that cutting relationships with the people around you can make it easier for artists to concentrate on their art and stoically pursue what they believe in without concern for popular trends, I think there is in a sense some validity in the ability of digital means to clear away live human relationships. But it is also not a good thing (laughs). Because it is natural to question whether it is right or not.
For people of our generation, it has become difficult to maintain a minority community like the ones built around the arts of sound art and avant-garde performing arts today. Vocoder (*13)
technology, which can be seen as the predecessor of Vocaloid, was born at Bell Laboratories and was used in wartime coded transmissions, and due to this unique demand and its relationship to the national laboratories, demand from the contemporary music industry has dwindled. But in the world of avant-garde expression, it is digital native technologies like Vocaloid that continue to be evolved. I think this is due the diversity of soundtracks used for applications, the rapid increase in music that can be broadly defined as ambient music (a music genre pioneered by Brian Eno) and the shift to digital formats.
By the way, I believe you lived for a long time in Sapporo. May I ask what kinds of experiences in the arts and cultural led to what you are doing today?
There was a Butoh company named Hoppo Butoh-ha (Butoh of the North) and some of its members moved to Otaru and Sapporo in Hokkaido and some of them continued to perform there when I was young. I don’t know why they went there though (laughs), but I used to go to the establishment where they hung out, and that is how I first encountered Butoh. I was uncomfortable with my surroundings and believe I was looking for a place and people that I could feel something in common with. From there I went on to watch other contemporary dance like Kim Itoh
+ The Glorious Future and Noism
, and I thought the work of other groups like Nibroll
and Baneto that I couldn’t understand at all was really amazing. Looking back now, I think I had frightening interests (laughs).
There were also a good number of things in Sapporo like clubs where people who liked stoic techno music or hip hop. For me it was easy to get into things like that which were so different from the everyday world, so I found myself going to places where people gathered to listen to minimal music like junkies and listen along with them. It was a wonder to me that this kind of music existed and there were people like these who were happy to spend hours listening to it long into the night. “What does this mean?” I wondered (laughs).
Normally people would become bound to something like the Butoh community, but your obsessions extended to a wide variety of genres, didn’t they (laughs)?
Basically I think of myself as someone not suited to belonging to such communities, and just as an annoying, argumentative kid (laughs). Overall, unique forms of expression like Butoh and the others are in a weak position, and I think the issue is what to do about that weakness. And yet, I thought that hanging out at art spaces was also a rather distorted thing to do in a sense. Then I came to think of the Internet as something that could bring about change. A variety of genres were adapting themselves to accommodate the Internet, but I think the performing arts and other avant-garde things have been in a sense one step behind in adapting to the Internet. And since those were the genres that I originally liked, I thought that the digital programs or things like algorithms that geeks created could eventually be used to by people into the avant-garde and minority groups to connect to each other.
Hatsune Miku was able to boom in popularity because it had Nico Nico Douga as a stage to grow on. I would like to ask you now about your thoughts on the future.
I am 38 years old now and I definitely have a feeling that I would like to have people to pass my work on to and let them take over. Because of the moves I have been making, I think there are people in this circle that has gathered who are even more difficult than me, a group that says thinks that come from a perspective that even I can’t understand, and I think they are people I could gradually pass the baton on to. I think that the sense with which post-digital native young people accommodate things mathematical and things technical is changing. It isn’t a physical ability, but this up-coming generation excels in some kind of ability more than me, and I think it might be good if I were to pass the baton on to them and they could change things drastically.
With regard to Nico Nico Douga, it was originally a platform where the gay community and other sexual minorities also gather, and I think it would be good if Miku could continue to hang out there with minorities like that. In that sense, it is the same as the old loft jazz and House music scenes. In the end, regardless of my affinity for minorities, the thing is that it is important to continue to be conscious of minorities and the things they involve. So I am working together with some of the Vocaloid creators who can do that. As I continue to do things avant-garde, I guess I have something of a desire to connect it to forms of pop expression at the same time.
So, you have an image that the avant-garde and pops can intersect?
For example, one of the artists with most momentum in Japan today is Kenshi Yonezu (*14)
, and when he was releasing Hatsune Miku works, he created works like Musunde Hiraite Rasetsu to Mukuro
that were so dark they made Ankuro Butoh look tame. Even the biggest Miku hit of all, Senbonzakura “One Thousand Cherry Trees” (*15)
was satirical with a flavor of Taishō period (1910s, 20s) underground. They both became hits with a style that was unthinkable in J-POP, so I think Miku is a platform that makes it easy for alternative music to win a large following.
What are your thoughts about the future possibilities of Miku collaborations in the performing arts?
I would really like to be able to do stage productions using Miku, and I am now thinking about the possibilities of using Miku on stage. In order to do that, however, we have to be able to lower the cost of a live Miku performance. At present it costs tens of millions of yen to for one live performance. That is the price using assets like the repertory from past live performances. I would like to find a way to lower that cost to hundreds of thousands of yen, and if that isn’t possible I at least want to get it under 300 million yen. Otherwise, it will be too expensive to expect reasonable demand. We are now working on the potential for lowering the price of providing the equipment for the computer graphics, the motion and for synchronizing the song and the motion plus the necessary operator. If we can do that we will then be able to respond positively to requests from people doing stage productions.
If we make it so that people can see Miku close-up in person, it may change people’s perception of her. Due to the power of preconceptions, people can react in strange ways to an entity that is similar to a human being, and it can even cause accidents and injury, and I think it will be interesting to see how people react when they encounter her at point-blank range.
The core of the Miku business now is social network games and merchandising, and if we can reverse the way we invest the money we get from this business and manpower, I want to be able to deepen Miku’s presence, make it more weighty. If we can do that, it may change the live performances and installations the Net creators use her for. That is what I am most interested in of anything for the future.
If you are able to lower the price that way so that underground live performances can be held with her or she can appear on the streets, I think it could lead to another chaotic boom and expand the world for people whose position is weak now. I want to thank you for time and the interesting things you have talked about with us today.