Как устроена система распознавания лиц (и кто ей торгует) -

Please enable JavaScript to view the full PDF

HISTORY NOVEMBER 14, 2018 How the facial recognition system works (and who trades it) All about the technology by which women are poisoned and war crimes are investigated РЕКЛАМА Editor Yulia Ruzmanova You go live at the door of the entrance: old cameras above intercoms in Moscow now rec- ognize faces online, and the policeman re- ceives a push-notice on his smartphone, if the lens gets the right person. The Village tells the full story of the revolu- tionary technology that was first presented to the people and then taken away from them- and clearly explains how this recognition mechanism works. Text: Kirill Rukov, Damir Nigmatullin Illustrations: Ivan Annenkov I What we've done РЕКЛАМА Someone types on the keyboard the headline: TRAIM SKIN, WHAT ARE IN OR WORKING REALLY - so anonymous launches a monstrous flash mob on the forum Two. The first comment in the topic appears in 30 seconds, the first "ex- posure" - in three minutes. The recipe is the same: the girls were found in VKontakte by persons, through the website OF THE FIND‐ FACE.RU. Then the two men sent screenshots of videos to their husbands, relatives, classmates, colleagues and friends. They were bullied, ex- torted money and forced to quit their jobs. "It's nice to see a pully lying and spinning, aki in a frying pan," writes one of the authors of the baiting. "For experienced trolls there is a lot of delicious food, cunning Jews can cash in on whores, and the omes can take revenge on the sluts." The monstrous flash mob lasted two years. Gradually, two men submitted the idea to ordinary maniacs: now just random girls be- gan to take pictures in public toilets, then blackmail. Evil worked like a viral advertise- ment: the FindFace service, "face-recognition," was learned by the masses. Two years later, in the spring of 2018, Moritz Rakuzhitsky sits in front of a laptop and watches as the Chat Slack'a non-stop messages erupt - the investigation team BELLINGCAT re- veals a war crime. In the winter, Moritz was visited by a lawyer, an American representing Ukraine in the Hague court. He suggested that the team "make an examination" of the inci- dent in Mariupol. On the usual sleeping area then fell rockets "Grad" - someone marked in the checkpoint of the Ukrainian army, but missed. Thirty-one people were killed and sev- eral neighborhoods were on fire. Moritz calmly tells The Village how, using the same Findface service, the Bellingcat group identified specific Russian officers who al- legedly gave the order: "Among them was a man named Voronezh, in which we identified Lt. Col. Andrei Grashchenko (name changed). I'm going to do it. Ed.). He was sent to Ukraine for only six months, but shortly before return- ing home "Voronezh" began to call his girl- friend, missed. We knew both numbers, be- cause the Ukrainian authorities gave us an ar- ray of wiretaps,but the phones were "left", to find on them nothing was possible. Then we put the girl's number in one of the messengers, Viber, and then suddenly a profile with an avatar surfaced." The little userpik was driven through FindFace - the service recognized in it user "VKontakte" Tamara (name changed. In the pictures she posed with Grashchenko him- self, then with his car in different cities. "In to- tal, we then calculated eight Russian troops sent to Ukraine," Moritz continues, "in three cases only thanks to FindFace. This will be the first time the Bellingcat investigation will be- come a matter of evidence in The Hague (the UN International Court of Justice has already accepted the materials. The second such case is about the downed Malaysian Boeing MH-17 in Ukraine. There we found through facial recog- nition about 30 contractors who took selfies with the militia. It's a massive method." The second such case is about the downed Malaysian Boeing MH17 in Ukraine. There we found through facial recognition about 30 contrac- tors who took selfies with the mili- tia. It's a massive method. That same spring 2018, the developer of Find- Face, the Russian start-up NTECHLAB, suddenly invested in the state defense corporation Ros- tec together with billionaire Ruben Vardanyan. The infusion turned out to be the largest in the history of the company (even Roman Abramovich and the deputy director of Tele2 invested in it more modestly). Just FindFace celebrated 700,000 visits a month - the "peo- ple's" service steadily paid off, and the authors wanted to expand to "Single Classmates" and Twitter ... Suddenly, in June, the company an- nounces the closure of a public search engine. Panicked fans even organized a collection of signatures against, and speculators with still working (paid) accounts resold the service of finding a person on the photo for several hun- dred rubles. FindFace technology was very popular with the Moscow City Hall. In the Moscow metro, the face recognition zone will soon be expanded from the south to the red line for the entire subway. The NtechLab algorithm has also been connected to 120,000 of the same access cam- eras that we used to consider useless, and are also successfully sold to shopping malls and supermarkets (for targeted advertising), casi- nos and banks (for security). They are all look- ing for their own bases. "If someone wants to, he can certainly buy our license for the algo- rithm, index the database "VKontakte" and launch a clone FindFace. But it would have to be constantly refined, optimized and updated," says Artem Kukharenka, the creator. He clari- fies that so far none of the clients asked to set up a search on social networks. But police through his facial recognition system have al- ready detained 180 people during the World Cup: 15 of them were in the federal search database, the rest are simply "crooks, pick- pockets and unwanted fans." Engineers in this magical industry of "comput- er vision" promise to soon rid airports and rail- way stations of passport control, to make the city absolutely transparent. But right now fa- cial recognition is a "luxury" commodity that is traded by a narrow circle of inventors. Ordi- nary people have lost access to it - no one makes a second FindFace. Moritz Rakuzzycki of Bellingcat is going to fix it, in part: "Bellingcat intends to independently re-create such a pub- lic search engine, but this time to give access only to journalists and human rights activists, so that the service is not used for bullying. We have already conducted tests on free algo- rithms - they work perfectly. Now it takes time to train them, and money for the equipment. We're looking for an investor right now." Ii How the algorithm works: everything you wanted to ask Where did the hype around recognition technology come from? Haven't the security services had them before? I've seen it on TV shows. The series is lying. In fact, the world turned upside down only five years ago, thanks to the British Jeffrey Hinton. He brought his image analysis program to the Imagenet IT competi- tion at Stanford. To the faces of then it was not yet relevant, the task of the competition was more primitive: on 10 million photos to distin- guish chairs from cats, motorcycles from lawn- mowers, bananas from slugs and so on. If you can do that, you'll learn to distinguish between people. Competitors coped out of hands: all known al- gorithms were wrong at least in a quarter of pictures. No one knew that Hinton was already running his neural network on a regular home computer with two new Nvidia gamer graphics cards. They were perfect for the task: a lot of small calculations fell on a lot of small proces- sors in the graphics card - other algorithms needed whole computing cabinets. Hinton won the competition with an algorithm accuracy of 85%. It was a dramatic leap, the era of neural networks and general hysteria began, technol- ogy began to be used everywhere - from driver- less cars to stock exchanges. Hinton was "snouted" at Google, and mathematician Jan Lekun, who described himself as a "boring neural network" 30 years ago, took to Facebook to create artificial intelligence. One mistake per million - to such accuracy came modern neural networks for facial recog- nition. All the top companies in the world sell- ing this technology meet in the American NIST ranking. There are several nominations in the competition, and "street" recognition condi- tions are called "wild" - "wild." NtechLab (cre- ator of FindFace) regularly wins here. Two oth- er Russian companies, VisionLabs and Vokord, are in the world market leaders. It is the Russ- ian algorithms that have been getting prizes in NIST for the last few years. Only the Chinese compete with them. Why is this happening? Anton Maltsev has been engaged in "machine vision" for ten years. He believes that the answer is simple: "We have a country of barriers. On the surveillance all turned, and besides there is a state order. Although still not as much as in some Chinese provinces. In general, in the world of facial recognition, no one has given up on such a scale." This is true: in England, for example, the state's attempts to use face recognition in football stadiums even face public protests, and in the States the same wave is raised by employees of IT giants like Amazon and Google, who are forced to develop such algo- rithms at the request of the authorities. NEURAL NETWORK is a mathematical decision-making algorithm. The neural network can analyze any images - photos, mu‐ sic, text - and then make an appraisal choice: for ex‐ ample, whether a car should go on red, or which per‐ son is depicted in the picture. How exactly does the neural network see my face? Photography is a rectangle with a bunch of stains and lines. First, the neural network dis- tinguishes from this "informational noise" the person's face, and then looks for the difference between "spots" on one face and "spots" on the other. In the process, some spots can be com- bined to simplify the picture (such neural net- works are called"boring"). The finding is that the neural network is trained by itself, from scratch, by trial and error, like a rat in a labyrinth, which is constantly poking into the walls. With each collision, it slightly "twists" its settings in a random way, and the "rat" turns differently, until finally begins to dodge the walls or learn to choose the right spots in the picture (this is called"reverse error spread"). More faces can be recognized with a 3D scanner (as in an iPhone), but for the flow of people, for example in the subway, it is not suitable, and the existing neural networks for photos or videos are much more accurate. To learn how to recognize faces, neural net- works need an album of photos, where people are already signed by names. The engineer then puts two shots from the base next to her and asks her if it's one (this training method is called Triplet Loss). If the algorithm confuses faces, there is a "rat collision with the wall." The engineer "trains" the neural network in the laboratory, as if preparing a stallion for the next races, driving him on the racetrack. Engineer Anton Maltsev says that Russian pro- grammers fell in love with the VKontakte user avatar database and began to train their neural networks on it: "This was done by most of the now leading facial recognition companies. Pre- viously, VKontakte did not fight this at all (un- like, for example, Facebook). Now Instagram is also being actively pumped out on the market." There is a base - so there is something to train your algorithm. NEURAL NETWORK TRAINING Training the algorithm on large databases of "correct answers." By making mistakes on old data, the neural network itself finds patterns to make the right choice with new data. Still not understood. How does a computer compare a new image to a million others in a second? Did you see Shazam guessing a song in a noisy bar in a couple of seconds? It's exactly the same neural network, it's just been trained on music. And it doesn't compare the sound tracks themselves, nor does it apply the photos to each other's skylights. The neural network distinguishes the face from the stains, and then spits out a number of numbers - barrels in the lottery. What are those numbers? Let's imagine you go down to the subway, and the camera filmed you on the escalator in an uncomfortable angle - at this angle, the inse- cure program gives out a keg Number 8, al- though in the database you have already been recorded as No 10. The neural network got "somewhere near, as if not seeing a couple of letters in your name. It still does not confuse you with the guy on the next step, because his face is a keg number 500, and in this lottery always wins the nearest number to stored in memory. Your "No 8" is closer to No 10. This is not an analogy, so literally it happens: one such number-barrel is called "feature" or "fea- ture". Usually the neural network spits out a series of 128 separate barrel numbers in one portrait to accurately describe your face. This series is also called "feature map", that is, a map of signs, because on it you can even draw on a clean sheet, as if in the game "connect points." n image that the neural network chooses t" and distinctive if it compares a picture ha is a number, and it is not related to "fa‐ " such as the size of the nose or the color Typically, a number of 128 phychs are ne person's face to be recognized from a s. Even if there are a million portraits in the base of the neural network, it does not need to process old images again - each person already corresponds to a set of numbers. It is enough to get the features from a fresh camera shot, send them to the base, and then find the near- est number of numbers from the already saved sets. It is not strering if the numbers do not co- incide absolutely: in fact, the features are vec- tor, that is, "stretchable" values, and their sub- traction is a mathematical operation calledthe Hemming Distancecalculation. This is how all search services work on neural networks: Shazam, FindFace, Google Images and many others. It's not the content itself that com- pares, it's just the rows of numbers assigned to each individual picture. So the features that the neur- al network chooses are the fa- cial parameters? Like the col- or of the eyes and the size of the ears? No, and it's very important to understand. The core of the neural network is almost a black box. Programmers often do not know what signs and details their neural network de- scribes features, that is, chooses "important" for themselves. Through a million attempts, she finds an effective way to compare "spots." The final, "suffering" algorithm in each team is unique. It is kept secret, encrypted and not dis- closed to other companies. In this case, the features for it can be moles on your forehead, the color of a single spot on the picture, the length of the nose or just the brightness of the pixel at a particular point. "Now anyone can take even free neural networks, such as Google, train them in a week and do their project," con- tinues engineer Maltsev. " You can train the neural network in different ways or train it on large databases of very different faces. There- fore, for example, all systems in the market in Russia do not distinguish between Chinese girls." STANCE subtraction of two vectors. In the case of orks, vectors are fitch sets, i.e. rows of t describe one person. Recognition algo‐ ct the features of two different photos to es between them. The smaller the dis‐ mming, the more likely it is that there is n the photos. Iii o earns money m it What are the people selling this technology in Russia? Are they power-related? And why did FindFace be closed? MAXIM PERLIN is a master of scandalous adver- tising. It was thanks to him that NtechLab be- came central to the market. Perlin expected that girls would be bullied when he came up with FindFace: We also came to, when we were just developing the site. However, we thought that the search for prostitutes would be more likely to be shot." Perlin himself suggested Kukharenka to make a public search engine: "Artem then honestly answered that it was some bullshit. I said, "Let me take your tech- nology, I'll do the demo service myself, and you'll have a share there."" As a result, since 2015, NtechLab from six programmers has turned into an international company, began to receive five offers a day, and FindFace - dozens of complaints from victims and even several lawsuits. "I was personally threatened too, but it was rather amusing, mesmerizing, everything was rock and roll," Perlin continues. "I was bathed in the idea that FindFace had be- come a phenomenon, a symbol of the new time, a technology that had been dramatically opened up to society and for which society was not ready at all. Did I feel sorry for these girls? It's a pity, really. It's horrible, humiliating, will haunt them for the rest of their lives. Do I think it's our fault? No, I don't think so." ARTEM KUKHARENKA creates the core, the algo- rithm itself, which sells NtechLab. He is a real scientist, engineer, even the diploma work in MIPT has already written on computer vision. Recently, when NtechLab grew to a cost of hundreds of millions of dollars, Kukharenka gave the post of CEO - just to return to the shop and do less business. He is not interested in politics, but his partners for some reason often turn out to be people, or sympathizers of the Kremlin (the same Perlin spoke at a rally of supporters of Putin, in his office in the agency Blacklight hangs gratitude from the election headquarters, next to the portrait of Napoleon), or literally cooperating with the administrations of the president (more - in a footnote). According to him, the first investors of NtechLab are "one party of acquaintances". Kukharenka believes that it is pointless to lim- it technologies, because they develop indepen- dently of the psychology of society: "If a per- son has something wrong with his head, it does not matter what technologies to give him in his hands, he will do evil. But I can't think of a supervillain that can be done with facial recognition. ders of NtechLab (FindFace) are Alexan‐ Anatoly Gusev and Oleg Bratishko. The e from the influential PR agency "Mikhailov " of TASS CEO Sergey Mikhailov. Earlier, h Alexey Goreslavsky, they founded the One, which won many tenders for PR of ations and presidential elections. then went to the presidential administra‐ e Internet projects.") Alexander Kabakov - vice-president of the Mail.ru Group, his employees of the company jokingly called e AP": "Rollers, stickers, groups in "VKon‐ kov deals with everything that concerns urce told The Bell. The last important co- techLab is Igor Simonov. Political technol‐ an of the board of directors of "Mikhailov ." In 2017, he is a public advisor to the of the AP Sergey Kirienko. n the Cypriot company N-tech.lab ltd re‐ st four co-founders (11.4%) Artem 16.2%) is all hidden under offshore legal Two interlocutors of The Village in the venture market said that Ntechlab investors could de- mand to close the public search engine Find- Face, afraid of problems with personal data and scandal. Kukharenka refutes the fears: the company has never even stored photos of users - only "search indices", and these are just num- bers (sets of features. In addition, NtechLab dis- connects from the customer's system immedi- ately after installation, so that the company does not have access to other people's databas- es. At the same time, Kukharenka does not deny that those who want to re-create an open service may have problems: "Risks with per- sonal data will still be, with ethical issues and interaction with the owners of social networks. If we weigh the pros and cons, it was more profitable for us as an algorithm developer to sell it to another business that already has the