Printed Edition of the Special Issue Published in Entropy Information, Entropy and Their Geometric Structures Edited by Frédéric Barbaresco and Ali Mohammad-Djafari www.mdpi.com/journal/entropy Frédéric Barbaresco and Ali Mohammad-Djafari (Eds.) Information, Entropy and Their Geometric Structures This book is a reprint of the Special Issue that appeared in the online, open access journal, Entropy (ISSN 1099-4300) from 2014–2015 (available at: http://www.mdpi.com/journal/entropy/special_issues/entropy-Geome). Guest Editors Frédéric Barbaresco Advanced Radar Concepts Business Unit Thales Air Systems S.A., Voie Pierre-Gilles de Gennes F91470 Limours France Ali Mohammad-Djafari Laboratoire des Signaux et Systmes UMR 8506 CNRS-SUPELEC-UNIV PARIS SUD Gif-sur-Yvette France Editorial Office MDPI AG Klybeckstrasse 64 Basel, Switzerland Publisher Shu-Kun Lin Managing Editor Jely He 1. Edition 2015 MDPI • Basel • Beijing • Wuhan • Barcelona ISBN 978-3-03842-103-0 (Hbk) ISBN 978-3-03842-104-7 (PDF) Articles in this volume are Open Access and distributed under the Creative Commons Attribution license (CC BY), which allows users to download, copy and build upon published articles even for commercial purposes, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications. The book taken as a whole is © 2015 MDPI, Basel, Switzerland, distributed under the terms and conditions of the Creative Commons by Attribution (CC BY-NC-ND) license (http://creativecommons.org/licenses/by-nc-nd/4.0/). III Table of Contents List of Contributors ............................................................................................................ VII About the Guest Editors....................................................................................................... IX Preface From Information Theory to Geometric Science of Information ........................................... XI Chapter 1: Origins of Entropy and Information Theory Stefano Bordoni J.J. Thomson and Duhem’s Lagrangian Approaches to Thermodynamics Reprinted from: Entropy , 16 (11), 5876 ± 5890 http://www.mdpi.com/1099-4300/16/11/5876 ......................................................................... 3 Olivier Rioul and José Carlos Magossi On Shannon’s Formula and Hartley’s Rule: Beyond the Mathematical Coincidence Reprinted from: Entropy , 16 (9), 4892 ± 4910 http://www.mdpi.com/1099-4300/16/9/4892 .........................................................................18 Chapter 2: Mathematical and Physical Foundations of Information and Entropy Geometric Structures Misha Gromov Symmetry, Probabiliy, Entropy: Synopsis of the Lecture at MAXENT 2014 Reprinted from: Entropy 2015 , 17 (3), 1273 ± 1277 http://www.mdpi.com/1099-4300/17/3/1273 .........................................................................39 Pierre Baudot and Daniel Bennequin The Homological Nature of Entropy Reprinted from: Entropy , 17 (5), 3253 ± 3318 http://www.mdpi.com/1099-4300/17/5/3253 .........................................................................43 Nina Miolane and Xavier Pennec Computing Bi-Invariant Pseudo-Metrics on Lie Groups for Consistent Statistics Reprinted from: Entropy 2015 , 17 (4), 1850 ± 1881 http://www.mdpi.com/1099-4300/17/4/1850.......................................................................112 IV Frédéric Barbaresco Koszul Information Geometry and Souriau Geometric Temperature/Capacity of Lie Group Thermodynamics Reprinted from: Entropy 2014 , 16 (8), 4521 ± 4565 http://www.mdpi.com/1099-4300/16/8/4521 .......................................................................146 Roger Balian The Entropy-Based Quantum Metric Reprinted from: Entropy 2014 , 16 (7), 3878 ± 3888 http://www.mdpi.com/1099-4300/16/7/3878 .......................................................................193 Mitsuhiro Itoh and Hiroyasu Satoh Geometry of Fisher Information Metric and the Barycenter Map Reprinted from: Entropy 2015 , 17 (4), 1814 ± 1849 http://www.mdpi.com/1099-4300/17/4/1814 .......................................................................204 Chapter 3: Applications of Information/Entropy Geometric Structures Ali Mohammad-Djafari Entropy, Information Theory, Information Geometry and Bayesian Inference in Data, Signal and Image Processing and Inverse Problems Reprinted from: Entropy 2015 , 17 (6), 3989 ± 4027 http://www.mdpi.com/1099-4300/17/6/3989 .......................................................................243 Jérémy Bensadon Black-Box Optimization Using Geodesics in Statistical Manifolds Reprinted from: Entropy 2015 , 17 (1), 304 ± 345 http://www.mdpi.com/1099-4300/17/1/304 .........................................................................284 Luigi Malagò and Giovanni Pistone Natural Gradient Flow in the Mixture Geometry of a Discrete Exponential Family Reprinted from: Entropy 2015 , 17 (6), 4215 ± 4254 http://www.mdpi.com/1099-4300/17/6/4215 .......................................................................328 Anass Bellachehab Distributed Consensus for Metamorphic Systems Using a Gossip Algorithm for CAT (0) Metric Spaces Reprinted from: Entropy 2015 , 17 (3), 1165 ± 1180 http://www.mdpi.com/1099-4300/17/3/1165 .......................................................................369 Jaehyung Choi and Andrew P. Mullhaupt Geometric Shrinkage Priors for Kählerian Signal Filters Reprinted from: Entropy 2015 , 17 (3), 1347 ± 1357 http://www.mdpi.com/1099-4300/17/3/1347 .......................................................................385 V Jaehyung Choi and Andrew P. Mullhaupt Kählerian Information Geometry for Signal Processing Reprinted from: Entropy 2015 , 17 (4), 1581 ± 1605 http://www.mdpi.com/1099-4300/17/4/1581 .......................................................................396 Youssef Bennani, Luc Pronzato and Maria João Rendas Most Likely Maximum Entropy for Population Analysis with Region-Censored Data Reprinted from: Entropy 2015 , 17 (6), 3963 ± 3988 http://www.mdpi.com/1099-4300/17/6/3963 .......................................................................422 Udo von Toussaint General Hyperplane Prior Distributions Based on Geometric Invariances for Bayesian Multivariate Linear Regression Reprinted from: Entropy 2015 , 17 (6), 3898 ± 3912 http://www.mdpi.com/1099-4300/17/6/3898 .......................................................................448 Geert Verdoolaege A New Robust Regression Method Based on Minimization of Geodesic Distances on a Probabilistic Manifold: Application to Power Laws Reprinted from: Entropy 2015 , 17 (7), 4602 ± 4626 http://www.mdpi.com/1099-4300/17/7/4602 .......................................................................463 Jun Zhang On Monotone Embedding in Information Geometry Reprinted from: Entropy 2015 , 17 (7), 4485 ± 4499 http://www.mdpi.com/1099-4300/17/7/4485 .......................................................................488 Takashi Takenouchi, Osamu Komori and Shinto Eguchi Binary Classification with a Pseudo Exponential Model and Its Application for Multi-Task Learning Reprinted from: Entropy 2015 , 17 (8), 5673 ± 5694 http://www.mdpi.com/1099-4300/17/8/5673 .......................................................................504 VII List of Contributors Roger Balian: Institut de Physique Théorique, CEA/Saclay, F-91191 Gif-sur-Yvette Cedex, France Frédéric Barbaresco: Thales Air Systems, Advanced Radar Concepts Business Unit, Voie Pierre-Gilles de Gennes, Limours F-91470, France Pierre Baudot: Max Planck Institute for Mathematics in the Sciences, Inselstrasse 22, 04103 Leipzig, Germany Anass Bellachehab: Telecom SudParis, Institut Mines-Télécom, UMR CNRS 5157 SAMOVAR, 9 Rue Charles Fourier, 91000 Évry, France Youssef Bennani: CNRS, Laboratoire I3S-UMR 7271, Université de Nice-Sophia Antipolis/CNRS, 06900 Sophia Antipolis, France Daniel Bennequin: Universite Paris Diderot-Paris 7, UFR de Mathematiques, Equipe Geometrie et Dynamique, Batiment Sophie Germain, 5 rue Thomas Mann, 75205 Paris Cedex 13, France Jérémy Bensadon: Laboratoire de Recherche en Informatique, Université Paris-Sud, 91400 Orsay, France Stefano Bordoni: Department of Pharmacy and Biotechnology, University of Bologna — Rimini Campus, Via Dei Mille 39 — 47921 Rimini, Italy Jaehyung Choi: Department of Applied Mathematics and Statistics, State University of New York (SUNY), StonyBrook, NY 11794, USA Shinto Eguchi: The Institute of Statistical Mathematics, 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan Misha Gromov: Institut Hautes Études Scientifiques, 35, Route de Chartres, F-91440 Bures-sur-Yvette, France Mitsuhiro Itoh: Institute of Mathematics, University of Tsukuba, 1-1-1, Ten-noudai, Tsukuba, 305-8571, Japan Osamu Komori: The Institute of Statistical Mathematics, 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan José Carlos Magossi: School of Technology (FT), University of Campinas (Unicamp), Rua Paschoal Marmo 1.888, 13484-370 Limeira, São Paulo, Brazil Luigi Malagò: Department of Electrical and Electronic Engineering, Shinshu University, Nagano, Japan; Inria Saclay, Île-de-France, Orsay Cedex, France Nina Miolane: INRIA, Asclepios project-team, 2004 Route des Lucioles, BP93, Sophia Antipolis Cedex F-06902, France Ali Mohammad-Djafari: Laboratoire des Signaux et Systèmes, UMR 8506 CNRS-SUPELEC-UNIV PARIS SUD, SUPELEC, Plateau de Moulon, 3 rue Juliot-Curie, 91192 Gif-sur-Yvette, France Andrew P. Mullhaupt: Department of Applied Mathematics and Statistics, State University of New York (SUNY), StonyBrook, NY 11794, USA Xavier Pennec: INRIA, Asclepios project-team, 2004 Route des Lucioles, BP93, Sophia Antipolis Cedex F-06902, France VIII Giovanni Pistone: De Castro Statistics, Collegio Carlo Alberto, Moncalieri, Italy Luc Pronzato: CNRS, Laboratoire I3S-UMR 7271, Université de Nice-Sophia Antipolis/CNRS, 06900 Sophia Antipolis, France Maria João Rendas: CNRS, Laboratoire I3S-UMR 7271, Université de Nice-Sophia Antipolis/CNRS, 06900 Sophia Antipolis, France Olivier Rioul: Télécom ParisTech, Institut Mines-Télécom, CNRS LTCI, 46 Rue Barrault, 75013, Paris, France Hiroyasu Satoh: Nippon Institute of Technology, Saitama, 345-8501, Japan Takashi Takenouchi: Future University Hakodate, 116-2 Kamedanakano, Hakodate Hokkaido 041-8655, Japan Geert Verdoolaege: Department of Applied Physics, Ghent University, Sint-Pietersnieuwstraat 41, B-9000 Ghent, Belgium; Laboratory for Plasma Physics — Royal Military Academy (LPP-ERM/KMS), Avenue de la Renaissancelaan 30, B-1000 Brussels, Belgium Udo von Toussaint: Max-Planck-Institute for Plasmaphysics, Boltzmannstrasse 2, 85748 Garching, Germany Jun Zhang: Department of Psychology and Department of Mathematics, University of Michigan, 530 Church Street, Ann Arbor, MI 48109, USA IX About the Guest Editors Frédéric Barbaresco received his State Engineering degree from the French Grand Ecole SUPELEC, Paris, France, in 1991. Since then, he has worked for the THALES Group where he is now Senior Scientist and Advanced Studies Manager in the Advanced Radar Concept Business Unit of THALES AIR SYSTEMS. He has been an Emeritus Member of SEE since 2011 and he was awarded the Aymé Poirson Prize (for application of sciences to industry) by the French Academy of Sciences in 2014, the SEE Ampere Medal in 2007, the Thévenin Prize in 2014 and the NATO SET Lecture Award in 2012. He is President of SEE Tech nical Club ISIC “Engineering of Information and Communications Systems” (https://www.see.asso.fr/ct-isic) and a member of the SEE administrative board. In 2009, he founded the Leon Brillouin Seminar on “ Geometric Sciences of Information ” (http://repmus.ircam.fr/brillouin/home) hosted by IRCAM in Paris, France. He has organized a tutorial on “ Modern Radar Processing based on Geometry of Structured Matrices and Information Geometry ” (http://www.radar2014.org/tutorials/98#TUTORIAL1) for the Radar’14 Conferenc e, and a Short Course on “ Geometric Radar Processing Breakthrough based on Fisher Information Geometry ” (http://www.eumweek.com/docs/workshops/SCW01.pdf) for the EuRAD’15 Conference. He was co-organizer of the French- Indian MIG’11 Workshop at Ecole Polytec hnique and Thales Research and Technology (https://www.sonycsl.co.jp/person/nielsen/infogeo/MIG/) and a coordinator of Springer Lecture Notes on “ Matrix Information Geometry ” published in 2012 (http://www.springer.com/us/book/9783642302312). He was an invited speaker at the SMC-NEGAA “ Structured Matrix Computations in Non Euclidean Geometries ” workshop (http://www-lmpa.univ-littoral.fr/SMC-NEGAA2012/participants.html) at CIRM Luminy in 2012. In 2012 he organized a Symposium at Institut Henri Poincaré on “ Optimal Transport and Information Geometry ” (https://www.ceremade.dauphine.fr/~peyre/mspc/mspc-thales-12/). He was an invited lecturer for UNESCO on “ Advanced School and Workshop on Matrix Geometries and Applications ” in Trieste (http://indico.ictp.it/event/a12193) at the ITCP (International Center for Theoretical Physics) in June 2013. He is the General Co-chairman of the new international conference GSI “ Geometric Sciences of Information ”, first edition GSI’13 at Ecole des Mines in 2013 (https://www.see.ass o.fr/gsi2013) and 2nd edition GSI’15 at Ecole Polytechnique in 2015 (https://www.see.asso.fr/gsi2015). He is the co- editor of “ Geometric Science of Information ” (http://www.springer.com/us/book/9783642400193), a book published by Springer in 2013, and one of the contributors to the Springer book “ Geometric Theory of Information ” (http://www.springer.com/us/book/9783319053165) published in 2014. He co-organized MaxEnt’14 conference in Amboise in 2014 (https://www.see.asso.fr/maxent14) and is the editor of Pr oceedings published by the AIP “American Institute of Physics” (http://scitation.aip.org/content/aip/proceeding/aipcp/1641). He was invited to be a keynote speaker for the GIO’14 workshop in Bordeaux on “ Geometry of Information and Optimization ” (https://sites.google.com/site/bordeauxgio2014/). X Ali Mohammad-Djafari received his B.Sc. degree in electrical engineering from the Polytechnic of Teheran, in 1975, a diploma degree (M.Sc.) from Ecole Supérieure d'Electricit (SUPELEC), Gif sur Yvette, France, in 1 977, a “Docteur - Ingénieur” (Ph.D.) degree and “Doctorat d'Etat” in Physics, from the University of Paris Sud 11 (UPS), Orsay, France, respectively in 1981 and 1987. He was Assistant Professor at UPS for two years (1981 – 1983). Since 1984, he has held a permanent position at “Centre National de la Recherche Scientifique (CNRS)” and works at “Laboratoire des signaux et systèmes (L2S)” at SUPELEC. He was a visiting Associ ate Professor at the University of Notre Dame, Indiana, USA, from 1997 – 1998. From 1998 to 2002, he has been the Head of Signal and Image Processing Division at this laboratory. Presently, he is the “Directeur de recherche” and his main scientific interes ts are in developing new probabilistic methods based on Bayesian inference, information theory and maximum entropy approaches for inverse problems in general, in all aspects of data processing, and, more specifically, in imaging and vision systems: image reconstruction; signal and image deconvolution; blind source separation; sources localization; data fusion; multi and hyper spectral image segmentation. The main application domains of his interests are medical imaging, computed tomography (X rays, PET, SPECT, MRI, microwave, ultrasound and eddy current imaging) either for medical imaging or for non-destructive testing (NDT) in industry, multivariate and multi-dimensional data, signal and image processing, data mining, clustering, classification and machine learning methods for biological or medical applications. He has supervised over 20 Ph.D. thesis’, 20 post -doc research activities and 50 M.Sc. student research projects. He has published over 50 full journal papers, 10 book and proceedings editions and more than 200 papers in national and international conferences. He has organized or co-organized more than 10 international workshops and conferences. He has been further been the expert and consultant for a great number of French national and international projects. Since 1988 he has held many teaching activities as Professor in M.Sc. and Ph.D. Level in SUPELEC, University of Paris Sud (UPS), ENSTA and Ecole Centrale de Paris (ECP). He also participated and managed industrial contracts with many French national industries such as EDF, RENAUL, PEUGEOT, THALES, SAFRAN, CARESTREAM and great research institutions such as CEA, INSERM, INRIA, ONERA, as well as regional (such as Digiteo), national (such as ANR) and European projects (such as ERASYSBIO). For an overview and access to more details of his activities, please see his web page: http://djafari.free.fr for general information; http://djafari.free.fr/news.htm for news and activities; and http://publicationslist.org/djafarie for the list of publications. XI Preface From Information Theory to Geometric Science of Information Venus at the Forge of Vulcan, Le Nain Brothers, Musée Saint-Denis, Reims (Vulcan is the god of fire and god of metalworking and the forge, often depicted with a blacksmith’s hammer) “ Intelligence is the faculty of manufacturing artificial objects, especially tools to make tools, and of indefinitely varying the manufacture.”— Henri Bergson XII Information theory was founded in the 1950s based on the work of Claude Shannon and Jacques Laplume in communication and Léon Brillouin in statistical physics, among other main contributors. These foundations have conventionally been built on linear algebra theory and probability models in conventional spaces (vector space, normed spaces, ...). At the turn of the century, new and fruitful interactions were found between several branches of science: Information Science (information theory, digital communications, statistical signal processing, ...), Mathematics (group theory, geometry and topology, probability, statistics, ...) and Physical Sciences (geometric mechanics, thermodynamics, statistical physics, quantum mechanics, ...). From Probability to Geometry The probability theory was conceived by Blaise Pascal and Jacob Bernoulli. Pierre de Fermat also helped in his exchange of correspondence with Blaise Pascal to develop the foundations of probability theory, a mathematical accident that caused the study of Chevalier de Méré’s game (Antoine Gombaud, Chevalier de Méré, a French nobleman with an interest in gaming and gambling questions, called Pascal’s attention to an apparent contradiction concerning a popular dice game). Then, probability theory was consolidated by many contributors, such as Pierre Simon Laplace, Abraham de Moivre and Carl Friedrich Gauss during the XVIII century and by Emile Borel, Andreï Kolmogorov and Paul Levy last century. Probability is again the subject of a new foundation to apprehend new structures and generalize the theory to more abstract spaces (metric spaces, homogeneous manifolds, graphs ....). A first attempt at probability generalization in metric spaces was developed by Maurice Fréchet in the middle of last century, in the framework of abstract spaces topologically affine and “distance space” (“espace distancié”) with triangular inequality constraint. What’s new since 1950s INFORMATION THEORY Claude Shannon MIT Léon Brillouin Collège de France Jacques Laplume Radar Dept. Thomson-Houston XIII From Statistics to Geometry In the middle of last century, another branch of geometric approaches of statistical problems has been initiated by Calyampudi Radhakrishna Rao that introduced a metric space in the parameters space of probability densities. The metric tensor was proved to be equal to the Fisher Information matrix. This result was axiomatized by Nikolai Nikolaevich Chentsov in the framework of category theory. Having been introduced in 1939, the lower bound in statistics, six years before C.R. Rao, this idea was latent in the work of Maurice Fréchet, who had noticed that the “ distinguished densities ” that reach this lower bound are defined by a function that is given by a solution of Legendre-Clairaut equation. Nowadays, this Legendre-Clairaut equation is the cornerstone of “Information Geometry” theory linking two dual potential functions in dual spaces. In parallel, Jean-Louis Koszul had constructed a Hessian geometry on convex cones, through the concept of Koszul-Vinberg characteristic function and Koszul forms. Koszul Information Geometry is a generalization of information geometry theory, where invariance with respect to densities parameters is replaced by invariance with respect to automorphisms of these convex cones where these parameters lie. In 1957, the framework was consolidated by the principle of Maximum Entropy, expounded by E. T. Jaynes in two papers where he emphasized a natural correspondence between statistical mechanics and information theory. In particular, Jaynes offered a link to statistical physics and a rationale as to why the Gibbsian method of statistical mechanics works. He argued that the entropy of statistical mechanics and the information entropy of information theory are principally the same thing. XIV Consequently, statistical mechanics should be seen just as a particular application of a general tool of logical inference and information theory. From Thermodynamics to Geometry On the side of statistical physics and thermodynamics — which were based on the seminal works of Sadi Carnot, Rudolf Clausius, Ludwig Boltzmann, François Massieu and Williard Gibbs — several geometric attempts were developed later as a “general equation of thermodynamics” by Pierre Duhem unifying in the same equations all changes of systems’ positions and states. In his 1891 Paper, « Sur les équations générales de la Thermodynamique », Pierre Duhem wrote “ We made a special case, the dynamics of thermodynamics, a science that embraces common principles in all the changes of state of the bodies, both changes of places and changes in physical qualities. ” Four scientists were credited by Duhem with having carried out the most important researches on that subject: François Massieu to derive thermodynamics from a characteristic function and its partial derivatives; J. W. Gibbs to show that Ma ssieu’s functions could play the role of potentials in the determination of the states of equilibrium in a given system; H. von Helmholtz to put forward similar ideas (and analogy between thermodynamic and mechanics); and A. von Oettingen to give an exposition of thermodynamics of remarkable generality based on the general duality concept. More recently, we can make references to the “Lie Group thermodynamics” theory created by Jean -Marie Souriau in the framework of geometric mechanics and symplectic geometry, or the concept of thermodynamics contact manifolds that was conceptualized by Vladimir Arnold. Thi geometrization of thermodynamics and mechanics was also extended to quantum mechanics by Roger Balian, providing also a bridge with information geometry. Roger Balian, in 1986, introduced a geometric structure through extension of the Fisher metric in statistical physics and quantum mechanics, compatible with gauge theory of thermodynamics. From Mechanics to Geometry The last branch of geometric structure elaboration for information is emerging through the inter- relations between “geometric mechanics” and the “geometric science of information”, that will be largely debated at the GSI’15 conference (www.gsi2015.org). We can imagine that other links could be discovered between mechanics and geometry, for instance based on the elastic theory of the Cosserat brothers that should enlighten new seminal works as discovered by Jean-François Pommaret. In 1926, Louis-Maurice R oy, published in Annals, “a thermodynamic theory of elastic line [ ...]”, directly inspired by the relatively recent work of Duhem and M. M. Cosserat. This idea was also developed in Louis de Broglie’s book on thermodynamics. Regarding geometry and mechanics, for the anecdote, we can observe that the master of geometry during the last century, Elie Cartan, was the son of Joseph Cartan who was the village blacksmith, and Elie recalled that his childhood had passed under “blows of the anvil, which started every morning from dawn”. We can imagine easily that the child, Elie Cartan, watching his father Joseph “coding curvature” on metal between the hammer and the anvil, insidiously influencing Elie’s mind with germinal intuition of fundamental geometric concepts. T he alliance of geometry XV and mechanics is beautifully given by this image of Forge, as illustrated in this painting of Velasquez about Vulcan God. This concordance of meaning is confirmed by the etymology of the word “Forge”, that comes from the late XIV century, “a smithy”, from Old French forge “forge, smithy” (XII century), earlier faverge, from Latin fabrica “workshop, smith’s shop”, from faber (genitive fabri) “workman in hard materials, smith”. One can imagine the hammer blows given by Joseph on the anvil, giving shape and curvature to the metal, inspired the curious mind of Elie that surely inspired later his intuition of “moving frame” and “nonholonomic space” in geometry. Elie Cartan was motivated by the objective to build new foundations of geometry . He said “ distinguished service that has rendered and will make even the absolute differential calculus of Ricci and Levi-Civita should not prevent us to avoid too exclusively formal calculations, where debauchery indices often mask a very simple geometric fact. It is this reality that I have sought to put in evidence everywhere. ” (« Les services éminents qu’a rendus et que rendra encore le Calcul différentiel absolu de Ricci et Levi- Civita ne doivent pas nous empêcher d’éviter les calculs trop exclusiveme nt formels, où les débauches d’indices masquent une réalité géométrique souvent très simple. C’est cette réalité que j’ai cherché à mettre partout en évidence. » in É. Cartan, Leçons sur la théorie des espaces de Riemann, Paris: Gauthier-Villars, 2e éd., 1946, p. VII). Into the Flaming Forge of Vulcan, into the Ninth Sphere, Mars descends in order to retemper his flaming sword and conquer the heart of Venus (Diego Velázquez, Museo Nacional del Prado) XVI Groups Everywhere and Metrics Everywhere Geometric structure can also be considered through group theory. As observed by Gaston Bachelard, mathematical physics, incorporating at its core the concept of group, brand supremacy. All rational geometries, and without doubt more generally all mathematical organizations of experience, are characterized by a special group of transformations. The group provides evidence of mathematics closed on itself. Its discovery closes the era of conventions, more or less independent, more or less coherent. Henri Poincaré said that if we strip the mathematical theory of which appears to be an accident, that is to say its material, there will remain only the essential, that is to say, the form; and this form, which is as it were the solid skeleton of the theory, will be the group’s structure. Concerning Elie Cartan’s work, Henri Poincaré said that “ the problems addressed by Elie Cartan are among the most important, most abstract and most general dealing with mathematics; group theory is, so to speak, the whole mathematics, stripped of its material and reduced to pure form. This extreme level of abstraction has probably made my presentation a little dry; to assess each of the results, I would have had virtually render him the material which he had been stripped; but this refund can be made in a thousand different ways; and this is the only form that can be found as well as a host of various garments, which is the common link between mathematical theories that are often surprised to find so near ”. “Groups everywhere” and “metrics everywhere” are then the new leitmotiv in mathematics and physics. In particular, a central role could be attributed to Misha Gromov and his contribution to metric spaces. The analysis of the invariants and the transformations preserving them is at the core of Gr omov’s work on “geometrical group theory”. XVII Entropy Everywhere From its beginnings, the theory of information has also been linked to statistical physics through the concept of entropy. This intimate relationship between information and entropy was studied by Léon Brillouin and Claude Shannon. The latter writes: “ My biggest concern was what to call it. I thought to call information, but the term was used too, so I decided to call it uncertainty. When I was talking with John von Neumann, he had a better idea. He said to me, you should call it entropy, for two reasons. First, your uncertainty function was used in statistical mechanics under that name, so it already has a name. Second, and most important, no one really knows what the entropy, so in a debate you would always have the advantage ” . René Thom also tried to show in which direction a real information theory could go, being halfway between semantics and semiotics, thermodynamics of real forms, would attempt to return a proper analysis of morphological forms of messages. Linguistics Everywhere To conclude this preface, if we go back further in history, let’s look at the etymological origins of the word “information”. First written as “enformer” the word “inform” appears in French in 1286, the Latin word “informare”, literally “shape”. The word “information” appears in the XIII century. From the Greek etymology, ȝȠȡijȒ , morphs (“shape”), we reached the sense of morphology, the science of forms. For Plato, the concept of “form” is designated “morph”, “Eidos” and “idea”; Henri Bergson gave his definition of the Greek concept of “Eidos” in the book “Creative Evolution”: “ The word, eidos, which we translate here by “Idea”, has, in fact, this threefold meaning. It denotes (1) the quality, (2) the form or essence, (3) the end or design (in the sense of intention) of the act being performed, that is to say, at bottom, the design (in the sense of drawing) of the act supposed accomplished .” These three aspects are those of the adjective, substantive and verb, an d correspond to the three essential categories of language, proving, as Jean-Marie Souriau did, that we have to apprehend “the grammar of nature”. XVIII Geometric Science of Information as a Federative Structure and Grammar Henri Poincaré said that “Mathematics is the art of giving the same name to different things” (« La mathématique est l'art de donner le même nom à des choses différentes» in «Science et méthode”, 1908). By paraphrasing Henri Poincaré, we could claim that « Geometric Science of Information » is the art of giving the same name to different sciences. The rules, the structure and architecture of this new “manufacture” is a kind of new Grammar for Sciences. Book Chapters Survey The aim of this book is to provide an overview of current work addressing this topic of research that explores the geometric structures of information and entropy. These papers are an extended version of the paper published in Proceedings (http://printorders.aip.org/proceedings/1641) of the 34th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering (MaxEnt 2014), Amboise, France, 21 – 26 September 2014 (https://www.see.asso.fr/maxent14). Chapter 1 of the book is a historical review of the origins of thermodynamics and information theory: x Stefano Bordoni analyses and puts in perspective crosswise the work of J.J. Thomson and P. Duhem in thermodynamics in recent decades of the nineteenth century, with two abstract and phenomenological approaches to thermodynamics. After the analysis of intermediate solutions by Helmholtz, Planck and Oettingen, he describes J.J. Thomson’s general theory for physical and chemical processes, and P. Duhem’s design of energetics as unification between physics and chemistry. Lie Groups, Geometry & Topology Information Theory Geometric Mechanics & Quantum Physics Statistical Physics & Thermodynamics Probability & Statistics Geometric Science of Information XIX x Olivier Rioul and José Carlos Magossi then detail the history of the discovery of the Shannon’s formula for Gaussian channel, with seminal Hartley’s rule twenty years before Shannon for uniform channel, and the first published work in April 1948 of the French engineer, Jacques Laplume from Radar/Hyper Department of Thomson-Houston, among others. Chapter 2 discusses the mathematical and physical foundations of geometric structures related to information and entropy: x Misha Gromov, IHES (Institute of Advanced Scientific Studies), Abel Prize 2009, indicates possibilities for (homological and non-homological) linearization of basic notions of the probability theory and also the replacement of the real numbers as values of probabilities by objects of suitable combinatorial categories. x Pierre Baudot from Max-Planck Institute, and Daniel Bennequin from Institu mathématique de Jussieu, observe that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. This gives rise to a new kind of topology for information processes that accounts for the main information function. x Nina Miolane and Xavier Pennec compute bi-Invariant pseudo-Metrics on Lie Groups for consistent statistics to define a Riemannian metric compatible with the group structure, to perform statistics on Lie groups for computational anatomy. x Frédéric Barbaresco introduces the Symplectic Structure of Information Geometry based on Souriau’s “Lie Group Thermodynamics model”, with a covariant definition of Gibbs equilibrium via invariances through co-adjoint action of a group on its momentum space. The Fisher metric is identified as a Souriau Geometric Heat Capacity. This model is compared with hessian Geometry of Jean-Louis Koszul, which is the main pillar of Information Geometry theory. x Roger Balian introduces in the space of quantum density matrices, a Riemann metric as hessian of the von Neumann entropy, which is physically founded and which characterizes the amount of quantum information lost, underlying the canonical mapping between the spaces of states and of observables, which involves the Legendre transform. Roger Balian provides then its general expression and its explicit form for q-bits. x Mitsuhiro Itoh and Hiroyasu Satoh study the geometry of Fisher metrics and geodesics on a space of probability measures defined on a compact manifold and its application to geometry of a barycenter map associated with Busemann function on a Hadamard manifold X. They describe a fibre space structure of barycenter map.